What’s new in Swift 5.9?

Macros, if and switch expressions, noncopyable types, and more!

Paul Hudson    June 16th 2023    @twostraws

Although Swift 6 is looming on the horizon, the 5.x releases still have a lot to give – simpler ways to use if and switch, macros, noncopyable types, custom actor executors, and more are all coming in Swift 5.9, making yet another mammoth release.

In this article I’ll walk you through the most important changes in this release, providing code examples and explanations so you can try it all yourself. You’ll need the latest Swift 5.9 toolchain installed in Xcode 14, or the Xcode 15 beta.

Sponsor Hacking with Swift and reach the world's largest Swift community!

if and switch expressions

SE-0380 adds the ability for us to use if and switch as expressions in several situations. This produces syntax that will be a little surprising at first, but overall it does help reduce a little extra syntax in the language.

As a simple example, we could set a variable to either “Pass” or “Fail” depending on a condition like this:

let score = 800
let simpleResult = if score > 500 { "Pass" } else { "Fail" }
print(simpleResult)

Or we could use a switch expression to get a wider range of values like this:

let complexResult = switch score {
    case 0...300: "Fail"
    case 301...500: "Pass"
    case 501...800: "Merit"
    default: "Distinction"
}

print(complexResult)

You don’t need to assign the result somewhere in order to use this new expression syntax, and in fact it combines beautifully with SE-0255 from Swift 5.1 that allows us to omit the return keyword in single expression functions that return a value.

So, because both if and switch can now both be used as expressions, we can write a function like this one without using return in all four possible cases:

func rating(for score: Int) -> String {
    switch score {
    case 0...300: "Fail"
    case 301...500: "Pass"
    case 501...800: "Merit"
    default: "Distinction"
    }
}

print(rating(for: score))

You might be thinking this feature makes if work more like the ternary conditional operator, and you’d be at least partly right. For example, we could have written our simple if condition from earlier like this:

let ternaryResult = score > 500 ? "Pass" : "Fail"
print(ternaryResult)

However, the two are not identical, and there is one place in particular that might catch you out – you can see it in this code:

let customerRating = 4
let bonusMultiplier1 = customerRating > 3 ? 1.5 : 1
let bonusMultiplier2 = if customerRating > 3 { 1.5 } else { 1.0 }

Both those calculations produce a Double with the value of 1.5, but pay attention to the alternative value for each of them: for the ternary option I’ve written 1, and for the if expression I’ve written 1.0.

This is intentional: when using the ternary Swift checks the types of both values at the same time and so automatically considers 1 to be 1.0, whereas with the if expression the two options are type checked independently: if we use 1.5 for one case and 1 for the other then we’ll be sending back a Double and an Int, which isn’t allowed.

Value and Type Parameter Packs

SE-0393, SE-0398, and SE-0399 combined to form a rather dense knot of improvements to Swift that allow us to use variadic generics.

This is a fairly advanced feature, so let me sum it up in a way that will make most people listen: this almost certainly means the old 10-view limit in SwiftUI is about to disappear. We’ll only know tomorrow once everything is public, but I’d be genuinely astonished if the issue remains in iOS 17.

Still here? Okay. These proposals solve a significant problem in Swift, which is that generic functions required a specific number of type parameters. These functions could still accept variadic parameters, but they still had to use the same type ultimately.

As an example, we could have three different structs that represent different parts of our program:

struct FrontEndDev {
    var name: String
}

struct BackEndDev {
    var name: String
}

struct FullStackDev {
    var name: String
}

In practice they would have lots more properties that make those types unique, but you get the point – three different types exist.

We could make instances of those structs like this:

let johnny = FrontEndDev(name: "Johnny Appleseed")
let jess = FrontEndDev(name: "Jessica Appleseed")
let kate = BackEndDev(name: "Kate Bell")
let kevin = BackEndDev(name: "Kevin Bell")

let derek = FullStackDev(name: "Derek Derekson")

And then when it came to actually doing work, we could pair developers together using a simple function like this one:

func pairUp1<T, U>(firstPeople: T..., secondPeople: U...) -> ([(T, U)]) {
    assert(firstPeople.count == secondPeople.count, "You must provide equal numbers of people to pair.")
    var result = [(T, U)]()

    for i in 0..<firstPeople.count {
        result.append((firstPeople[i], secondPeople[i]))
    }

    return result
}

That uses two variadic parameters to receive a group of first people and a group of second people, then returns them as an array.

We can now use that to create programmer pairs who can work on some back-end and front-end work together:

let result1 = pairUp1(firstPeople: johnny, jess, secondPeople: kate, kevin)

So far this is old, but here’s where things get interesting: Derek is a full-stack developer, and can therefore work as either a back-end developer or a front-end developer. However, if we tried to use johnny, derek as the first parameter then Swift would refuse to build our code – it needs the types of all the first people and second people to be the same.

One way to fix this would be to throw away all our type information using Any, but parameter packs allow us to solve this much more elegantly.

The syntax might be a little intense at first, so I’m going to show you the code then try to break it down. Here it is:

func pairUp2<each T, each U>(firstPeople: repeat each T, secondPeople: repeat each U) -> (repeat (first: each T, second: each U)) {
    return (repeat (each firstPeople, each secondPeople))
}

There are four independent things happening there, so let’s work through them one by one:

1. <each T, each U> creates two type parameter packs, T and U.
2. repeat each T is a pack expansion, which is what expands the parameter pack into actual values – it’s the equivalent of T..., but avoids some confusion with ... being used as an operator.
3. The return type means we’re sending back tuples of paired programmers, one each from T and U.
4. Our return keyword is what does the real work: it uses a pack expansion expression to take one value from T and one from U, putting them together into the returned value.

What it doesn’t show is that the return type automatically ensures both our T and U types have the same shape – they have the same number of items inside them. So, rather than using assert() like we had in the first function, Swift will simply issue a compiler error if we try to pass in two sets of data of different sizes.

With the new function in place, we can now pair up Derek with other developers, like this:

let result2 = pairUp2(firstPeople: johnny, derek, secondPeople: kate, kevin)

Now, what we’ve actually done is implement a simple zip() function, which means we can write nonsense like this:

let result3 = pairUp2(firstPeople: johnny, derek, secondPeople: kate, 556)

That tries to pair Kevin with the number 556, which clearly doesn’t make any sense. This is where parameter packs really come into their own, because we could define protocols such as these:

protocol WritesFrontEndCode { }
protocol WritesBackEndCode { }

Then add some conformances:

• FrontEndDev should conform to WritesFrontEndCode
• BackEndDev should conform to WritesBackEndCode
• FullStackDev should conform to both WritesFrontEndCode and WritesBackEndCode

And now we can add constraints to our type parameter packs:

func pairUp3<each T: WritesFrontEndCode, each U: WritesBackEndCode>(firstPeople: repeat each T, secondPeople: repeat each U) -> (repeat (first: each T, second: each U)) {
    return (repeat (each firstPeople, each secondPeople))
}

That now means only sensible pairs can happen – we always get someone who can write front-end code paired with someone who can write back-end code, regardless of whether they are full-stack developers or not.

To transfer this over to something you’re more likely to be experienced with, we have a similar situation in SwiftUI. We regularly want to be able to create views with many subviews, and if we were working with a single view type such as Text then you could imagine something like Text... working great. But that wouldn’t work if we wanted to have some text, then an image, then a button, and more – any non-uniform layout would simply not be possible.

Trying to use AnyView... or similar to erase the types throws away all the type information, so before Swift 5.9 this problem was solved by creating lots of function overloads. For example, SwiftUI’s view builder has buildBlock() overloads that can combine two views, three views, four views, etc, all the way up to 10 views – but no further, because they need to draw a line somewhere.

So, we ended up with a 10-view limit in SwiftUI, and fingers crossed that should be about to disappear…

Macros

SE-0382, SE-0389, and SE-0397 combine to add macros to Swift, which allow us to create code that transforms syntax at compile time.

Note: Macros are complicated and rather tricky to work with right now; below is my best understanding of them having worked with them for a few weeks, but I’d appreciate any and all feedback.

Macros in something like C++ are a way to pre-process your code – to effectively perform text replacement on the code before it’s seen by the main compiler, so that you can generate code you really don’t want to write by hand.

Swift’s macros are similar, but significantly more powerful – and thus also significantly more complex. They also allow us to dynamically manipulate our project’s Swift code before it’s compiled, allowing us to inject extra functionality at compile time.

The key things to know are:

• They are type-safe rather than simple string replacements, so you need to tell your macro exactly what data it will work with.
• They run as external programs during the build phase, and do not live in your main app target.
• Macros are broken down into multiple smaller types, such as ExpressionMacro to generate a single expression, AccessorMacro to add getters and setters, and ConformanceMacro to make a type conform to a protocol.
• Macros work with your parsed source code – we can query individual parts of the code, such as the name of a property we’re manipulating or it types, or the various properties inside a struct.
• They work inside a sandbox and must operate only on the data they are given.

That last part is particularly important: Swift’s macros support are built around Apple’s SwiftSyntax library for understanding and manipulating source code. You must add this as a dependency for your macros.

Let’s start with a simple macro, so you can see how they work. Because macros are run at compile time, we can make a tiny macro that returns the date and time our app was built – a helpful thing to have in your debug diagnostics. This takes several steps, some of which should take place in a separate module from your main target.

First we need to create the code that performs the macro expansion – the thing that will turn #buildDate into something like 2023-06-05T18:00:00Z:

public struct BuildDateMacro: ExpressionMacro {
    public static func expansion(
        of node: some FreestandingMacroExpansionSyntax,
        in context: some MacroExpansionContext
    ) -> ExprSyntax {
        let date = ISO8601DateFormatter().string(from: .now)
        return "\"\(raw: date)\""
    }
}

Important: This code should not be in your main app target; we don’t want that code being compiled into our finished app, we just want the finished date string in there.

Inside that same module we create a struct that conforms to the CompilerPlugin protocol, exporting our macro:

import SwiftCompilerPlugin
import SwiftSyntaxMacros

@main
struct MyMacrosPlugin: CompilerPlugin {
    let providingMacros: [Macro.Type] = [
        BuildDateMacro.self
    ]
}

We would then add that to our list of targets in Package.swift:

.macro(
  name: "MyMacrosPlugin",
  dependencies: [
    .product(name: "SwiftSyntax", package: "swift-syntax"),
    .product(name: "SwiftSyntaxMacros", package: "swift-syntax"),
    .product(name: "SwiftCompilerPlugin", package: "swift-syntax")
  ]
),

That finishes creating the macro in an external module. The rest of our code takes place wherever we want to use the macro, such as in our main app target.

This takes two steps, starting with a definition of what the macro is. In our case this is a free-standing expression macro that will return a string, it exists inside the MyMacrosPlugin module, and has the strict name BuildDateMacro. So, we’d add this definition to our main target:

@freestanding(expression)
macro buildDate() -> String =
  #externalMacro(module: "MyMacrosPlugin", type: "BuildDateMacro")

And the second step is to actually use the macro, like this:

print(#buildDate)

When you read through this code, the most important thing to take away is that the main macro functionality – all that code inside the BuildDateMacro struct – is run at build time, with its results being injected back into the call sites. So, our little print() call above would be rewritten to something like this:

print("2023-06-05T18:00:00Z")

This in turn means the code inside your macros can be as complex as you need: we could have crafted our date in any way we wanted, because all that finished code actually sees is the string we returned.

Now, in practice the Swift team recommends against this kind of macro, because they want us to build things with consistent output – they prefer macros that produce the same output given the same output, because it allows things like incremental builds to function efficiently.

Let’s try a slightly more useful macro, this time making a member attribute macro. When applied to a type such as a class, this lets us apply an attribute to every member in a class. This is identical in concept to the older @objcMembers attribute, which adds @objc to each of the properties in a type.

For example, if you have an observable object that uses @Published on every one of its properties, you could write a simple @AllPublished macro that does the job for you. First, write the macro itself:

public struct AllPublishedMacro: MemberAttributeMacro {
    public static func expansion(
        of node: AttributeSyntax,
        attachedTo declaration: some DeclGroupSyntax,
        providingAttributesFor member: some DeclSyntaxProtocol,
        in context: some MacroExpansionContext
    ) throws -> [AttributeSyntax] {
        [AttributeSyntax(attributeName: SimpleTypeIdentifierSyntax(name: .identifier("Published")))]
    }
}

Second, include that in your list of provided macros:

struct MyMacrosPlugin: CompilerPlugin {
    let providingMacros: [Macro.Type] = [
        BuildDateMacro.self,
        AllPublishedMacro.self,
    ]
}

Third, declare the macro in your main app target, this time marking it as an attached member-attribute macro:

@attached(memberAttribute)
macro AllPublished() = #externalMacro(module: "MyMacrosPlugin", type: "AllPublishedMacro")

And now use it to annotate your observable object class:

@AllPublished class User: ObservableObject {
    var username = "Taylor"
    var age = 26
}

Our macros are able to accept parameters to control their behavior, although here it’s easy for the complexity to really shoot upwards. As an example, Doug Gregor from the Swift team maintains a small GitHub repository of example macros, including one neat one that checks hard-coded URLs are valid at build time – it becomes impossible to type a URL wrongly, because the build won’t proceed.

Declaring the macro in our app target is straightforward, including adding a string parameter:

@freestanding(expression) public macro URL(_ stringLiteral: String) -> URL = #externalMacro(module: "MyMacrosPlugin", type: "URLMacro")

Using it is also straightforward:

let url = #URL("https://swift.org")
print(url.absoluteString)

That makes url into a full URL instance rather than an optional one, because we will have checked the URL is correct at compile time.

What’s harder is the actual macro itself, which needs to read the "https://swift.org" string that was passed in and convert it into a URL. Doug’s version is more thorough, but if we boil it down to the bare minimum we get this:

public struct URLMacro: ExpressionMacro {
    public static func expansion(
        of node: some FreestandingMacroExpansionSyntax,
        in context: some MacroExpansionContext
    ) -> ExprSyntax {
        guard let argument = node.argumentList.first?.expression,
              let segments = argument.as(StringLiteralExprSyntax.self)?.segments
        else {
            fatalError("#URL requires a static string literal")
        }

        guard let _ = URL(string: segments.description) else {
            fatalError("Malformed url: \(argument)")
        }

        return "URL(string: \(argument))!"
    }
}

SwiftSyntax is marvelous, but it’s not what I’d call discoverable.

There are three more things I want to add before moving on.

First, the MacroExpansionContext value we’re given has a very helpful makeUniqueName() method, which will produce a new variable name that’s guaranteed not to conflict with any other names in the current context. If you’re looking to inject new names into the finished code, makeUniqueName() is a smart move.

Second, one of the concerns with macros is the ability to debug your code when you hit a problem – it’s hard to trace what’s going on when you can’t actually step through code easily. Some work has already taken place inside SourceKit to expand macros as a refactoring operation, but really we need to see what ships in Xcode.

And finally, the extensive transformations that macros enable may mean that Swift Evolution itself will evolve over the next year or two, because so many features that might previously have required extensive compiler support and discussion can now be prototyped and perhaps even shipped using macros.

Noncopyable structs and enums

SE-0390 introduces the concept of structs and enums that cannot be copied, which in turn allows a single instance of a struct or enum to be shared in many places – they still ultimately have one owner, but can now be accessed in various parts of your code.

Important: This change has a number of subtleties that I have tried to clarify below, but don’t be surprised if you need to read some things a few times.

First, this change introduces new syntax to suppress a requirement: ~Copyable. That means “this type cannot be copied”, and this suppression syntax is not available elsewhere at this time – we can’t use ~Equatable, for example, to opt out of == for a type.

So, we could create a new noncopyable User struct like this:

struct User: ~Copyable {
    var name: String
}

Note: Noncopyable types cannot conform to any protocols other than Sendable.

Once you create a User instance, its noncopyable nature means that it’s used very differently from previous versions of Swift. For example, this kind of code might read like nothing special:

func createUser() {
    let newUser = User(name: "Anonymous")

    var userCopy = newUser
    print(userCopy.name)
}

createUser()

But we’ve declared the User struct as being noncopyable – how can that take a copy of newUser? The answer is that it can’t: assigning newUser to userCopy causes the original newUser value to be consumed, which means it can no longer be used because ownership now belongs to userCopy. If you try changing print(userCopy.name) to print(newUser.name) you’ll see Swift throws up a compiler error – it’s just not allowed.

New restrictions also apply to how we use noncopyable types as function parameters: SE-0377 says that functions must specify whether they intend to consume the value and therefore render it invalid at the call site after the function finishes, or whether they want to borrow the value so that they can read all its data at the same time as other borrowing parts of our code.

So, we could write one function that creates a user, and another function that borrows the user to gain read-only access to its data:

func createAndGreetUser() {
    let newUser = User(name: "Anonymous")
    greet(newUser)
    print("Goodbye, \(newUser.name)")
}

func greet(_ user: borrowing User) {
    print("Hello, \(user.name)!")
}

createAndGreetUser()

In contrast, If we had made the greet() function use consuming User then the print("Goodbye, \(newUser.name)") would not be allowed – Swift would consider the newUser value to be invalid after greet() has run. On the flip side, because consuming methods must end the lifetime of the object, they can mutate its properties freely.

This shared behavior gives noncopyable structs a superpower that was previously restricted to classes and actors: we can give them deinitializers that will automatically be run when the final reference to a noncopyable instance is destroyed.

Important: This behaves a little differently from deinitializers on classes, which might either be an early implementation glitch or deliberate behavior.

First, here’s some code that uses a deinitializer with a class:

class Movie {
    var name: String

    init(name: String) {
        self.name = name
    }

    deinit {
        print("\(name) is no longer available")
    }
}

func watchMovie() {
    let movie = Movie(name: "The Hunt for Red October")
    print("Watching \(movie.name)")
}

watchMovie()

When that runs it prints “Watching The Hunt for Red October” then “The Hunt for Red October is no longer available”. But if you change the type’s definition from class Movie to struct Movie: ~Copyable you’ll see those two print() statements run in reverse order – it says the movie is no longer available, then says it’s being watched.

Methods inside a noncopyable type are borrowing by default, but they can be marked as mutating just like copyable types, and they can also be marked as consuming to mean that the value is invalid after the method has been run.

As an example, you might know the movie and TV series Mission Impossible, where secret agents are given their mission instructions in a self-destructing tape that can be played only once. This is perfect for a consuming method like this:

struct MissionImpossibleMessage: ~Copyable {
    private var message: String

    init(message: String) {
        self.message = message
    }

    consuming func read() {
        print(message)
    }
}

That marks the message itself as private, so it can only be access by calling the read() method that consumes the instance.

Unlike mutating methods, consuming methods can be run on constant instances of your type. So, code like this is fine:

func createMessage() {
    let message = MissionImpossibleMessage(message: "You need to abseil down a skyscraper for some reason.")
    message.read()
}

createMessage()

Note: Because message.read() consumes the message instance, it is an error to attempt to call message.read() a second time.

Consuming methods are made a little more complex when combined with deinitializers because they might double up on any clean up work you do. For example, if you were tracking high scores in a game you might want to have a consuming finalize() method that writes the latest high score to permanent storage and stops anyone else from changing the score further, but you might also have a deinitializer that saves the latest score to disk when the object is destroyed.

To avoid this problem, Swift 5.9 introduces a new discard operator that can be used on consuming methods of noncopyable types. When you use discard self in a consuming method, it stop the deinitializer from being run for this object.

So, we could implement our HighScore struct like this:

struct HighScore: ~Copyable {
    var value = 0

    consuming func finalize() {
        print("Saving score to disk…")
        discard self
    }

    deinit {
        print("Deinit is saving score to disk…")
    }
}

func createHighScore() {
    var highScore = HighScore()
    highScore.value = 20
    highScore.finalize()
}

createHighScore()

Tip: When that code runs you’ll see the deinitializer message is printed twice – once when we change the value property, which effectively destroys and recreates the struct, and once when the createHighScore() method finishes.

There are a few extra complexities you need to be aware of when working with this new functionality:

• Classes and actors cannot be noncopyable.
• Noncopyable types don’t support generics at this time, which rules out optional noncopyable objects and also arrays of noncopyable objects for the time being.
• If you use a noncopyable type as a property inside another struct or enum, that parent struct or enum must also be noncopyable.
• You need to be very careful adding or removing Copyable from existing types, because it dramatically changes how they are used. If you’re shipping code in a library, this will break your ABI.

consume operator to end the lifetime of a variable binding

SE-0366 extends the concept of consuming values to local variables and constants of copyable types, which might benefit developers who want to avoid excess retain/release calls happening behind the scenes as their data is passed around.

In its simplest form, the consume operator looks like this:

struct User {
    var name: String
}

func createUser() {
    let newUser = User(name: "Anonymous")
    let userCopy = consume newUser
    print(userCopy.name)
}

createUser()

The important line there is the let userCopy line, which does two things at once:

1. It copies the value from newUser into userCopy.
2. It ends the lifetime of newUser, so any further attempt to access it will throw up an error.

This allows us to tell the compiler explicitly “do not allow me to use this value again,” and it will enforce the rule on our behalf.

I can see this being particularly common with the so-called black hole, _, where we don’t want a copy of the data but simply want to mark it as being destroyed, like this:

func consumeUser() {
    let newUser = User(name: "Anonymous")
    _ = consume newUser
}

In practice, though, it’s possible the most common place the consume operator will be used is when passing values into a function like this:

func createAndProcessUser() {
    let newUser = User(name: "Anonymous")
    process(user: consume newUser)
}

func process(user: User) {
    print("Processing \(name)…")
}

createAndProcessUser()

There are two extra things I think are particularly worth knowing about this feature.

First, Swift tracks which branches of your code have consumed values, and enforces the rules conditionally. So, in this code only one of the two possibilities consumes our User instance:

func greetRandomly() {
    let user = User(name: "Taylor Swift")

    if Bool.random() {
        let userCopy = consume user
        print("Hello, \(userCopy.name)")
    } else {
        print("Greetings, \(user.name)")
    }
}

greetRandomly()

Second, technically speaking consume operates on bindings not values. In practice this means if we consume using a variable, we can reinitialize the variable and use it just fine:

func createThenRecreate() {
    var user = User(name: "Roy Kent")
    _ = consume user

    user = User(name: "Jamie Tartt")
    print(user.name)
}

createThenRecreate()

Convenience Async[Throwing]Stream.makeStream methods

SE-0388 adds a new makeStream() method to both AsyncStream and AsyncThrowingStream that sends back both the stream itself alongside its continuation.

So, rather than writing code like this:

var continuation: AsyncStream<String>.Continuation!
let stream = AsyncStream<String> { continuation = $0 }

We can now get both at the same time:

let (stream, continuation) = AsyncStream.makeStream(of: String.self)

This is going to be particularly welcome in places where you need to access the continuation outside of the current context, such as in a different method. For example, previously we might have written a simple number generator like this one, which needs to store the continuation as its own property in order to be able to call it from the queueWork() method:

struct OldNumberGenerator {
    private var continuation: AsyncStream<Int>.Continuation!
    var stream: AsyncStream<Int>!

    init() {
        stream = AsyncStream(Int.self) { continuation in
            self.continuation = continuation
        }
    }

    func queueWork() {
        Task {
            for i in 1...10 {
                try await Task.sleep(for: .seconds(1))
                continuation.yield(i)
            }

            continuation.finish()
        }
    }
}

With the new makeStream(of:) method this code becomes much simpler:

struct NewNumberGenerator {
    let (stream, continuation) = AsyncStream.makeStream(of: Int.self)

    func queueWork() {
        Task {
            for i in 1...10 {
                try await Task.sleep(for: .seconds(1))
                continuation.yield(i)
            }

            continuation.finish()
        }
    }
}

Add sleep(for:) to Clock

SE-0374 adds a new extension method to Swift’s Clock protocol that allows us to suspend execution for a set number of seconds, but also extends duration-based Task sleeping to support a specific tolerance.

The Clock change is a small but important one, particularly if you’re mocking a concrete Clock instance to remove delays in tests that would otherwise exist in production.

For example, this class can be created with any kind of Clock, and will sleep using that clock before triggering a save operation:

class DataController: ObservableObject {
    var clock: any Clock<Duration>

    init(clock: any Clock<Duration>) {
        self.clock = clock
    }

    func delayedSave() async throws {
        try await clock.sleep(for: .seconds(1))
        print("Saving…")
    }
}

Because that uses any Clock<Duration>, it’s now possible to use something like ContinuousClock in production but your own DummyClock in testing, where you ignore all sleep() commands to keep your tests running quickly.

In older versions of Swift the equivalent code would in theory have been try await clock.sleep(until: clock.now.advanced(by: .seconds(1))), but that wouldn’t work in this example because clock.now isn’t available as Swift doesn’t know exactly what kind of clock has been used.

As for the change to Task sleeping, it means we can go from code like this:

try await Task.sleep(until: .now + .seconds(1), tolerance: .seconds(0.5))

To just this:

try await Task.sleep(for: .seconds(1), tolerance: .seconds(0.5))

Discarding task groups

SE-0381 adds new discardable task groups that fix an important gap in the current API: tasks that are created inside a task group are automatically discarded and destroyed as soon as they finish, which means task groups that run for extended periods of time (or perhaps forever, as in the case of a web server) won’t leak memory over time.

When using the original withTaskGroup() API, a problem can occurs because of the way Swift only discards a child task and its resulting data when we call next() or loop over the task group’s children. Calling next() will cause your code to suspend if all child tasks are currently executing, so we hit the problem: you want a server that’s always listening for connections so you can add tasks to process them, but you also need to stop every so often to clean up old tasks that have completed.

There was no clean solution to this until Swift 5.9, which adds withDiscardingTaskGroup() and withThrowingDiscardingTaskGroup() functions that create new discarding task groups. These are task groups that automatically discard and destroy each task as soon as it completes, without us needing to call next() to consume it manually.

To give you an idea of what triggers the problem, we could implement a naive directory watcher that loops forever and reports back the names of any files or directories that have been added or removed:

struct FileWatcher {
    // The URL we're watching for file changes.
    let url: URL

    // The set of URLs we've already returned.
    private var handled = Set<URL>()

    init(url: URL) {
        self.url = url
    }

    mutating func next() async throws -> URL? {
        while true {
            // Read the latest contents of our directory, or exit if a problem occurred.
            guard let contents = try? FileManager.default.contentsOfDirectory(at: url, includingPropertiesForKeys: nil) else {
                return nil
            }

            // Figure out which URLs we haven't already handled.
            let unhandled = handled.symmetricDifference(contents)

            if let newURL = unhandled.first {
                // If we already handled this URL then it must be deleted.
                if handled.contains(newURL) {
                    handled.remove(newURL)
                } else {
                    // Otherwise this URL is new, so mark it as handled.
                    handled.insert(newURL)
                    return newURL
                }
            } else {
                // No file difference; sleep for a few seconds then try again.
                try await Task.sleep(for: .microseconds(1000))
            }
        }
    }
}

We could then use that from inside a simple app, although for brevity we’ll just print the URLs rather than do any actual complicated processing:

struct FileProcessor {
    static func main() async throws {
        var watcher = FileWatcher(url: URL(filePath: "/Users/twostraws"))

        try await withThrowingTaskGroup(of: Void.self) { group in
            while let newURL = try await watcher.next() {
                group.addTask {
                    process(newURL)
                }
            }
        }
    }

    static func process(_ url: URL) {
        print("Processing \(url.path())")
    }
}

That will run forever, or at least until either the user terminates the program or the directory we’re watching stops being accessible. However, because it uses withThrowingTaskGroup() it has a problem: a new child task is created every time addTask() is called, but because it doesn’t call group.next() anywhere those child tasks are never destroyed. Little by little – maybe only a few hundred bytes each time – this code will eat more and more memory until eventually the operating system runs out of RAM and is forced to terminate the program.

This problem goes away entirely with discarding task groups: just replacing withThrowingTaskGroup(of: Void.self) with withThrowingDiscardingTaskGroup means each child task is automatically destroyed as soon as its work finishes.

In practice, this problem is mainly going to be faced by server code, where the server must be able to accept new connections while handling existing ones smoothly.

And there’s more…

SE-0392 adds the ability to create custom actor executors, which gives developers fine-grained control over how an actor run its code. This is a feature specifically aimed at very precise, very advanced requirements, with even the Swift Evolution proposal saying there is “an expectation that custom executors will be primarily implemented by experts.”

Before Swift 5.9 we mostly didn’t care where concurrent code ran – we didn’t say “run this function on thread X, and this other function on thread Y,” but instead let Swift manage that for us. Custom executors allow us to be much more specific, perhaps because we want groups of actors to run on the same thread, or because the operating system requires certain work to take place on an exact thread.

Let’s not forget that there’s a good chance some more Swift Evolution proposals will make it in before the final release of 5.9. One I’m watching closely is: SE-0395: Observability, which is being used to power SwiftData in iOS 17 and later – clearly that's going to make its way through Evolution!

And if you still use much Objective-C in your project, it’s possible that SE-0384 might help you – if it manages to make it into Swift 5.9. This proposal improves Swift’s Objective-C importing code so that forward declarations become partially available in Swift code.

For folks who haven’t previously used Objective-C, a forward declaration is code that says something like, “a class called User will exist at some point, and although I want to reference it here I’ll define what it actually contains somewhere else.”

In previous versions of Swift these forward declarations were ignored, along with any code that used them, which often limited how much Objective-C code could be imported. However, from Swift 5.9 and on – if the proposal manages to ship in time – this changes in two important ways:

• If you receive a forward-declared class or protocol from some Objective-C code, you can pass it on to other Objective-C code that uses it.
• If you attempt to use it directly in Swift, e.g. to create a new instance of a forward-declared class, you’ll receive a compiler error that explains you need to import the original Objective-C module to get the full class implementation.

Again, SE-0384 might not make it into Swift 5.9, so watch this space.

Which Swift features are you most looking forward to? Tweet me at @twostraws, or send me a message on Mastodon at @twostraws@mastodon.social and let me know!

Sponsor Hacking with Swift and reach the world's largest Swift community!