Collecting Xcode Cloud metrics using webhooks

Wednesday, March 29, 2023

Xcode Cloud has three kinds of webhooks which allow users to perform custom actions at different stages of a workflow’s lifecycle:

• Whenever a workflow build is created.
• Whenever a workflow build starts.
• Whenever a workflow build finishes.

These webhooks are great for building custom integrations which extend Xcode Cloud’s capabilities. In this article, I will go through how you can use Xcode Cloud webhooks to send workflow run metrics to an external analytics service.

The webhook’s payload

The data Xcode Cloud sends for each of the webhooks above is similar and is always sent as the body of a POST request in JSON format.

The payload contains abundant information about the workflow, the specific run that triggered it and even the product that it was triggered for!

You can find an example JSON with all the available data in Xcode Cloud’s documentation.

From all the information Xcode Cloud webhooks provide us with, we’re only interested in a few fields, which can be condensed into the following JSON object:

{
  "ciBuildRun": {
    "attributes": {
      "completionStatus": "SUCCEEDED",
      "startedDate": "2023-03-27T18:07:37.233Z",
      "executionProgress": "COMPLETE",
      "finishedDate": "2023-03-27T18:14:10.194Z",
      "sourceCommit": {
        "author": {
          "displayName": "Pol Piella Abadia"
        }
      }
    }
  },
  "ciWorkflow": {
    "attributes": {
      "name": "Build and Test"
    }
  },
  "scmGitReference": {
    "attributes": {
      "name": "main",
      "kind": "BRANCH"
    }
  },
  "scmRepository": {
    "attributes": {
      "repositoryName": "QRBuddy"
    }
  }
}

Receiving data from Xcode Cloud

In the server’s code, the first thing we need to do is turn the payload above into a Codable struct:

struct WebhookPayload: Decodable {
  let ciBuildRun: CIBuildRun
  let ciWorkflow: CIWorkflow
  let scmGitReference: SCMGitReference
  let scmRepository: SCMRepository

  struct CIBuildRun: Decodable {
    let attributes: Attributes

    struct Attributes: Decodable {
      let completionStatus: String
      let startedDate: Date
      let executionProgress: String
      let finishedDate: Date
      let sourceCommit: SourceCommit

      struct SourceCommit: Decodable {
        let author: Author

        struct Author: Decodable {
          let displayName: String
        }
      }
    }
  }

  struct CIWorkflow: Decodable {
    let attributes: Attributes

    struct Attributes: Decodable {
      let name: String
    }
  }

  struct SCMGitReference: Decodable {
    let attributes: Attributes

    struct Attributes: Decodable {
      let name: String
      let kind: String
    }
  }

  struct SCMRepository: Decodable {
    let attributes: Attributes

    struct Attributes: Decodable {
      let repositoryName: String
    }
  }
}

As there is no way to subscribe to specific webhooks in Xcode Cloud, data might vary based on the event that triggered the webhook. For example, the finishedDate field in the ciBuildRun object is only present in the webhook triggered when a workflow run finishes.

We can now use a JSONDecoder with a custom dateDecodingStrategy to cope with Xcode Cloud’s date formats to turn the request’s body into a Swift struct:

@main
struct XcodeCloudWebhookLambda: SimpleLambdaHandler {
    let decoder: JSONDecoder

    init() {
        let dateFormatter = DateFormatter()
        dateFormatter.dateFormat = "yyyy-MM-dd'T'HH:mm:ss.SSSZ"

        let decoder = JSONDecoder()
        decoder.dateDecodingStrategy = .formatted(dateFormatter)
        self.decoder = decoder
    }

    func handle(_ request: APIGatewayV2Request, context: LambdaContext) async throws -> APIGatewayV2Response {
        guard let body = request.body,
              let bodyData = body.data(using: .utf8),
              let request = try? decoder.decode(WebhookPayload.self, from: bodyData) else {
            return .init(statusCode: .badRequest, body: "Could not parse the request content...")
        }
    }
}

I have chosen to use an AWS Lamdba for this example as it allowed me to write my back-end service in Swift, but you can use any technology or language you want. If you’re interested in learning more about how to handle webhooks using Swift and AWS lambdas, please refer to my article on GitHub webhooks and Xcode Cloud.

Only listening to webhooks when the workflow finishes

As we are only interested in logging metrics for completed workflow runs, we need to make sure we filter out any other webhooks the server receives.

While the optional try operator in the previous section’s decoding code would have allowed us to ignore any payload missing any of the fields we are interested in (e.g. the run finished webhook is the only one that sends a finishedDate property in its body), it would also be good to have some extra validation in place and make the event filtering more explicit.

We can do this by checking the executionProgress field in the ciBuildRun object. This field should be COMPLETE when the workflow run finishes:

@main
struct XcodeCloudWebhookLambda: SimpleLambdaHandler {
    let decoder: JSONDecoder

    init() {
        let dateFormatter = DateFormatter()
        dateFormatter.dateFormat = "yyyy-MM-dd'T'HH:mm:ss.SSSZ"

        let decoder = JSONDecoder()
        decoder.dateDecodingStrategy = .formatted(dateFormatter)
        self.decoder = decoder
    }

    func handle(_ request: APIGatewayV2Request, context: LambdaContext) async throws -> APIGatewayV2Response {
        guard let body = request.body,
              let bodyData = body.data(using: .utf8),
              let payload = try? decoder.decode(WebhookPayload.self, from: bodyData),
              request.ciBuildRun.attributes.executionProgress == "COMPLETE" else {
            return .init(statusCode: .badRequest, body: "Could not parse the request content...")
        }
    }
}

Sending the data to a third-party service

Now that we have a server that can receive data from Xcode Cloud, we need to map this data into the format that our analytics service expects.

I am not going to go into any detail about this analytics system, but all you need to know is that it expects a POST request with the following payload and it aggregates data from multiple repositories and providers in a single database to later display it on a dashboard:

{
  "workflow": "unit-tests",
  "duration": 10.5,
  "date": "2023-03-27T18:14:10.194Z",
  "provider": "Xcode Cloud",
  "outcome": "success",
  "repository": "QRBuddy",
  "branch": "main",
  "author": "Pol Piella Abadia"
}

Calculating the duration

The first thing we need to do is calculate how long the workflow took to run. The hard work for this calculation is done by setting the dateDecodingStrategy to a custom format which matches the one used by Xcode Cloud in the JSONDecoder.

Doing this allows us to decode any date strings returned by Xcode Cloud into Date types automatically.

All that’s left for us to do then is to get the TimeInterval in seconds between the startedDate and finishedDate fields:

func handle(_ request: APIGatewayV2Request, context: LambdaContext) async throws -> APIGatewayV2Response {
    // ...

    let duration = payload
        .ciBuildRun
        .attributes
        .finishedDate
        .timeIntervalSince(payload.ciBuildRun.attributes.startedDate)
}

The code above will return the duration in seconds, which is what we need to send to the analytics service 🎉.

Mapping the outcome

The next thing we need to do is map Xcode Cloud’s completionStatus field to what the analytics service expects. Our analytics service expects the outcome to be one of the following: success, failure or cancelled.

On the other hand, Xcode Cloud’s CiCompletionStatus docs state that the outcome of a workflow run can be: SUCCEEDED, FAILED, ERRORED, CANCELED or SKIPPED.

We can start by modifying the Decodable model we defined earlier in the article to use an enum for the completionStatus field instead of a raw string:

enum CompletionStatus: String, Decodable {
    case succeeded = "SUCCEEDED"
    case failed = "FAILED"
    case errored = "ERRORED"
    case canceled = "CANCELED"
    case skipped = "SKIPPED"
}

struct CIBuildRun: Decodable {
  let attributes: Attributes

  struct Attributes: Decodable {
    let completionStatus: CompletionStatus
    let startedDate: Date
    let executionProgress: String
    let finishedDate: Date
    let sourceCommit: SourceCommit

    struct SourceCommit: Decodable {
      let author: Author

      struct Author: Decodable {
        let displayName: String
      }
    }
  }
}

We can then write a function which handles the mapping across both domains:

// Analytics service domain
enum Outcome: String {
    case success
    case failure
    case cancelled
}

func handle(_ request: APIGatewayV2Request, context: LambdaContext) async throws -> APIGatewayV2Response {
    // ...
    guard let outcome = Self.adapt(xcodeCloudOutcome: payload.ciBuildRun.attributes.completionStatus) else {
        return .init(statusCode: .ok, body: "Not handling skipped run")
    }
}

private static func adapt(xcodeCloudOutcome: CompletionStatus) -> Outcome? {
    switch xcodeCloudOutcome {
    case .succeeded: return .success
    case .failed, .errored: return .failure
    case .canceled: return .cancelled
    // Not logging skipped runs
    case .skipped: return nil
    }
}

Putting the pieces together

We now have all the pieces we need to put together a request to the analytics service:

func handle(_ request: APIGatewayV2Request, context: LambdaContext) async throws -> APIGatewayV2Response {
    // ...
    let payload = AnalyticsPayload(
        workflow: payload.ciWorkflow.attributes.name,
        duration: duration,
        date: payload.ciBuildRun.attributes.startedDate,
        provider: "Xcode Cloud",
        outcome: outcome,
        repository: payload.scmRepository.attributes.repositoryName,
        branch: payload.scmGitReference.attributes.name,
        author: payload.ciBuildRun.attributes.sourceCommit.author.displayName
    )

    // An example of a service that makes a POST request to an API
    analyticsService.send(payload: payload)

Registering the webhook

Once we have our server ready to receive data from Xcode Cloud and send it to our analytics service, we have to tell Xcode Cloud to send messages to the URL where we have deployed it.

The only way to do this currently is by going to the app you want to add the webhook to in App Store Connect, navigating to the Xcode Cloud section and selecting the ‘Settings’ tab. In the ‘Settings’ page, select the ‘Webhooks’ tab and click on the ’+’ button to add a webhook:

Doing this will open a modal where you can give your webhook a name and provide the URL where you have deployed your server:

That’s it 🎉! Now, every time a workflow run finishes, Xcode Cloud will send a message to your server with the data you need to send to your analytics service.

Want to learn more?

The code I have shared in this article is part of a talk I am putting together for Swift Heroes and iOS Dev UK called ‘Making developer tools with Swift’.

If you want to learn more about how to make complex developer tooling systems with Swift and are interested in attending one of the conferences, make sure to book a ticket and come along!