Skip to content

Add S3EventNotifier example #477

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Feb 11, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Examples/S3EventNotifier/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.DS_Store
/.build
/.index-build
/Packages
xcuserdata/
DerivedData/
.swiftpm/configuration/registries.json
.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata
.netrc
52 changes: 52 additions & 0 deletions Examples/S3EventNotifier/Package.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
// swift-tools-version: 6.0
import PackageDescription

// needed for CI to test the local version of the library
import struct Foundation.URL

let package = Package(
name: "CSVUploadAPINotificationLambda",
platforms: [.macOS(.v15)],
dependencies: [
.package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", branch: "main"),
.package(url: "https://github.com/swift-server/swift-aws-lambda-events", branch: "main"),
.package(url: "https://github.com/swift-server/async-http-client.git", from: "1.24.0"),
],
targets: [
.executableTarget(
name: "CSVUploadAPINotificationLambda",
dependencies: [
.product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime"),
.product(name: "AWSLambdaEvents", package: "swift-aws-lambda-events"),
.product(name: "AsyncHTTPClient", package: "async-http-client"),
]
)
]
)

if let localDepsPath = Context.environment["LAMBDA_USE_LOCAL_DEPS"],
localDepsPath != "",
let v = try? URL(fileURLWithPath: localDepsPath).resourceValues(forKeys: [.isDirectoryKey]),
v.isDirectory == true
{
// when we use the local runtime as deps, let's remove the dependency added above
let indexToRemove = package.dependencies.firstIndex { dependency in
if case .sourceControl(
name: _,
location: "https://github.com/swift-server/swift-aws-lambda-runtime.git",
requirement: _
) = dependency.kind {
return true
}
return false
}
if let indexToRemove {
package.dependencies.remove(at: indexToRemove)
}

// then we add the dependency on LAMBDA_USE_LOCAL_DEPS' path (typically ../..)
print("[INFO] Compiling against swift-aws-lambda-runtime located at \(localDepsPath)")
package.dependencies += [
.package(name: "swift-aws-lambda-runtime", path: localDepsPath)
]
}
52 changes: 52 additions & 0 deletions Examples/S3EventNotifier/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# S3 Event Notifier

This example demonstrates how to create a Lambda that notifies an API of an S3 event in a bucket.

## Code

In this example the lambda function receives an `S3Event` object from the `AWSLambdaEvents` library as input object instead of a `APIGatewayV2Request`. The `S3Event` object contains all the information about the S3 event that triggered the lambda, but what we are interested in is the bucket name and the object key, which are inside of a notification `Record`. The object contains an array of records, however since the lambda is triggered by a single event, we can safely assume that there is only one record in the array: the first one. Inside of this record, we can find the bucket name and the object key:

```swift
guard let s3NotificationRecord = event.records.first else {
throw LambdaError.noNotificationRecord
}

let bucket = s3NotificationRecord.s3.bucket.name
let key = s3NotificationRecord.s3.object.key.replacingOccurrences(of: "+", with: " ")
```

The key is URL encoded, so we replace the `+` with a space.

Once the event is decoded, the lambda sends a POST request to an API endpoint with the bucket name and the object key as parameters. The API URL is set as an environment variable.

## Build & Package

To build & archive the package you can use the following commands:

```bash
swift build
swift package archive --allow-network-connections docker
```

If there are no errors, a ZIP file should be ready to deploy, located at `.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/S3EventNotifier/S3EventNotifier.zip`.

## Deploy

To deploy the Lambda function, you can use the `aws` command line:

```bash
aws lambda create-function \
--function-name S3EventNotifier \
--zip-file fileb://.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/S3EventNotifier/S3EventNotifier.zip \
--runtime provided.al2 \
--handler provided \
--architectures arm64 \
--role arn:aws:iam::<YOUR_ACCOUNT_ID>:role/lambda_basic_execution
```

The `--architectures` flag is only required when you build the binary on an Apple Silicon machine (Apple M1 or more recent). It defaults to `x64`.

Be sure to replace <YOUR_ACCOUNT_ID> with your actual AWS account ID (for example: 012345678901).

> [!WARNING]
> You will have to set up an S3 bucket and configure it to send events to the lambda function. This is not covered in this example.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add minimum command line instructions to do so

  1. create a bucket
  2. configure the event
  3. upload a file to show it works

Something like this (not tested)

aws s3api create-bucket --bucket my-test-bucket --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1

aws lambda add-permission \
    --function-name ProcessS3Upload \
    --statement-id S3InvokeFunction \
    --action lambda:InvokeFunction \
    --principal s3.amazonaws.com \
    --source-arn arn:aws:s3:::my-test-bucket

aws s3api put-bucket-notification-configuration \
    --bucket my-test-bucket \
    --notification-configuration '{
        "LambdaFunctionConfigurations": [{
            "LambdaFunctionArn": "arn:aws:lambda:<REGION>:<YOUR-ACCOUNT-ID>:function:ProcessS3Upload",
            "Events": ["s3:ObjectCreated:*"]
        }]
    }'

aws s3 cp testfile.txt s3://my-test-bucket/

86 changes: 86 additions & 0 deletions Examples/S3EventNotifier/Sources/main.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the SwiftAWSLambdaRuntime open source project
//
// Copyright (c) 2024 Apple Inc. and the SwiftAWSLambdaRuntime project authors
// Licensed under Apache License v2.0
//
// See LICENSE.txt for license information
// See CONTRIBUTORS.txt for the list of SwiftAWSLambdaRuntime project authors
//
// SPDX-License-Identifier: Apache-2.0
//
//===----------------------------------------------------------------------===//

import AWSLambdaEvents
import AWSLambdaRuntime
import AsyncHTTPClient

#if canImport(FoundationEssentials)
import FoundationEssentials
#else
import Foundation
#endif

let httpClient = HTTPClient.shared

enum LambdaError: Error {
case noNotificationRecord
case missingEnvVar(name: String)

var description: String {
switch self {
case .noNotificationRecord:
"No notification record in S3 event"
case .missingEnvVar(let name):
"Missing env var named \(name)"
}
}
}

let runtime = LambdaRuntime { (event: S3Event, context: LambdaContext) async throws -> APIGatewayV2Response in
do {
context.logger.debug("Received S3 event: \(event)")

guard let s3NotificationRecord = event.records.first else {
throw LambdaError.noNotificationRecord
}

let bucket = s3NotificationRecord.s3.bucket.name
let key = s3NotificationRecord.s3.object.key.replacingOccurrences(of: "+", with: " ")

guard let apiURL = ProcessInfo.processInfo.environment["API_URL"] else {
throw LambdaError.missingEnvVar(name: "API_URL")
}

let body = """
{
"bucket": "\(bucket)",
"key": "\(key)"
}
"""

context.logger.debug("Sending request to \(apiURL) with body \(body)")

var request = HTTPClientRequest(url: "\(apiURL)/upload-complete/")
request.method = .POST
request.headers = [
"Content-Type": "application/json"
]
request.body = .bytes(.init(string: body))

let response = try await httpClient.execute(request, timeout: .seconds(30))
return APIGatewayV2Response(
statusCode: .ok,
body: "Lambda terminated successfully. API responded with: Status: \(response.status), Body: \(response.body)"
)
} catch let error as LambdaError {
context.logger.error("\(error.description)")
return APIGatewayV2Response(statusCode: .internalServerError, body: "[ERROR] \(error.description)")
} catch {
context.logger.error("\(error)")
return APIGatewayV2Response(statusCode: .internalServerError, body: "[ERROR] \(error)")
}
}

try await runtime.run()
Loading