AWS Lambda functions are a great way to automate certain tasks and processes in the cloud. They can be triggered by events, such as a file upload to an S3 bucket or a message sent to an SNS topic, allowing you to execute some code in response.
In this post, we’ll show you how to write data to an S3 bucket from a Lambda function. This can be useful for a variety of tasks, such as archiving log files or uploading data to a data lake.
S3 (Simple Storage Service) is Amazon’s cloud storage solution that allows you to store and access data from anywhere in the world. Writing to an S3 bucket from a Lambda function is a simple way to store and access data in the cloud.
In this post, I’ll walk you through how to write to an S3 bucket from a Lambda function. We’ll use the AWS SDK for Node.js to access S3.
Take this example as a starting point. This is not a production-ready code, probably some tweaks for permissions will be necessary to meet your requirements.
AWS resources we need
- Lambda Function
- S3 Bucket
- Lambda Role
- Bucket Policy
The Lambda function
const AWS = require("aws-sdk");
const s3 = new AWS.S3({
region: "us-east-1",
});
let response;
exports.lambdaHandler = async (event, context) => {
try {
console.log(event);
const params = {
Bucket: process.env.MY_BUCKET,
Key: 'test-file.txt',
Body: "test content"
}
console.log('writing to s3', params);
const result = await s3.putObject(params).promise();
console.log(result);
response = {
'statusCode': 200,
}
} catch (err) {
console.log(err);
return err;
}
return response
};
The SAM template
For the sake of simplicity, we are going to use Principal: “*” which makes your bucket 100% public!
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
Sample SAM Template for Lambda and S3
Parameters:
Environment:
Type: String
Description: Environment name. Example, staging
Resources:
MyLambda:
Type: AWS::Serverless::Function
DependsOn:
- "MyBucket"
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs12.x
Environment:
Variables:
MY_BUCKET:
Ref: "MyBucket"
Role:
Fn::GetAtt:
- "MyLambdaRole"
- "Arn"
Tags:
Name: !Sub "${Environment}-my-test-lambda"
MyBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub "${Environment}-my-test-bucket"
MyBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
PolicyDocument:
Id: BucketPolicy
Version: 2012-10-17
Statement:
- Sid: AccessAll
Action: s3:*
Effect: Allow
# Beware: this makes your bucket public!
Principal: "*"
Resource: !Join
- ''
- - 'arn:aws:s3:::'
- !Ref MyBucket
- /*
Bucket: !Ref MyBucket
MyLambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Action: "sts:AssumeRole"
Principal:
Service:
- "lambda.amazonaws.com"
Version: "2012-10-17"
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: MyLambdaPolicy
PolicyDocument:
Statement:
- Effect: Allow
Action: "s3:*"
Resource:
- Fn::GetAtt:
- "MyBucket"
- "Arn"
Version: "2012-10-17"As a good practice, always receive the environment as a parameter. This way, all your resources will be easier to identify.
Deploy
Build our lambda and template. This will check also the syntax of your template
$ sam buildOnly for the first time run and follow the steps:
$ sam deploy --guidedThe second time and so on you can execute:
$ sam deploy --no-confirm-changesetClean up your test AWS resources. Delete unused lambdas, buckets, etc to keep your account organized and the most important: no extra costs
