This post will cover building Python Lambdas that will make use of boto3 – AWS’ powerful Python SDK. We will also demonstrate how to package other Python libraries which our Lambda will use. It will also demonstrate how to structure a serverless cloudformation project to use Python Lambdas and finally show how to build and deploy them using the AWS SAM CLI.
This post assumes you have your AWS remote access already configured – if not – please take a look at this article which covers that. We will focus more on building and deploying Python Lambdas here.
Setting up a project with a Python Lambda
Create a directory for the project – lets name it “python-lambda” and open it in your code editor of choice.
I personally like to use PyCharm.
Next create another directory inside the project directory that will be for our Lambda – name this one “dynamodb-reader” which as you might guess is one of things this Lambda will be doing – reading a Dynamodb table.
And under that create a _init_.py file, a requirements.txt file and our python script – “dynamodb-reader.py”
Your structure should not look like this.
python-lambda
--dynamodb-reader
--__init__.py
--dynamodb-reader.py
--requirements.txt
Coding the Python Lambda
The Python Lambda is going to do two things – read a key from a dynamodb table – which we will be creating – and make a post request to httpbin.org – an api testing endpoint – to which we will post the data.
We will use the boto3 library to read Dynamodb and the Python requests library to make the post request. We will pass the Dynamodb table key we want to read as an input to the Lambda
To do this in Lambda – we simply need to define an event_handler function – it always takes two arguments – the event which will have the details of the request to the Lambda – and the context – which AWS provides certain information such as the generated id of the Lambda invocation.
This is how it is coded in Python.
import boto3
import requests
dynamodb = boto3.resource("dynamodb")
dynamodb_table = dynamodb.Table("dynamodb-table")
def event_handler(event, context):
key = event["key"]
dynamodb_data = dynamodb_table.get_item(
Key={
"primary_key": key
}
)
httpbin_json = dynamodb_data["Item"]
headers = {
"Content-Type": "application/json",
}
response = requests.post("http://httpbin.org/post", headers=headers, json=httpbin_json)
print(response.content)
event_handler(None, None)
Simple enough – but note that we need to create our table with the same name as we have used to define the boto3 table. As a best practice, we should always make initialization calls outside the event_handler definition as AWS reuses Lambdas for a while after one is started – and it is more efficient to reuse those things like Dynamodb connections.
# Make these initialization calls outside the handler function
dynamodb = boto3.resource("dynamodb")
dynamodb_table = dynamodb.Table("dynamodb-table")
Next – add requests to the requirements.txt to have AWS SAM bundle the requests library with our Lambda.
#requirements.txt body
requests
Creating the Cloudformation Template
Create a template.yaml file at the root of the python-lambda project.
The name can be anything – but template.yaml is the default name AWS SAM will look for so we don’t need to provide it.
The template will define two resources – a Dynamodb table which our Lambda is going to use.
And our Lambda itself.
First – this is how we create the Dynamodb Table named “dynamodb-table”
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Python Lambda with Cloudformation
Resources:
DynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: dynamodb-table
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: primary_key
AttributeType: S
KeySchema:
- AttributeName: primary_key
KeyType: HASH
Next – this is how we add the Lambda we just wrote in the dynamodb-reader directory.
DynamoDBReader:
Type: AWS::Serverless::Function
Properties:
Architectures:
- arm64
Runtime: python3.9
CodeUri: ./dynamodb-reader/
Handler: dynamodb-reader.event_handler
Description: Python Lambda Demo
FunctionName: dynamodb-reader
Policies:
- DynamoDBCrudPolicy:
TableName:
Ref: DynamoDBTable
The Runtime should have have the Python version – the architectures is explained in this article – the CodeUri should point to the relative location of where the Python directory is located. The policy section gives our Lambda CRUD grants against the table.
Building the project with AWS SAM
Navigate to the root directory and simply type sam build.
$ sam build
Building codeuri: <your_path>\dynamodb-reader runtime: python3.9 metadata: {} architecture: arm64 functions: ['DynamoDBReader']
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam\build
Built Template : .aws-sam\build\template.yaml
Deploying the Python Lambda Cloudformation template
First create an appropriate samconfig.toml file as outlined here and also set your profile environment variable. Then to deploy, sam deploy –config-env <deployment_profile> and you should see some thing like this – go ahead and deploy it.
Waiting for changeset to be created..
CloudFormation stack changeset
-----------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType Replacement
-----------------------------------------------------------------------------------------
+ Add DynamoDBReaderRole AWS::IAM::Role N/A
+ Add DynamoDBReader AWS::Lambda::Function N/A
+ Add DynamoDBTable AWS::DynamoDB::Table N/A
-----------------------------------------------------------------------------------------
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y
Concluding
Once your cloudformation stack has been deployed – go ahead and add some data to the Dynamodb table through the Dynamodb console and then test your Lambda by passing one of the keys to the Lambda and running. You should see the httpbin response printed out.
This can be a base for deploying more complex Lambdas using Python with boto3.