AWS EventBridge is a serverless event bus that can publish “events” – messages with your own custom payloads – to subscribers in real time and of course at very massive scale like most stuff in the Amazon cloud platform. The nature of using an event bus to support event driven architectures makes the EventBridge service an excellent choice for many application use cases that can benefit from the pub/sub design pattern. The loosely coupled nature of working with EventBridge also makes any such platform very easy to compartmentalize and add and remove parts of the system very easily.
In my previous article posting, I had shown one such example application for using EventBridge as part of a notification platform – shooting out messages to different mediums based on the “event” triggers. In this article, I will show another example architecture that can possible make use of EventBridge – that of an Order Fulfillment System – where there will be different event triggers related to an order lifecycle – ordering, packaging, dispatching, invoicing and a final “customer experience” message like a thank you note or satisfaction. We will of course be simulating the workflow of such a system – understandably in a real world situation there would be external and possibly human inputs to actually trigger some of the events – but for the purposes of demonstrating how EventBridge could be incorporated in an actual system – the examples here should be useful.
We will build this example with EventBridge of course. The individual cogs of the overall fulfillment system will be built with Lambdas – coded in Python. Cloudformation will be used to define the infrastructure – including the EventBridge subscriptions – this is one of the big benefits of working with cloudformation – we do not need to manually wire the various components in the console and we can seamlessly update subscription triggers by adding appropriate EventBridge filter pattern schemas directly to our cloudformation template – and seamlessly replicate the stack in any AWS region.
The complete system will have the flow as outlined by the feature image which I will go over in the next section.
Describing the Order Fulfillment system flow
The flow diagram pictured above is the hypothetical system with the following workflow. New orders are sent to the Order Creation system – a Lambda that will simulate creating the order by generating an order id that can be the correlation for tracking the order throughout the rest of the system and process. The “packaging” system represents a real life warehouse application with a notification to the Warehouse staff to take the product that has been ordered out from inventory and package and label. Once the order is ready to ship – event subscriptions are triggered for both the Shipping department as well as the invoicing system to bill the client. Once shipped to the client, a customer experience system gets its own trigger to send out any communications like a thank you email and or a survey about the order or the product.
For the purposes of building this out – just repeating – the various “systems” will be Lambdas – coded in Python – subscribing to and/or sending out specific event types. Everything is loosely coupled together by EventBridge – more specifically and EventBridge event bus that is dedicated to the order fulfilment system. The various cogs are agnostic to the other systems or their implementation but simply do their respective actions of posting specific event types – which will be simple json payloads – and acting on those triggers.
Before we start coding all this – a quick word on how EventBridge subscriptions are made and triggered.
EventBridge Payloads & Subscriptions
Subscriptions will be easy to understand by taking a look at the structure of an EventBridge Payload object – which is a json object with the following keys:
{
'Time': publish_time_of_the_message,
'Source': "name_of_the_publisher",
'Detail': json.dumps(actual_event_data),
'DetailType': actual_event_detail, #new_order, ready_to_ship, delivered
'EventBusName': order_fulfillment_bus
}
Pretty straightforward and self explanatory – any publisher to the bus will post a payload with those details – and subscribers can configure themselves with EventBridge to specific subscriptions based on any of those fields – including specifics of the “Detail” object – this is more useful in the case of dynamic subscriptions – for instance if there is a need to only request subscriptions on a particular order ID instead of a higher level event that would get triggered on every order.
For our system simulation, we are going to work with the higher level events and not use dynamic subscriptions – but just be aware it is possible if necessary. In order to define a subscription – we just need to define a Lambda EventBridge event type – and this is done in cloudformation as shown below.
Events:
NewOrder:
Type: EventBridgeRule
Properties:
EventBusName: !Ref OrderFulfillmentEventBus
Pattern:
detail-type:
- new_order
Basically the snippet shows how to have EventBridge wire a Lambda with a subscription to the EventBridge pattern of DetailType of value “new_order”. We will demonstrate the full examples as we build out this system in the next few sections.
A word just in case you are not familiar with Cloudformation, please take a look at these articles where I walk through some of the setup and basics.
Deploying Python Lambdas with Cloudformation.
Creating a dedicated EventBus for our system
This is done quite easily using cloudformation as defined below – we will reference this “OrderFulfillmentEventBus” in the rest of the cloudformation stack. Also in the yaml, I am just setting the Lambda runtime to Python globally as that is what we are going to use.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Order Fulfillment with AWS EventBridge
Globals:
Function:
Runtime: python3.9
Resources:
OrderFulfillmentEventBus:
Type: AWS::Events::EventBus
Properties:
Name: "orders-event-bus"
EventBridge Publishing with the Boto3 SDK
Let’s demonstrate how to do this by defining and coding the first cog in the fulfillment system – the Order Creator – which is a simple Lambda that will take in some random order details generate an ID which it will return to the invoker – and publish the details to the bus which we just created above. We will make use of the excellent boto3 SDK to do the publishing – however in order for the Lambda to actually publish stuff – it will need IAM permissions to do so. We will wire up the permissions first in our cloudformation template definition before coding the publishing in Python. Note I am using a rather blanket utility provided by AWS SAM – AmazonEventBridgeFullAccess – but this can be tailored down to be more restrictive depending on the use case. We are also passing the name of the event bus as an environmental variable for the Lambda to publish to.
OrderCreator:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./order-creator/
Handler: order-creator.order_handler
FunctionName: order-creator
Policies:
- AmazonEventBridgeFullAccess
Environment:
Variables:
EVENTBUS: !Ref OrderFulfillmentEventBus
import boto3
import json
import os
import uuid
from datetime import datetime
eventbus = os.environ["EVENTBUS"]
eventbridge = boto3.client("events")
def order_handler(event, context):
# Simulated Order Details
order_id = str(uuid.uuid4())
product_id = event["product_id"]
quantity = event["quantity"]
client = event["client"]
address = event["address"]
start_time = datetime.now()
new_order_event = {
'Time': start_time,
'Source': "order-creator",
'Detail': json.dumps({
'order_id': order_id,
'product_id': product_id,
'quantity': order_id,
'client': client,
'address': address,
}),
'DetailType': 'new_order',
'EventBusName': eventbus
}
eventbridge.put_events(
Entries=[
new_order_event
]
)
It should be pretty straightforward to follow the Python segment above – but basically we have used the put_events api to drop a new_order message to the event bus. Now let’s segue to the Event subscription example by coding the “packaging” Lambda
EventBridge Subscription with the “Packaging” system
Next we want to create a subscription system to get those new_order events from EventBridge. We will do that just as we explained how subscriptions work in the previous section above. Since the Packaging system needs to also publish events – it’s definition is similar to the order creator Lambda. As far as any special handling for subscriptions in the code – none really – EventBridge makes available the payload message shown above to the subscribers – and these are all seamlessly retrieved as a simple Python dictionary – unlike some other services in AWS which require working with byte arrays.
Packager:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./packager/
Handler: packager.package_handler
FunctionName: packager
Policies:
- AmazonEventBridgeFullAccess
Environment:
Variables:
EVENTBUS: !Ref OrderFulfillmentEventBus
Events:
NewOrder:
Type: EventBridgeRule
Properties:
EventBusName: !Ref OrderFulfillmentEventBus
Pattern:
detail-type:
- new-order
import json
import os
import boto3
from datetime import datetime
eventbus = os.environ["EVENTBUS"]
eventbridge = boto3.client("events")
def package_handler(event, context):
payload = event["detail"]
order_id = payload["order_id"]
product_id = payload["product_id"]
quantity = payload["quantity"]
client = payload["client"]
address = payload["address"]
start_time = datetime.now()
#DO PACKAGING THEN SIGNAL FOR DELIVERY/BILLING
print(f"Packaging {order_id} of {quantity}x{product_id} for {client} in {address}")
ready_to_ship_event = {
'Time': start_time,
'Source': "packager",
'Detail': json.dumps(event),
'DetailType': 'ready_to_ship',
'EventBusName': eventbus
}
eventbridge.put_events(
Entries=[
ready_to_ship_event
]
)
Defining the cloudformation resources for the Shipping, Invoice and Customer Experience Systems
The Python code demonstrated above for publishing and subscribing should hopefully be sufficient to get the idea for using a baseline implementation for specific actual components – however I will complete out the yaml definitions for the rest – the shipping system in our design does need to publish a delivery notification – so it again needs IAM access to the bus along with the event bus name – the invoice and customer experience systems do not and are much simpler to define. Also note that the shipping and invoice systems both share the same subscription pattern filter – which means EventBridge will ensure both these Lambdas get the payload for the ready_to_ship event type.
Shipping:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./shipping/
Handler: shipping.shipping_handler
FunctionName: shipping
Policies:
- AmazonEventBridgeFullAccess
Environment:
Variables:
EVENTBUS: !Ref OrderFulfillmentEventBus
Events:
ShipEvent:
Type: EventBridgeRule
Properties:
EventBusName: !Ref OrderFulfillmentEventBus
Pattern:
detail-type:
- ready_to_ship
Invoice:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./invoice/
Handler: invoice.invoice_handler
FunctionName: invoice
Events:
InvoiceEvent:
Type: EventBridgeRule
Properties:
EventBusName: !Ref OrderFulfillmentEventBus
Pattern:
detail-type:
- ready_to_ship
CustomerExperience:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./customer-experience/
Handler: customer-experience.customer_experience_handler
FunctionName: customer-experience
Events:
InvoiceEvent:
Type: EventBridgeRule
Properties:
EventBusName: !Ref OrderFulfillmentEventBus
Pattern:
detail-type:
- delivered
Next Steps
The most obvious next steps would be to integrate the system with a system of record – since we are working with serverless infrastructure – why not DynamoDB? If you like, take a look at this article I wrote which makes use of DynamoDB in a similarly complex application.
The more general next steps would be to spend time considering whether some of the benefits of the loosely coupled event driven nature of EventBridge would be something that makes sense for your next application. In that regard I hope this demo application with examples of how to set up and wire EventBridge was useful for you.