In my previous AWS projects, I worked with services such as DynamoDB, Lambda, API Gateway, S3, IAM, and Terraform. This time, I wanted to build something a little closer to a real production architecture. Instead of processing an order synchronously inside an API request, we'll build an event-driven order processing system where the API accepts an order, stores it, places a message on Amazon SQS, and lets a separate Lambda function process it asynchronously. We'll also handle: Automatic retries Failed messages Dead-letter queues Duplicate messages Idempotent processing EventBridge events CloudWatch logging IAM permissions Infrastructure as Code with Terraform The entire infrastructure is deployed using Terraform. What We Are Building The application accepts an order through an HTTP API. The flow is: Client │ ▼ API Gateway │ ▼ Order Lambda │ ├──────────────► DynamoDB │ Orders │ └──────────────► SQS │ ▼ Processor Lambda │ ├──────► DynamoDB │ Update Order │ └──────► EventBridge OrderCompleted If the processor repeatedly fails, Amazon SQS moves the message to a Dead-Letter Queue. SQS │ │ processing fails │ ▼ Retry │ │ fails again ▼ Retry │ │ maxReceiveCount reached ▼ DLQ The important part is that the API does not need to wait for the order to finish processing. It accepts the order and returns a PENDING status. The actual processing happens asynchronously. Architecture The final architecture contains the following AWS services: API Gateway AWS Lambda Amazon DynamoDB Amazon SQS Amazon EventBridge Amazon CloudWatch AWS IAM Terraform We will use two DynamoDB tables: Orders └── OrderId Products └── ProductId The order lifecycle is: PENDING ↓ PROCESSING ↓ COMPLETED If processing repeatedly fails, the SQS message is moved to the DLQ. Prerequisites Before starting, make sure you have: An AWS account AWS CLI Terraform Node.js AWS credentials configured Basic knowledge of Lambda, DynamoDB and SQS You can verify Terraform with: terraform version And verify AWS authentication with: aws sts get-caller-identity Project Structure The Terraform project looks like this: terraform/ ├── main.tf ├── variables.tf ├── terraform.tfvars ├── sqs.tf ├── iam.tf ├── lambda.tf ├── api_gateway.tf ├── eventbridge.tf ├── cloudwatch.tf ├── outputs.tf └── lambda/ ├── order/ │ └── index.mjs └── processor/ └── index.mjs Each Terraform file is responsible for a specific part of the infrastructure. Step 1: Configure Terraform In main.tf: terraform { required_version = ">= 1.6.0" required_providers { aws = { source = "hashicorp/aws" version = "~> 6.0" } archive = { source = "hashicorp/archive" version = "~> 2.7" } } } provider "aws" { region = var.aws_region } The AWS provider allows Terraform to communicate with AWS. The archive provider is used to package the Lambda source code into ZIP files. Step 2: Define Variables Create variables.tf: variable "aws_region" { type = string description = "AWS region" } variable "order_table_name" { type = string description = "DynamoDB order table name" } variable "product_table_name" { type = string description = "DynamoDB product table name" } Then configure the values in terraform.tfvars: aws_region = "ap-southeast-1" order_table_name = "Orders" product_table_name = "Products" Using variables makes the Terraform configuration easier to reuse. Step 3: Create DynamoDB Tables The first table stores orders. The second table stores products. module "dynamodb_order_table" { source = "terraform-aws-modules/dynamodb-table/aws" name = var.order_table_name hash_key = "OrderId" billing_mode = "PAY_PER_REQUEST" attributes = [ { name = "OrderId" type = "S" } ] } module "dynamodb_product_table" { source = "terraform-aws-modules/dynamodb-table/aws" name = var.product_table_name hash_key = "ProductId" billing_mode = "PAY_PER_REQUEST" attributes = [ { name = "ProductId" type = "S" } ] } The Orders table uses: OrderId as its partition key. The Products table uses: ProductId as its partition key. We use: billing_mode = "PAY_PER_REQUEST" because this project does not require us to manage provisioned read and write capacity. Step 4: Create the SQS Queue Now we need a queue between the API and the processing Lambda. First, create the Dead-Letter Queue: resource "aws_sqs_queue" "order_processing_dlq" { name = "order-processing-dlq" message_retention_seconds = 1209600 sqs_managed_sse_enabled = true } The DLQ retains messages for 14 days. Now create the main queue: resource "aws_sqs_queue" "order_processing" { name = "order-processing" visibility_timeout_seconds = 60 message_retention_seconds = 345600 receive_wait_time_seconds = 10 sqs_managed_sse_enabled = true redrive_policy = jsonencode({ deadLetterTargetArn = aws_sqs_queue.order_processing_dlq.arn maxReceiveCount = 4 }) } There are a few important settings here. Visibility timeout visibility_timeout_seconds = 60 When Lambda receives a message, SQS temporarily hides that message from other consumers. Message retention message_retention_seconds = 345600 The main queue keeps messages for up to four days. Dead-Letter Queue maxReceiveCount = 4 If the message is received four times without being successfully processed, SQS moves it to the DLQ. This prevents a permanently broken message from being retried forever. Step 5: Create the Order Lambda The Order Lambda is responsible for accepting the API request. It performs the following operations: Validate the request Check that the product exists Generate an order ID Store the order in DynamoDB Send the order ID to SQS Return the order ID to the client The environment variables are: environment { variables = { ORDER_TABLE = module.dynamodb_order_table.dynamodb_table_id PRODUCT_TABLE = module.dynamodb_product_table.dynamodb_table_id QUEUE_URL = aws_sqs_queue.order_processing.url } } The Lambda function uses Node.js 22: resource "aws_lambda_function" "order" { function_name = "order-service" filename = data.archive_file.order_lambda.output_path source_code_hash = data.archive_file.order_lambda.output_base64sha256 runtime = "nodejs22.x" handler = "index.handler" role = aws_iam_role.order_lambda.arn timeout = 10 memory_size = 256 environment { variables = { ORDER_TABLE = module.dynamodb_order_table.dynamodb_table_id PRODUCT_TABLE = module.dynamodb_product_table.dynamodb_table_id QUEUE_URL = aws_sqs_queue.order_processing.url } } } The Lambda source code uses the AWS SDK for DynamoDB and SQS. A simplified version of the request flow looks like this: const orderId = crypto.randomUUID(); await dynamodb.send( new PutItemCommand({ TableName: ORDER_TABLE, Item: { OrderId: { S: orderId }, CustomerId: { S: customerId }, ProductId: { S: productId }, Quantity: { N: String(quantity) }, Status: { S: "PENDING" }, CreatedAt: { S: new Date().toISOString() } } }) ); await sqs.send( new SendMessageCommand({ QueueUrl: QUEUE_URL, MessageBody: JSON.stringify({ orderId }) }) ); The important thing is that the API does not process the order itself. It only creates the order and places a message on SQS. Step 6: Create API Gateway Now we expose the Lambda through an HTTP API. The API contains two routes: POST /orders GET /orders/{orderId} The POST endpoint creates an order. The GET endpoint retrieves its current status. The architecture becomes: Client ↓ API Gateway ↓ Order Lambda API Gateway uses an AWS_PROXY integration, allowing the Lambda function to receive the HTTP request directly. Step 7: Connect SQS to the Processor Lambda Now we create the second Lambda. Its job is to consume messages from SQS. resource "aws_lambda_function" "processor" { function_name = "order-processor" filename = data.archive_file.processor_lambda.output_path source_code_hash = data.archive_file.processor_lambda.output_base64sha256 runtime = "nodejs22.x" handler = "index.handler" role = aws_iam_role.processor_lambda.arn timeout = 30 memory_size = 256 environment { variables = { ORDER_TABLE = module.dynamodb_order_table.dynamodb_table_id EVENT_BUS_NAME = aws_cloudwatch_event_bus.orders.name } } } Then connect SQS to Lambda: resource "aws_lambda_event_source_mapping" "order_processing" { event_source_arn = aws_sqs_queue.order_processing.arn function_name = aws_lambda_function.processor.arn batch_size = 1 } This means Lambda automatically polls the SQS queue and invokes the processor when messages are available. Step 8: Process the Order The processor first reads the order from DynamoDB. const existingOrder = await dynamodb.send( new GetItemCommand({ TableName: ORDER_TABLE, Key: { OrderId: { S: orderId } } }) ); If the order doesn't exist, we throw an error: if (!existingOrder.Item) { console.error(`Order ${orderId} does not exist`); throw new Error(`Order ${orderId} does not exist`); } This is important. We want Lambda to fail in this situation. Why? Because SQS uses the Lambda invocation result to determine whether the message was successfully processed. If Lambda throws an error: Lambda failure ↓ SQS message becomes available again ↓ Lambda retries ↓ Repeated failures ↓ DLQ Step 9: Prevent Duplicate Processing Amazon SQS provides at-least-once delivery. That means the same message can potentially be delivered more than once. So we cannot assume: 1 message = 1 Lambda invocation Instead, our processor needs to be idempotent. We store the order status in DynamoDB. Before processing: PENDING The processor conditionally changes it to: PROCESSING using: await dynamodb.send( new UpdateItemCommand({ TableName: ORDER_TABLE, Key: { OrderId: { S: orderId } }, UpdateExpression: "SET #status = :processing", ConditionExpression: "#status = :pending", ExpressionAttributeNames: { "#status": "Status" }, ExpressionAttributeValues: { ":pending": { S: "PENDING" }, ":processing": { S: "PROCESSING" } } }) ); The condition is important: Status must currently be PENDING Only then can the processor claim the order. Step 10: Handle Duplicate Messages After the order is already completed, another copy of the message might arrive. We check the current status: if ( currentStatus === "PROCESSING" || currentStatus === "COMPLETED" ) { console.log( `Order ${orderId} has already been processed or is being processed` ); return; } This makes the processor idempotent. For example: Message 1 ↓ PENDING ↓ PROCESSING ↓ COMPLETED Message 2 ↓ COMPLETED ↓ Ignore The second message is successfully consumed without processing the order again. Step 11: Complete the Order After processing, we update the order: await dynamodb.send( new UpdateItemCommand({ TableName: ORDER_TABLE, Key: { OrderId: { S: orderId } }, UpdateExpression: "SET #status = :completed", ConditionExpression: "#status = :processing", ExpressionAttributeNames: { "#status": "Status" }, ExpressionAttributeValues: { ":processing": { S: "PROCESSING" }, ":completed": { S: "COMPLETED" } } }) ); The state transition is: PENDING ↓ PROCESSING ↓ COMPLETED The conditional update prevents the order from being completed from an unexpected state. Step 12: Publish an Event with EventBridge Once the order is completed, we publish an event. First, create the custom event bus: resource "aws_cloudwatch_event_bus" "orders" { name = "order-events" } Then the processor publishes: await eventbridge.send( new PutEventsCommand({ Entries: [ { EventBusName: EVENT_BUS_NAME, Source: "order-service", DetailType: "OrderCompleted", Detail: JSON.stringify({ orderId }) } ] }) ); The resulting event contains information such as: { "source": "order-service", "detail-type": "OrderCompleted", "detail": { "orderId": "b91eb8b4-0c80-4305-9651-518f6347b8dc" } } This gives us another useful architectural property. The processor does not need to know what happens after an order is completed. Other systems can subscribe to the event later. For example: ┌── Email Service │ OrderCompleted ──┼── Analytics │ ├── Notification Service │ └── Inventory Service This is one of the main benefits of event-driven architecture. Step 13: Configure IAM Each Lambda gets its own execution role. The Order Lambda only needs permissions required for its job. For example: DynamoDB ├── GetItem └── PutItem SQS └── SendMessage The Processor Lambda has different permissions: DynamoDB ├── GetItem └── UpdateItem SQS ├── ReceiveMessage ├── DeleteMessage └── GetQueueAttributes EventBridge └── PutEvents This follows the principle of least privilege. The processor does not need permission to create DynamoDB tables. The Order Lambda does not need permission to publish EventBridge events. Each role gets only the permissions required by that Lambda. Step 14: Add CloudWatch Logs Each Lambda gets a dedicated CloudWatch log group. resource "aws_cloudwatch_log_group" "order_lambda" { name = "/aws/lambda/${aws_lambda_function.order.function_name}" retention_in_days = 7 } resource "aws_cloudwatch_log_group" "processor_lambda" { name = "/aws/lambda/${aws_lambda_function.processor.function_name}" retention_in_days = 7 } This gives us visibility into the application. For example, the processor logs: Order b91eb8b4-0c80-4305-9651-518f6347b8dc currently has status: PENDING Started processing order b91eb8b4-0c80-4305-9651-518f6347b8dc Order b91eb8b4-0c80-4305-9651-518f6347b8dc completed OrderCompleted event published for order b91eb8b4-0c80-4305-9651-518f6347b8dc Step 15: Initialize Terraform Now initialize the project: terraform init Then validate the configuration: terraform validate You should see: Success! The configuration is valid. Step 16: Review the Deployment Before creating anything: terraform plan Terraform will show the resources it intends to create. Always review the plan before applying infrastructure. Step 17: Deploy Run: terraform apply Terraform will create: API Gateway Lambda Lambda IAM Roles DynamoDB SQS SQS DLQ EventBridge CloudWatch Log Groups Once the deployment finishes, Terraform should report the resources it created. Step 18: Seed a Product Before creating an order, we need a product. For example: aws dynamodb put-item \ --table-name Products \ --item '{ "ProductId": {"S":"product-456"}, "Name": {"S":"Test Laptop"}, "Price": {"N":"999.99"} }' \ --region ap-southeast-1 Now the order API can reference: product-456 Step 19: Create an Order Our API accepts: { "customerId": "customer-123", "productId": "product-456", "quantity": 2 } Send the request: curl -X POST \ "https://YOUR_API_ID.execute-api.ap-southeast-1.amazonaws.com/orders" \ -H "Content-Type: application/json" \ -d '{ "customerId":"customer-123", "productId":"product-456", "quantity":2 }' The API immediately returns: { "orderId": "b91eb8b4-0c80-4305-9651-518f6347b8dc", "status": "PENDING" } Notice that the API doesn't wait for the processor. The order is now waiting for asynchronous processing. Step 20: Check the Order Now request: curl \ "https://YOUR_API_ID.execute-api.ap-southeast-1.amazonaws.com/orders/b91eb8b4-0c80-4305-9651-518f6347b8dc" After processing, the result is: { "orderId": "b91eb8b4-0c80-4305-9651-518f6347b8dc", "customerId": "customer-123", "productId": "product-456", "quantity": 2, "status": "COMPLETED", "createdAt": "2026-09-10T18:28:39.336Z" } The complete flow is now working: API Gateway ↓ Order Lambda ↓ DynamoDB ↓ SQS ↓ Processor Lambda ↓ DynamoDB ↓ COMPLETED ↓ EventBridge Step 21: Test Failure Handling A good distributed system should not only handle successful requests. We also need to test failures. I sent a message to SQS containing an order ID that doesn't exist: aws sqs send-message \ --queue-url "YOUR_QUEUE_URL" \ --message-body '{"orderId":"does-not-exist"}' \ --region ap-southeast-1 The processor detects that the order doesn't exist and throws an error: ERROR Order does-not-exist does not exist Because Lambda failed, SQS retries the message. After the configured number of failures, the message is moved to the DLQ. The result: SQS ↓ Lambda ↓ Failure ↓ Retry ↓ Failure ↓ Retry ↓ Failure ↓ Retry ↓ Failure ↓ DLQ We can verify the DLQ: aws sqs get-queue-attributes \ --queue-url "YOUR_DLQ_URL" \ --attribute-names ApproximateNumberOfMessages \ --region ap-southeast-1 The queue reported: ApproximateNumberOfMessages: 1 This confirmed that the failure path was working. Step 22: Test Duplicate Processing This is one of the most important tests. SQS provides at-least-once delivery, so we need to make sure a duplicate message does not process the order again. I manually sent the already completed order back to SQS: aws sqs send-message \ --queue-url "YOUR_QUEUE_URL" \ --message-body '{"orderId":"b91eb8b4-0c80-4305-9651-518f6347b8dc"}' \ --region ap-southeast-1 The processor found: Order b91eb8b4-0c80-4305-9651-518f6347b8dc currently has status: COMPLETED Then: Order b91eb8b4-0c80-4305-9651-518f6347b8dc has already been processed or is being processed The Lambda invocation succeeded without processing the order again. This proves our duplicate protection works. What We Have Tested At this point, the system has three important paths. 1. Successful order PENDING ↓ PROCESSING ↓ COMPLETED 2. Failed order Lambda failure ↓ SQS retry ↓ SQS retry ↓ SQS retry ↓ SQS retry ↓ DLQ 3. Duplicate order COMPLETED ↓ Duplicate message ↓ Detected ↓ Ignored These tests are more useful than simply checking whether Terraform created the AWS resources. They verify that the actual distributed workflow behaves as expected. Why Use SQS Here? Without SQS, the API could directly invoke the processing logic: API ↓ Lambda ↓ Process Order But now the API is coupled to the processing operation. With SQS: API ↓ SQS ↓ Processor The producer and consumer are separated. This gives us: Asynchronous processing Automatic retries Buffering Failure isolation DLQ support Independent scaling If the processor temporarily fails, the API can still accept orders. Why Use EventBridge? SQS is useful for work that needs to be processed. EventBridge is useful for events that other systems may want to react to. For example: Order Processor │ ▼ OrderCompleted │ ├── Notification ├── Analytics ├── Inventory └── Billing The order processor doesn't need to directly call every downstream service. It simply publishes an event. Why Use DynamoDB for the Order State? DynamoDB gives us a simple way to store the current state of an order: OrderId CustomerId ProductId Quantity Status CreatedAt More importantly, DynamoDB conditional writes allow us to make state transitions safely. For example: PENDING → PROCESSING only succeeds when the order is actually PENDING. That becomes important when multiple Lambda invocations could potentially see the same message. Infrastructure as Code Everything in this project is defined using Terraform. Instead of manually creating: Lambda SQS DynamoDB API Gateway IAM EventBridge CloudWatch through the AWS Console, the infrastructure is represented as code. The workflow becomes: Terraform ↓ terraform plan ↓ Review ↓ terraform apply ↓ AWS Infrastructure This makes the environment reproducible and easier to maintain. One Important Production Consideration The current architecture is intentionally simple, but there are two reliability issues that I would address before calling this production-ready. DynamoDB → SQS Currently the order flow is: Write Order to DynamoDB ↓ Send Message to SQS Imagine DynamoDB succeeds but SQS fails. The order would exist as: PENDING but there would be no message to process it. A production system could use a transactional outbox pattern or another reliable event publication strategy. DynamoDB → EventBridge There is a similar issue here: Update Order → COMPLETED ↓ Publish EventBridge event If the DynamoDB update succeeds but EventBridge publishing fails, the order could remain COMPLETED while the event is never published. A more robust architecture would make event publication retry-safe as well. These are important considerations once the system moves beyond a learning project. Final Architecture The complete system looks like this: ┌──────────────────┐ │ Client │ └────────┬─────────┘ │ ▼ ┌──────────────────┐ │ API Gateway │ └────────┬─────────┘ │ ▼ ┌──────────────────┐ │ Order Lambda │ └───────┬───┬──────┘ │ │ ┌────────────┘ └─────────────┐ ▼ ▼ ┌─────────────┐ ┌─────────────┐ │ DynamoDB │ │ SQS │ │ Orders │ │ Queue │ └─────────────┘ └──────┬──────┘ │ ▼ ┌──────────────┐ │ Processor │ │ Lambda │ └──────┬───────┘ │ ┌────────────┴────────────┐ ▼ ▼ ┌─────────────┐ ┌──────────────┐ │ DynamoDB │ │ EventBridge │ │ Orders │ │ OrderEvents │ └─────────────┘ └──────────────┘ SQS failures │ ▼ ┌─────────────┐ │ DLQ │ └─────────────┘ Lambda Logs │ ▼ ┌─────────────┐ │ CloudWatch │ └─────────────┘ Key Takeaways The main concepts demonstrated in this project are: API Gateway can expose a serverless HTTP API. Lambda can handle the API request without managing servers. DynamoDB can store the order state. SQS can decouple order creation from order processing. SQS automatically retries messages when processing fails. A Dead-Letter Queue provides a place to investigate repeatedly failed messages. SQS provides at-least-once delivery, so consumers should be designed to handle duplicates. DynamoDB conditional updates can help implement idempotent state transitions. EventBridge can publish domain events such as OrderCompleted. IAM roles can restrict each Lambda to only the AWS actions it needs. CloudWatch provides logs for observing the serverless application. Terraform allows the entire infrastructure to be deployed as code. The important lesson for me was that building an event-driven system is not just about connecting AWS services. The interesting part is handling what happens when things go wrong: What if processing fails? What if the same message arrives twice? What if an order doesn't exist? What happens after repeated failures? How do we prevent duplicate processing? How do we allow other systems to react to completed orders? Those failure cases are where the architecture becomes much more interesting than simply deploying a Lambda function. What's Next? The next step for this project is to build a small frontend that visualizes the complete order pipeline. Something like: Create Order ↓ API Gateway ↓ Order Lambda ↓ DynamoDB ↓ SQS ↓ Processor Lambda ↓ COMPLETED ↓ EventBridge The frontend will also be able to demonstrate successful orders, duplicate messages, processing failures, retries, and DLQ behavior. That should make the architecture much easier to understand visually.