James Beswick

The TestState API helps developers to iterate faster, resolve issues efficiently, and deliver high-quality applications with greater confidence.

This post demonstrates techniques that can be used for running and profiling different types of Lambda extensions. This post focuses on Lambda extensions written in C# and Rust.

This blog shows how web and mobile applications can upload large objects to Amazon S3 in a secured and efficient manner when using presigned URLs and multipart upload.

Welcome to the 15th edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share all of the most recent product launches, feature enhancements, blog posts, webinars, Twitch live streams, and other interesting things that you might have missed!

This blog post shows how to create a scalable difference checking tool for objects stored in S3 buckets. The Lambda function is invoked when S3 writes new versions of an object to the bucket. This example also shows how to remove earlier versions of object and define a set number of versions to retain.

Part 2 of this blog post expands on some of the advanced topics around scaling Lambda in parallelized workloads. It explains how the asynchronous invocation mode of Lambda scales and different ways to scale the worker Lambda function.

Many video streaming services show GIF animations in the frontend when users fast forward and rewind throughout a video. This helps customers see a preview and makes the user interface more intuitive. Generating these GIF files is a compute-intensive operation that becomes more challenging to scale when there are many videos.

You can now create new functions or upgrade existing Python functions to Python 3.9. Lambda’s support of the Python 3.9 runtime enables you to take advantage of improved performance and new features in this version. Additionally, the Lambda service now runs the __init_.py code before the handler, supports TLS 1.3, and provides enhanced logging for errors.

This post looks at implementing common relational database patterns using DynamoDB. Instead of using multiple tables, the single-table design pattern can use adjacency lists to provide many-to-many relational functionality.

In this post, I introduce some of the core streaming concepts for serverless applications. I explain some of the benefits of streaming architectures and how Kinesis works with producers and consumers. I compare different ways to ingest data, how streams are composed of shards, and how partition keys determine which shard is used. Finally, I explain the payload formats at the different stages of a streaming workload, how message ordering works with shards, and why idempotency is important to… Show more

In this post, I introduce some of the core streaming concepts for serverless applications. I explain some of the benefits of streaming architectures and how Kinesis works with producers and consumers. I compare different ways to ingest data, how streams are composed of shards, and how partition keys determine which shard is used. Finally, I explain the payload formats at the different stages of a streaming workload, how message ordering works with shards, and why idempotency is important to handle. Show less

A review of everything that happened in AWS Serverless in Q2 2021.

In this post, I show how the existing settings in the Alleycat application are not sufficient for handling the expected amount of traffic. I walk through the metrics visualizations for Kinesis Data Streams, Lambda, and DynamoDB to find which quotas should be increased.

In this post, I explain the all-time leaderboard logic in the Alleycat application. This is an asynchronous, eventually consistent process that checks batching of incoming records for new personal records. This uses Kinesis Data Firehose to provide a zero-administration way to deliver and process large batches of records continuously.

In this post, I explain the all-time leaderboard logic in the Alleycat application. This is an asynchronous, eventually consistent process that checks batching of incoming records for new personal records. This uses Kinesis Data Firehose to provide a zero-administration way to deliver and process large batches of records continuously.

This post focuses on ingesting data into Kinesis Data Streams. I explain the two approaches used by the Alleycat frontend and the simulator application and highlight other approaches that you can use. I show how messages are routed to shards using partition keys. Finally, I explore additional factors to consider when ingesting data, to improve efficiency and reduce cost.

In this post, I introduce the Alleycat racing application for processing streaming data. I explain the virtual racing logic and provide an overview of the application architecture. I summarize the deployment process for the different parts of the solution and show how to test the frontend once the deployment is complete.

With bus-to-bus event routing in EventBridge, you can now route events from any type of event bus to the default and custom buses. This post explains how to configure bus-to-bus event routing in the console and CLI and discusses potential use-cases.

This post is the final part in a 3-part series on performance optimization in Lambda. The Lambda service makes frequent performance improvements in the underlying hardware, software, and architecture of the service. This post identifies the parts of the Lambda lifecycle where developers can make the most impact on performance.

This post is the second in a 3-part series on performance optimization in Lambda. It explains the effect of the memory configuration on Lambda performance, and why the memory setting also controls the compute power and networking I/O available to a function.

This post is the first in a 3-part series on performance optimization in Lambda. It explains how the Lambda execution environment works and why cold starts occur.

With cross-Region event routing in EventBridge, you can now route events from any AWS Region to other supported Regions. This post explains how to configure cross-Region event routing in the console and CLI and explains how to restrict access to routing capabilities. Finally, I walk through an example you can deploy to your AWS account.

This blog post outlines a general approach to debugging Lambda performance issues and errors. This provides a repeatable process for isolating and resolving problems in your serverless workloads.

Using API Destinations, you can use EventBridge to invoke third-party APIs without writing any business logic. The service provides configurable throttling and secure key management, while buffering messages for up to 24 hours for messages that exceed this limit. You can use this integration to extend the capabilities of your workloads by filtering for events generated by AWS services or custom applications. API Destinations builds on existing EventBridge functionality, introducing the… Show more

Using API Destinations, you can use EventBridge to invoke third-party APIs without writing any business logic. The service provides configurable throttling and secure key management, while buffering messages for up to 24 hours for messages that exceed this limit. You can use this integration to extend the capabilities of your workloads by filtering for events generated by AWS services or custom applications. API Destinations builds on existing EventBridge functionality, introducing the Connection and API resources to help configure credentials and define API endpoints. Show less

Many existing monitoring and observability concepts also apply to Lambda-based applications. This post introduces key terms, application performance monitoring, with broad metrics can be useful for monitoring workloads.

All the feature launches, blogs, videos, tech talks and more happenings from the AWS Serverless service teams in Q1 2021.

CloudWatch Logs Insights allows you to search and analyze log data to find the causes of issues and help validate fixes when they are deployed. This post shows how to enable the feature for a Lambda function and search across logs. It explains why structured logging can be helpful for parsing data in analysis.

On July 15, 2021, AWS Lambda will deprecate Python 2.7 as a supported runtime, formally ending our Python 2.7 support.

This post explains common integration errors in Lambda-based applications. These include running an unintended version or alias of a function, triggering infinite loops unintentionally, and issues with downstream availability. In each case, I explain steps you can take to remediate the issue.

This is the second post in a series on debugging Lambda-based applications. This post shows how to identity and resolve memory and CPU-bound functions, and how to understand and use timeouts effectively in production applications.

Debugging serverless applications is different to debugging single-server or monolithic applications. You must consider debugging across multiple invocations and services, and understanding the state of a distributed workload.

The API destinations feature of EventBridge enables developers to integrate workloads with third-party applications using REST API calls. This provides an easier way to build decoupled, extensible applications that work with applications outside of the AWS Cloud.

X-Ray is a powerful tool for providing observability in serverless applications. With the launch of X-Ray trace context propagation in EventBridge, this allows you to trace requests across distributed applications more easily.

In this blog post, I explain how to secure workloads with public endpoints and the different authentication and authorization options available. I also show different approaches to exposing APIs publicly.

This post explains the Lambda execution environment and how the service protects customer data. It covers important steps you should take to prevent data leakage between invocations and provides additional security resources to review.

Lambda works with other AWS services to process and manage requests and data. This post explains how to understand and manage Service Quotas, when to request increases, and architecting with quotas in mind. It also explains how to control traffic for downstream server-based resources.

This post discusses common anti-patterns in event-driven architectures using Lambda. I show some of the issues when using monolithic Lambda functions or custom code to orchestrate workflows. I explain how to avoid recursive architectures that may cause invocation loops and why you should avoid calling functions from functions. I also explain different approaches to handling waiting in functions to minimize cost.

This post discusses the design principles that can help you develop well-architected serverless applications. I explain why using services instead of code can help improve your application’s agility and scalability. I also show how statelessness and function design also contribute to good application architecture. I cover how using events instead of batches helps serverless development, and how to plan for retries and failures in your Lambda-based applications.

Welcome to the 12th edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share all of the most recent product launches, feature enhancements, blog posts, webinars, Twitch live streams, and other interesting things that you might have missed!

In the Operating Lambda series, I cover important topics for developers, architects, and systems administrators who are managing AWS Lambda-based applications. This three-part series discusses event-driven architectures and how these relate to Lambda-based applications. Part 1 covers the benefits of the event-driven paradigm and how it can improve throughput, scale and extensibility, while also reducing complexity and the overall amount of code in an application.

The default behavior for stream processing in Lambda functions enables entire batches of messages to succeed or fail. You can also use batch bisecting functionality to retry batches iteratively if a single message fails. Now with custom checkpoints, you have more control over handling failed messages.

With tumbling windows, you can calculate aggregate values in near-real time for Kinesis data streams and DynamoDB streams. Unlike existing stream-based invocations, state can be passed forward by Lambda invocations. This makes it easier to calculate sums, averages, and counts on values across multiple batches of data.

Lambda now supports self-hosted Kafka as an event source so you can invoke Lambda functions from messages in Kafka topics to integrate into other downstream serverless workflows. This post shows how to configure a self-hosted Kafka cluster on EC2 and set up the network configuration.

With the new container image support for Lambda, you can use Docker to package your custom code and dependencies for Lambda functions. The 10 GB deployment package limit makes it possible to deploy larger workloads that do not fit into the existing 250 MB quota for zip files. In this post, I show how to build a Docker image and deploy the image in the Lambda service. I also show how to use AWS SAM to simplify the generation of a boilerplate project, build the image, and deploy the function.

With the new 1 ms billing for Lambda functions, optimizing code duration has a direct impact on the cost of running Lambda functions. Shorter-lived functions can often run much faster with only a few small changes, and this translates to lowering cost.

The new event replay feature in Amazon EventBridge enables you to automatically archive and replay events on an event bus. This can help for testing new features or new code, or hydrating services in development and test to more closely approximate a production environment.

Amazon MQ provide a fully managed, highly available message broker service for Apache ActiveMQ. Now Lambda supports Amazon MQ as an event source, you can invoke Lambda functions from messages in Amazon MQ queues to integrate into your downstream serverless workflows.

Lambda provides a comprehensive range of storage options to meet the needs of web application developers. These include other AWS services such as Amazon S3 and Amazon EFS. There are also native storage options available, such as temporary storage or Lambda layers. In this blog post, I explain the differences between these options, and discuss common use-cases to help you choose for your own applications.

Web application backends are one of the most popular workload types for serverless applications. The pay-per-value model works well for this type of workload. As traffic grows, it’s important to consider the design choices and service configurations used to optimize your cost.

Welcome to the 11th edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share all of the most recent product launches, feature enhancements, blog posts, webinars, Twitch live streams, and other interesting things that you might have missed!

Queues, publish/subscribe services, and event buses are important parts of a resilient, well-architected serverless application. These are provided in AWS by SQS, SNS, and EventBridge.

Messaging is an important part of serverless applications and AWS services provide queues, publish/subscribe, and event routing capabilities. This post reviews the main features of SNS, SQS, and EventBridge and how they provide different capabilities for your workloads.

By directly uploading these files to Amazon S3, you can avoid proxying these requests through your application server. This can significantly reduce network traffic and server CPU usage, and enable your application server to handle other requests during busy periods. S3 also is highly available and durable, making it an ideal persistent store for user uploads.

Serverless developers frequently import libraries and dependencies into their AWS Lambda functions. While you can zip these dependencies as part of the build and deployment process, in many cases it’s easier to use layers instead. In this post, I explain how layers work, and how you can build and include layers in your own applications.

Using AWS SAM, you can specify serverless resources, configure permissions, and define substitutions for the ASL template. You can deploy a standalone Step Functions-based application using the AWS SAM CLI, separately from other parts of your application.

In this post, I introduce the Happy Path example web application. I show the main features of the application, enabling end-users to upload maps and photos to the backend application.

Now Lambda supports Amazon MSK as an event source, you can invoke Lambda functions from messages in Kafka topics to integrate into your downstream serverless workflows.

Web developers commonly use relational databases in building their applications. When migrating to serverless architectures, a web developer can continue to use databases like RDS, or take advantage of other options available. RDS Proxy enables developers to pool database connections and use connection-based databases with ephemeral functions.

I show how traditional web-server applications compare with their serverless counterparts. I show how the infrastructure is managed for you in serverless, and how code for serverless developers in primarily focused on business logic.

Serverless applications can help you develop more agile applications that can scale automatically. By using serverless services in your architecture, this reduces the amount of boilerplate code. It also helps offload complex tasks to specialized services.

Cold starts happen when you first invoke a function, or when a function is invoked after being inactive for an extended period. They also happen when Lambda scales up a function, since each new instance of the function is a new execution environment. This blog post shows how to eliminate cold starts in architectures supporting web applications.

In this post, I discuss focus areas for load testing of serverless applications, and highlight two tools commonly used. I show how to configure Artillery with customized functions, and how to run tests to simulate load on the Ask Around Me application.

Web and mobile applications usually interact with a backend service, often via an API. Many front-end applications pass requests for processing, wait for a result, and then display this to the user. This synchronous approach is only one way to handle messages, but modern applications have alternatives to provide a better user experience.

This blog post explores how you can solve geolocation queries using geohashing. I discuss how you should decide on the resolution of a geohash for your specific workload.

Serverless applications are event-driven, using ephemeral compute functions to integrate services and transform data. While AWS Lambda includes a 512-MB temporary file system for your code, this is an ephemeral scratch resource not intended for durable storage.

EventBridge is the evolution of the CloudWatch Events service. It brings new features, including the ability to integrate data from popular SaaS providers as events within AWS.

In part 2, I cover the API configuration, geohashing algorithm, and real-time messaging architecture used in the Ask Around Me web application. These are needed for receiving and processing questions and answers, and sending results back to users in real time.

This post explores the backend architecture of the Ask Around Me application. I compare the cost and features in deciding between REST APIs and HTTP APIs in API Gateway. I introduce geohashing and the npm library used to handle geo-location queries in DynamoDB. And I show how you can build real-time messaging into your web applications using the publish-subscribe pattern with AWS IoT Core.

In this post, I introduce the Ask Around Me example web application. Learn to build a serverless realtime, Vue.js web app in part 1 of this series.

S3 and Lambda are two highly scalable AWS services that can be powerful when combined in serverless applications. In this post, I summarize many of the patterns shown across this series.

Well-designed serverless applications are decoupled, stateless, and use minimal code. As projects grow, a goal for development managers is to maintain the simplicity of design and low-code implementation. This blog post provides recommendations for designing and managing code repositories in larger serverless projects, and best practices for deploying releases of production systems.

If you need to fan out notifications, or hold messages in queue, you are also able to route S3 events to Amazon SNS or Amazon SQS. These standard notification mechanisms work well for most applications, and are simple to implement. However, for more complex notification patterns, you can use Amazon EventBridge to route events dynamically. This blog post explores advanced use-cases and how to implement these in your serverless applications.

In this post, I show how you can import large amounts of data to DynamoDB using a serverless approach. This uses Amazon S3 as a staging area and AWS Lambda for the custom business logic.

Many applications start to grow in complexity as they mature, making it harder for developers to maintain code or add new features. This can lead to monolithic applications, where developers must know more about the entire architecture to make changes. This blog post shows how you can use an event-based architecture to decouple services and functional areas of applications.

Developing a scalable translation solution for thousands of documents can be challenging using traditional, server-based architecture. Using a serverless approach, this becomes much easier since you can use storage and compute services that scale for you.

In this blog post, I show how you can deploy a serverless application that uses machine learning to interpret your documents. The architecture includes a queuing mechanism for handling large volumes, and posting the indexing metadata to an Amazon Elasticsearch Service domain. This solution is scalable and cost effective, and you can modify the functionality to meet your organization’s requirements.

Many businesses operate call centers that record conversations with customers for training or regulatory purposes. These vast collections of audio offer unique opportunities for improving customer service. In this blog post, I show how you can use a serverless approach to analyze audio data from your call center.

The AWS Serverless Application Repository (SAR) enables builders to package serverless applications and reuse these within their own AWS accounts, or share with a broader audience. Previously, SAR applications could only be shared with specific AWS account IDs or made publicly available to all users. For organizations with large numbers of AWS accounts, this means managing a large list of IDs. It also involves tracking when accounts are added or removed to the organizational group.

EventBridge recently introduced additional content filtering functionality, which creates new possibilities for building sophisticated rules. This blog post explores how to use event patterns to build rules that make this routing process more powerful without needing custom code.

Describes an upcoming change to the AWS SDK that affects Python developers using the requests module in Botocore. This post explains why the changes are happening, and describes what Python developers must do to continue using the requests library.

This new feature provides an option for builders with the most demanding, latency-sensitive workloads to execute their functions with predictable start-up times at any scale.

Using SQS FIFO with Lambda is straight-forward, with only minor differences in configuration compared with standard SQS queues. This blog post explains how serverless developers can get started using SQS FIFO queues in their Lambda functions.

This example creates a weather bot that responds to a text message from a user, providing weather information for the request "weather zipcode".

Many of our users do not speak English as a first language, and it’s one of the most commonly overlooked aspects of user experience.

Converting from server-side form management is a key step in migrating websites to serverless.

Visual Studio Code has become an enormously popular tool for serverless developers, partly due to the intuitive user interface. It’s also because of the rich ecosystem of extensions that can customize and automate so much of the development experience. We are excited to announce that the AWS Toolkit for Visual Studio Code extension is now generally available, making it even easier for the development community to build serverless projects using this editor.

In the first article, we built a serverless voting website in under an hour. Now let’s do it again, this time using AWS Amplify.

James Beswick and Richard Boyd test how far a geohashing solution can scale serverlessly.

Learn how to use a serverless approach to build a complex mobile app for a zoo.

There are many ways to get results back to the front-end, but the real-time option is easiest with AWS IoT.

Using AWS Rekognition and serverless, we can build a scalable member management system to replace membership cards.

Serverless design fundamentally alters the limiting factors constraining product development.

One of the easiest ways to tighten your Lambda bill is to not pay for waiting — which means getting your arms around async.

WordPress sites are easy to stand up but when Bad Things Happen they get knocked down just as easily.

Serverless is Lego. Find the pieces you need, build fast and get to market as quickly as possible.

The serverless world is pretty awesome once you find the right tools — and burn the middle layer to the ground.

A no-nonsense summary for business leaders and IT managers who need to know about cloud fast.

The Kraken will destroy your projects and grind progress to a halt. If you’re not serving customers — then you’re serving competitors

Many IT departments are bloated bureaucracies of interconnected silos that work to enforce the status quo — let’s fix it now

Cloud is key to reinvigorating your IT infrastructure, but you won’t get very far until you stop the behavior that feeds your monolith

Welcome to the 15th edition of the AWS Serverless ICYMI (in case you missed it) quarterly recap. Every quarter, we share all of the most recent product launches, feature enhancements, blog posts, webinars, Twitch live streams, and other interesting things that you might have missed!

A serverless face blurring service can provide a simpler way to process photos in workloads with large amounts of traffic. This post introduces an example application that blurs faces when images are saved in an S3 bucket. The S3 PutObject event invokes a Lambda function that uses Amazon Rekognition to detect faces and GraphicsMagick to process the images.

Serverless design fundamentally alters the limiting factors constraining product development.