How to create maintainable Lambda functions using custom templates

Amazon introduced Lambda functions in 2014 and it took almost two full years before .NET developers were able to create functions using their preferred stack.

At their core, Lambda functions are amazing to run very simple snippets of code.

public class Function
{
    public string FunctionHandler(string input, ILambdaContext context)
    {
        return input.ToUpper();
    }
}

The function handler is extremely focused on solving an issue: accept an input, process it, return the output.

The problems come when we start introducing concepts like configuration, logging and dependency injection. All valid and reasonable concerns when modeling a complex system based on many moving parts. At that level, the simple programming model behind Lambda functions starts colliding with established practices like SOLID principles, unit testing and so on. Especially because the business logic gets mudded and drowned in a series of cross-cutting concerns that really shouldn’t be there. Also, unit testing becomes impossible.

To make things worse, .NET backend developers spend most of their time working with ASP.NET Core and it’s just normal to see most of the ecosystem gravitate around patterns introduced and pushed by the Microsoft application framework. This is even more evident when it comes to handling the aforementioned aspects. As it usually happens in the .NET space, when Microsoft introduces a basic library, the whole ecosystem follows.

After all, who wants to create a new configuration subsystem when ASP.NET Core uses such a powerful one, especially when it can be easily extended to fit any need? And the same goes for many of the Microsoft Extensions libraries.

This is where this project of mine comes to help: AWSLambdaSharpTemplate.

The idea behind this project is simple: use the Template Method pattern to give developers an experience that looks like the one offered by ASP.NET Core and its Startup class. Create sensible standards and package it all in a series of templates that can be hydrated using the facilities offered by the dotnet new command.

This gives the developers the ability to:

• reuse concepts and components from the ASP.NET Core ecosystem
• create business logic-dense components that are easily testable and pruned of any initialization logic
• focus on the business logic

Yes, I’ve mentioned twice the business logic because that’s what really adds value to the companies we work for. Stakeholders and customers won’t care about fancy new initialization script. They care about the product. And in a backend service, that’s the business logic.

Types of functions

C# is a typed language. Even if some magic can be introduced to allow duck-typing, the primary goal of the library was to lower the cognitive load required by the developers to create a functioning Lambda function.

This is why the library is based on the distinction between two types of functions:

• Functions that handle events and that are executed asynchronously and that are not expected to return a value
• Functions that process a request and return a response.

To reflect the difference, the library supports two base classes: EventFunction<TInput> and RequestResponseFunction<TOutput>. The programming model is the same, the only difference is the types involved.

Specifically, a developer creating an Event function will have to provide an implementation of the interface IEventHandler<TInput>. Similarly, creating a Request/Response function requires a class implementing IRequestResponseHandler<TInput, TOutput> to be registered.

I agree that RequestResponse is an awkward name but it’s the name that was used by AWS at the beginning and it’s still used in the AWS CLI and the various SDKs. Nowadays AWS uses the naming synchronous and asynchronous for the two types of functions.

Creating a simple function

Let’s start creating a function from scratch. We will then see how the templates help us getting there with less work.

Since the programming model is mostly the same, I’ll refer mostly to request/response functions.

First, we need to create a new project. We can use the built-in class library project template.

$ dotnet new classlib -o FindNationalityFunction
$ cd FindNationalityFunction

Then, we add the library to the project.

$ dotnet add package Kralizek.Lambda.Template

Here is the bare minimum example needed to create a Lambda function that uses Nationalize.io to guess the nationality of a person given its name.

using System.Net.Http.Json;
using System.Text.Json.Serialization;
using Amazon.Lambda.Core;
using Kralizek.Lambda;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace FindNationality;

public class Function : RequestResponseFunction<string, Country[]>
{
    protected override void ConfigureServices(IServiceCollection services, IExecutionEnvironment executionEnvironment)
    {
        RegisterHandler<FindNationalityHandler>(services);
    }
}

public class FindNationalityHandler : IRequestResponseHandler<string, Country[]>
{
    private readonly HttpClient http = new HttpClient();

    public async Task<Country[]> HandleAsync(string? input, ILambdaContext context)
    {
        var response = await http.GetFromJsonAsync<Response>($"https://api.nationalize.io/?name={input}");

        return response?.Countries ?? Array.Empty<Country>();
    }
}

public record Response(
    [property: JsonPropertyName("country")] Country[] Countries,
    [property: JsonPropertyName("name")] string Name
);

public record Country(
    [property: JsonPropertyName("country_id")] string CountryCode,
    [property: JsonPropertyName("probability")] double Probability
);

Let’s remove the existing Class1.cs file and replace it with a file named Function.cs with the snippet above.

That’s it. We’re ready to ship. Almost.

Before we ship it to production, let’s take a moment to realize that we didn’t do much more than simply lifting the business logic from the handler function of the first snippet to a stand-alone class, FindNationalityHandler.

If we were to compare the performance of this function with one written like the one in the top of the post, this one would be slower. But it would be an unfair comparison.

Let’s start adding some more interesting bits to the function.

I’ve create a repository on GitHub where I reproduced one by one the steps illustrated in this post. You can find the source code here: https://github.com/Kralizek/LambdaFunctionSample.
I’ve also executed each step as a stand-alone commit, so you can easily visualize the changes that each step introduces. This is the diff for the step of this paragraph.

Leveraging Dependency Injection

The first bit that we add is something that most .NET developers will have noticed immediatedly. If there is something that you always get wrong in .NET, it’s how to use HttpClient. I’ll not dive into the topic, there is already a lot of literature about it.

Let’s just thank Microsoft for introducing Microsoft.Extensions.Http to solve the issue they created.

First, we add the package.

$ dotnet add package Microsoft.Extensions.Http

And we change the handler to use the IHttpClientFactory. Also, let’s take the chance to add a logger, we will use it later.

public class FindNationalityHandler : IRequestResponseHandler<string, Country[]>
{
  private readonly IHttpClientFactory _httpClientFactory;
  private readonly ILogger<FindNationalityHandler> _logger;

  public FindNationalityHandler(IHttpClientFactory httpClientFactory, ILogger<FindNationalityHandler> logger)
  {
    _logger = logger ?? throw new ArgumentNullException(nameof(logger));
    _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
  }

  public async Task<Country[]> HandleAsync(string? input, ILambdaContext context)
  {
    var http = _httpClientFactory.CreateClient("Nationality");

    _logger.LogDebug("Finding the nationality of people named {Name}", input);

    var response = await http.GetFromJsonAsync<Response>($"https://api.nationalize.io/?name={input}");

    return response?.Countries ?? Array.Empty<Country>();
  }
}

Now that our handler has a dependency in the constructor, we need to instruct the function on how to provider this dependency. We can do this by customizing the ConfigureServices in the Function class.

protected override void ConfigureServices(IServiceCollection services, IExecutionEnvironment executionEnvironment)
{
  RegisterHandler<FindNationalityHandler>(services);

  // Add registration for HttpClient
  services.AddHttpClient("Nationality");
}

As you can see, since we are using ASP.NET Core constructs like IServiceCollection, we are able to quickly leverage libraries that were built for the Microsoft framework while working on our Lambda function.

We don’t need to register the logging infrastructure as it’s already taken care of.

If you are new to the dependency injection system in .NET, take a look at this guide.

You can find the changes introduced in this step here.

Customizing the logging pipeline

Now that we have enriched our handler with a logger, it’s time to customize the logging pipeline to make sure that the logs are written where we want.

If you are new to the logging system in .NET, take a look at this guide.

In this example, we will be writing our logs to CloudWatch using the Amazon.Lambda.Logging.AspNetCore package, released by the AWS team and used to glue together Lambda and the ASP.NET Core logging subsystem.

Let’s start adding the package.

$ dotnet add package Amazon.Lambda.Logging.AspNetCore

Then, we override the method ConfigureLogging in our Function class to customize the logging pipeline.

protected override void ConfigureLogging(ILoggingBuilder logging, IExecutionEnvironment executionEnvironment)
{
  logging.AddLambdaLogger(new LambdaLoggerOptions
  {
    IncludeCategory = true,
    IncludeLogLevel = true,
    IncludeNewline = true,
  });

  if (executionEnvironment.IsProduction())
  {
    logging.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Warning);
  }
}

In the snippet above, we use the AddLambdaLogger to specify which pieces of each log entry we want to be pushed to CloudWatch. Also, we make sure that we only log entries whose level is higher or equal than Warning. This is to avoid spamming our logbook and make sure unexpected messages are easy to discover.

Like for dependency injection, the fact that the logging configuration relies on established Microsoft components gives us the possibility to quickly switch to any provider that has built support for ASP.NET Core.

You’ll have noticed that our handler was ignored while working on the logging pipeline: this is a clear evidence that our function follows the Single Responsibility Principle by keeping the business logic and the configuration concerns neatly separated.

You can find the changes introduced in this step here.

Introducing external configuration

Next step is supporting external configuration. This is extremely important especially when working with containers that are supposed to be immutable and float from one environment to the next with only the configuration being mutable.

Also in this case, ASP.NET Core offers a very powerful and customizable system that can be leveraged to customize the behavior of Lambda functions.

For our example, let’s say that we want to return only those countries whose probability is greater or equal than a value that we receive from the configuration. For simplicity, I’ll hardcode the value using the in-memory provider but the value can be retrieved from environment variables, JSON or XML configuration files, and more.

If you are new to the configuration system in .NET, take a look at this guide.

First of all, let’s create a record that represents the options that we will use to customize the behavior of our handler.

public record FindNationalityOptions
{
  public double MinimumThreshold { get; init; }
}

Next, let’s modify the handler so that it can receive the customization record and use it.

public class FindNationalityHandler : IRequestResponseHandler<string, Country[]>
{
  private readonly IHttpClientFactory _httpClientFactory;
  private readonly ILogger<FindNationalityHandler> _logger;
  private readonly FindNationalityOptions _options;

  public FindNationalityHandler(
    IHttpClientFactory httpClientFactory,
    IOptions<FindNationalityOptions> options,
    ILogger<FindNationalityHandler> logger)
  {
    _logger = logger ?? throw new ArgumentNullException(nameof(logger));
    _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
    _options = options?.Value ?? throw new ArgumentNullException(nameof(options));
  }

  public async Task<Country[]> HandleAsync(string? input, ILambdaContext context)
  {
    var http = _httpClientFactory.CreateClient("Nationality");

    _logger.LogDebug("Finding the nationality of people named {Name}", input);

    var response = await http.GetFromJsonAsync<Response>($"https://api.nationalize.io/?name={input}");

    return response?.Countries.Where(c => c.Probability >= _options.MinimumThreshold).ToArray() ?? Array.Empty<Country>();
  }
}

Notice how we wrapped the FindNationalityOptions with the IOptions<> interface. This is because we use a relatively newer subsystem for gluing together the Configuration and the Dependency Injection systems.

If you are new to the Options system, take a look at this guide.

Next, we need to customize the Function class so that we can configure the Configuration system. We can do it by overriding its Configure method.

protected override void Configure(IConfigurationBuilder builder)
{
  builder.AddInMemoryCollection(new Dictionary<string, string>
  {
      ["Options:MinimumThreshold"] = "0.05"
  });
}

As mentioned earlier, I’m hardcoding the value but a real-world scenario would see us using a configuration file or environment variables.

Next, we need to instruct the function how to bind the options class to the configuration values. We can do this in the ConfigureServices method.

Before we can do that, we need to install a package that helps gluing the Configuration, the Options and the Dependency Injection systems together. Unlike the other subsystem, the Options system is not automatically added to the .NET projects created using the Class Library template.

$ dotnet add package Microsoft.Extensions.Options.ConfigurationExtensions

Once the package is installed, we can modify the ConfigureServices method as follows.

protected override void ConfigureServices(IServiceCollection services, IExecutionEnvironment executionEnvironment)
{
  RegisterHandler<FindNationalityHandler>(services);

  services.AddHttpClient("Nationality");

  services.AddOptions();

  services.Configure<FindNationalityOptions>(Configuration.GetSection("Options"));
}

Like in the previous paragraphs, if we wanted to change how our function receives its configuration values, we don’t need to alter the logic of the handler.

You can find the changes introduced in this step here.

Deploying into the cloud

Our function is ready to be deployed in the cloud.

To do that, we will use the .NET tool created and maintained by AWS. To install it, run the following command:

$ dontet tool install -g Amazon.Lambda.Tools

This console line utility helps managing Lambda functions supporting operations like deployment, invocation and deletion.

For brevity, I’m purposefully skipping how to authenticate your AWS calls. If this is your first time using AWS command line tools, consider checking this guide.

Even if it supports command line arguments, it’s better to use a configuration file so that we can track the changes using our VCS like GitHub. The file must be called aws-lambda-tools-defaults.json.

{
  "function-runtime": "dotnet6",
  "function-name": "find-nationality",
  "function-memory-size": 128,
  "function-timeout": 30,
  "function-handler": "FindNationalityFunction::FindNationality.Function::FunctionHandlerAsync"
}

The file above is almost self-explanatory but you can find more information in the Lambda Developer Guide.

A brief mention to the function-handler property is due. It’s value is composed so that it specifies:

• The assembly name
• The fully qualified name of the Function class
• The name of the handler method. In the case of functions created with Kralizek.Lambda.Template, it’s always FunctionHandlerAsync.

Once we’ve created the configuration file, we can deploy our Lambda function by running the command:

$ dotnet lambda deploy-function

Since we didn’t specify any, the tool assumes that we want a new execution role to be created and prompt us to type the name of the new role and which policy we want to attach to it.

For this sample, I’ve used the name find-nationality-role and attached the policy AWSLambdaExecute but your would normally consider create a bespoke policy to make sure that your function runs with the minimum required permissions.

...
Creating new Lambda function
Enter name of the new IAM Role:
find-nationality-role
Select IAM Policy to attach to the new role and grant permissions
    ...
    3) AWSLambdaExecute (Provides Put, Get access to S3 and full access to CloudWatch Logs.)
    ...
    15) *** No policy, add permissions later ***
3
Waiting for new IAM Role to propagate to AWS regions
...............  Done
New Lambda function created

Now that the function is created, we can invoke it and test that it works as expected. We use the Lambda .NET tool to invoke the function by specifying the function name and its payload.

$ dotnet lambda invoke-function --function-name find-nationality --payload "Renato"

And here is the output of the command we executed.

Amazon Lambda Tools for .NET Core applications (5.5.0)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet

Payload:
[{"country_id":"BR","probability":0.205},{"country_id":"PT","probability":0.09},{"country_id":"PE","probability":0.078},{"country_id":"IT","probability":0.074}]

Log Tail:
START RequestId: 70728520-d012-44ea-979c-4c0d650964e7 Version: $LATEST
2022-11-01T13:33:41.498Z        70728520-d012-44ea-979c-4c0d650964e7    [Information] Kralizek.Lambda.Function: Invoking handler
2022-11-01T13:33:41.499Z        70728520-d012-44ea-979c-4c0d650964e7    [Information] System.Net.Http.HttpClient.FindNationalityHandler.LogicalHandler: Start processing HTTP request GET https://api.nationalize.io/?name=Renato
2022-11-01T13:33:41.499Z        70728520-d012-44ea-979c-4c0d650964e7    [Information] System.Net.Http.HttpClient.FindNationalityHandler.ClientHandler: Sending HTTP request GET https://api.nationalize.io/?name=Renato
2022-11-01T13:33:41.603Z        70728520-d012-44ea-979c-4c0d650964e7    [Information] System.Net.Http.HttpClient.FindNationalityHandler.ClientHandler: Received HTTP response headers after 104.5277ms - 200
2022-11-01T13:33:41.603Z        70728520-d012-44ea-979c-4c0d650964e7    [Information] System.Net.Http.HttpClient.FindNationalityHandler.LogicalHandler: End processing HTTP request after 104.6293ms - 200
END RequestId: 70728520-d012-44ea-979c-4c0d650964e7
REPORT RequestId: 70728520-d012-44ea-979c-4c0d650964e7  Duration: 106.28 ms     Billed Duration: 107 ms Memory Size: 128 MB     Max Memory Used: 82 MB

You can notice that the output includes:

• The payload of the response
• The tail of the log our function generated
• A report that summarizes the function execution

As you can see, our function returned successfully and in its payload we can see the list of countries that the input name might belong to sorted by the probability.

You can find the changes introduced in this step here.

Here come the templates

We were able to create a function that clearly divides the business logic from cross-cutting concerns like dependency injection, logging and configuration while being able to reuse well-established ASP.NET Core libraries and concepts. But most of the work we did can be automated so that developers can spend their time where it benefits your stakeholders the most: the logic of the handler.

This can be done by using the template parts of the AWSLambdaSharpTemplate project.

To install the templates use the CLI command

$ dotnet new -i Kralizek.Lambda.Templates

This command will make it so that the NuGet package containing the templates is downloaded and installed.

The package contains six templates:

• A template for creating minimal event functions
• A template for creating minimal request/response functions
• A template for creating event functions with some additional boilerplate
• A template for creating request/response functions with some additional boilerplate
• A template for creating functions that handle SNS notifications
• A template for creating functions that handle SQS messages

We will focus our attention on the second template, that helps us create a function equivalent to the one we just created in few keystrokes.

To create a function, let’s run the command below.

$ dotnet new lambda-template-requestresponse-empty -o FindNationalityFunctionV2

The command above will create a function in the directory FindNationalityFunctionV2 (see commit).

Once the function is generated, make sure of:

• installing the additional packages from the previous paragraphs (see commit)
  • Microsoft.Extensions.Http
  • Amazon.Lambda.Logging.AspNetCore
  • Microsoft.Extensions.Options.ConfigurationExtensions
• replace the included ToUpperStringRequestResponseHandler class with our own FindNationalityHandler and copy the additional types we have added (see commit)
• modify the Function class so that logging, configuration and dependency injection systems are properly configured (see commit)
• check the included aws-lambda-tools-defaults.json file to make sure everything is correctly in place

The new function is ready to be deployed to AWS and invoked like we did for our hand-crafted one.

Annotation Framework

Those who follow AWS developers know that they are working on a new programming model for Lambda functions: the Annotation Framework.

The Annotation Framework is still in preview and aims at leveraging source generators to simplify the development of Lambda functions to something that looks like this

[LambdaFunction]
[HttpApi(LambdaHttpMethod.Get, "/add/{x}/{y}")]
public int Add([FromServices] ICalculatorService calculatorService, int x, int y, ILambdaContext context)
{
  context.Logger.LogInformation($"{x} plus {y} is {x + y}");
  return calculatorService.Add(x, y);
}

[Amazon.Lambda.Annotations.LambdaStartup]
public class Startup
{
  public void ConfigureServices(IServiceCollection services)
  {
    services.AddSingleton<ICalculatorService, MyCalculatorService>();
  }
}

I firmly believe that in the future the Annotation Framework will supercede the need for projects like AWSLambdaSharpTemplate, but right now the focus of the AWS team is towards functions backing HTTP/REST API services with API Gateway as front.

So I still think there is a need for this project and its templates. This is way I decided to write this blog post even if I started this project back in 2016 and released the 1.0.0 on March 25th 2017.

Recap

With the introduction of Lambda functions, AWS changed the world in a way they didn’t even anticipate. What was designed as a simple glue between the many services they offer, it has become the backend for very advanced and robusts systems.

Unfortunately the humble origins of Lambda make them very hard to maintain in a complex system. This is why frameworks like SAM have been created over the years.

In this post we’ve seen how cross-cutting concerns like dependency injection, logging and configuration management can be kept outside of the functions’ business logic and how we can borrow libraries and tools designed for a framework like ASP.NET Core. But these powers comes at a cost in terms of performance, so architects will have to decide whether the additional maintanability is worth the price. In most of the case, I think it is.

If the price were to be considered acceptable, we’ve seen how the provided dotnet new templates can be used to quickly create new functions and focus immediatedly on the business logic.

In a future post I’ll show how we can leverage the templates contained in the NuGet package Kralizek.Lambda.Templates to create functions that can handle events and specifically ones that can handle SNS notifications and SQS messages.

Support this blog

If you liked this article, consider supporting this blog by buying me a pizza!