AWS recently announced Go support for Lambda, giving developers more choice over how their functions are written.
In an attempt to kick the tires of the new runtime, I found myself rummaging around the open source library required when writing Lambda functions in Go, and was delighted to find a glimpse into what happens when your function is invoked.
This post is a brief tour of what I’ve gathered, and describes a simple way of invoking your function in a local environment.
tl;dr: AWS Go Lambdas are invoked using net/rpc over TCP and make use of the Go standard library. You can “simulate” a lambda being invoked, which could be useful for integration tests or sanity checking, see below for an example
A Go lambda needs two things to run
- A handler to handle requests
- A main function that calls
lambda.Start(...)with your handler as an argument
lambda.Start(ToUpperHandler) is where the magic happens, it is a blocking call that will block until the process is killed or an exception is propagated that cannot be handled.
This gives us our first clue that AWS isn’t simply just running your go binary every time a function is invoked, it’s actively listening for requests and passing them to your handler.
Doing it this way has a few performance benefits, it allows you to set up expensive, thread safe dependencies up front so they are warm if your function is called more than once1.
Taking a look at the code under the hood, we can see that it:
- Creates a TCP server listening on a port defined by the environment variable
- Uses the net/rpc package to handle
Invokerequests from remote clients
- Uses the context package to store and manage state
- Uses the encoding/json package to perform SerDe for objects you pass to and from your function
The use of net/rpc is really neat in a way, just plain old Go standard library code.
The two methods you can call via RPC are in function.go and they allow remote clients to:
Pingrequests by sending a
*messages.PingRequestobject, and unsurprisingly this does exactly what it says on the tin. I’m assuming AWS use this to check liveliness of your function and whether it is still reachable wherever they are hosting it.
Invokerequests by sending a
- I’m wondering what the
Deadlineattribute is for. At one point in the execution path, they use whatever this is set to with the
context.WithDeadlinefunction, which leads me to believe this might be the timeout you have configured against your lambda.
- I’m wondering what the
These functions will be called by something in AWS that is managing the lifetime of an invocation.
To help visualise this, I drew this crude, overly simplified diagram that describes the above interactions. Obviously the AWS Lambda service has a lot of components and infrastructure that we are not privy to, but I think conceptually it’s mostly right.
As the lambda is just listening on a port over TCP, it’s pretty simple to test the above behaviour locally.
By forcing the lambda to run on a known port2
_LAMBDA_SERVER_PORT=8001 go run lambda.go
And then writing a client to submit a
InvokeRequest to it, you can successfully execute the function end-to-end, which might be useful for integration testing or whatever.
$ go run client.go 8001 "\"daniel\"" "DANIEL"
I’ve created a small library imaginatively called go-lambda-invoke that wraps up this logic, meaning you can just make the following call in your code
response, err := golambdainvoke.Run(8001, "daniel")
It probably has limited uses, in most cases just writing plain old unit tests for your logic should be sufficient, rather than testing the scaffolding AWS erects around it. However I could see it being useful if you want to test that you’ve built a valid
linux binary and perform some pre-deploy sanity tests or something.
Speculating over what AWS are doing under the hood with Lambda is a pastime that has circled the internet ever since it launched, and this post doesn’t really reveal much to answer that question. Is it containers? Who knows. Probably.
But I think looking at how the Go programming model works, a plain old TCP server that allows RPC clients to connect to it, gives us at least a small glimpse into the interactions AWS are performing with your application.