Tasting some AWS Lambda Curry !

Lambda Curry!
Serve up your own event-driven recipes from Amazon's Cloud Cooker!

In this post I’ll  share my thoughts on AWS Lambda – a new offering from Amazon Web Services that is currently in technical preview.  I’ll  take you through the good and bad parts and explain why so there is so much excitement and hype in the community about this new offering.

I’ve created a small open source project on github to demonstrate how you can use AWS Lambda to synchronize files between an S3 bucket and a personal dropbox account. I’m sure there will be a whole cookbook of recipes appearing in community once the service goes to general release.

Lambda is currently in a closed technical preview, I’ve been fortunate enough to get early access to Lambda and I’ve spent the past few days writing and deploying some Lambda functions to the EU Ireland region.

Announcing_Lambda_reInvent2014_lg

(above) AWS CTO Werner Vogels announces AWS Lambda @ AWS reinvent 

So what is AWS Lambda?

AWS Lamdba is a new service  that runs your code (node.js functions) in response to events that happen in the AWS cloud. You only pay for the compute time required to run your code and billing is metered in increments of 100 milliseconds making it a very economic offering. This is a pure platform as a service play from Amazon , you don’t need to spin up and manage any infrastructure on AWS to run your lambda functions.To paraphrase the great Nigel Tufnel from Spinal Tap if platform as a service is a volume dial from one to ten then AWS Lambda goes to eleven!

Spinal_Tap_-_Up_to_Eleven

(above) The volume knobs of Tufnel’s Marshall amplifier went up to eleven 

Event-Driven Programming is a paradigm which has been around for many years now, in which the flow of the program is determined by external events.  Most desktop applications are event drive – the button click or key press event triggers a specific behavior in the program. Database triggers are another great example of event driven programming. The event of a row being updated in a database table can trigger the execution of custom code.AWS Lambda exposes an event driven programming model to the AWS cloud. For now the number of events on offer is quite small but this will grow over time. Currently you can tap into events such as

  • A new object was created in an S3 bucket ( PUT / POST / COPY operations)
  • DynamoDB table change tracking – any changes made to the table are tracked on a rolling 24-hour basis. You can retrieve this stream of update records with a single API call.
  • New messages arriving in an AWS Kinesis stream

You can also expose your own custom events and call these from as you like from you own application code.

Typically lambda functions will respond to a trigger within milliseconds of the event occurring and from my experience to date the response times have been in the teens.

So what’s the big deal with Lambda ?

Well prior to this service offering if you wanted to respond to an event like syncronising an S3 object with a dropbox folder you’d need to build something like this:

  • When you add a new file to an S3 bucket you’d then place a message on a dedicated SQS message queue.
  • You’d deploy a fleet of EC2 workers to poll the queue and process the messages
  • Finally you’d tweak the auto scale settings to ensure the workers can scale accordingly

With Lambda all this goes out the window, your lambda functions execute in a fully managed, shared compute environment.

How much does it cost?

Amazon has priced the service based on the number of Lambda function requests served and the compute time needed to run your code. The basic formula is that you pay $0.20 per 1 million requests, plus $0.00001667 per GB-second of compute time used, although the first million requests per month are free.There’s also a “free tier” that includes 1 million requests per month and 400,000GB-seconds of compute. Emm, simple eh ? well just in case you are confused Amazon have provided a few uses cases here to give you a feel for the prices

 

Programming Model

The entry point for your lambda function expects 2 parameters.

  • The event parameter contains all the data relating to the event, in this case its an an object being created in an S3 bucket.
  • The context parameter is used to notify lambda that your function has completed successfully.
exports.handler = function (event, context) {

    var srcBucket = event.Records[0].s3.bucket.name;
    var srcKey    = event.Records[0].s3.object.key;

    getToken().then(function (token) {
             getFile(srcBucket,srcKey)
            .then(function (fileStream) {
                  // sync  file
                  context.done(null, "file sync complete");
            })
            .catch(function (error) {
                console.log(error);
            })
    }).done();
};

 Anatomy of a Deployment Package

A deployment package consists of a zip folder containing your node code and any node module dependencies. As with any AWS resource you have the usual  fine grained access controls via  IAM roles and permissions.

Deploying your Code

You can upload you deployment package straight from the AWS console or use the AWS CLI. If you are doing any serious development work check out this grunt plug-in that lets you build and deploy your packages through the grunt task runner.
Author disclosure: I am a contributor to the grunt-aws-lambda project on github.

 

Monitoring and Health checks

Out of the box lambda comes with cloudwatch metrics including request duration, request count and execution error counts. It would be great to see the error handling functionality grow to include things like retry logic and different error notification options .
cloudwatch

What I would like to see

  • Many more events exposed, this will surely come with time
  • Time based triggering of custom events
  • Reserved Instances for executing your lambda functions. This will satisfy customers that are worried about their code executing in a multi-tenanted environment and guarantee consistent performance and response times.
  • Support for more languages. Don’t get me wrong, I’m a big fan of node.js but AWS Lambda should not be limited to JavaScript only.
  • A great eco-system to share AWS recipes with other AWS customers.
  • Support for NPM at run time , currently you need to zip up and deploy all the node modules that you lambda function relies on
  • Pre-canned retry and error handling patterns to choose from
  • Lambda is screaming for some great IFTTT integrations !
  • Some nice DevOps integration to enable AWS infrastructure deployments from a valilla S3 bucket upload

Lambda is still in technical preview, if you’d like to get your hands dirty and have a play head on over to the AWS Lambda Preview Site and register your details…

http://aws.amazon.com/lambda/preview/

 

Useful resources

Video – AWS Lambda Announcement at Reinvent

Github S3 to Dropbox file synchronization

AWS Official Documentation

AWS Lambda Walkthrough Command Line Companion

Lambdash – AWS Lambda Shell

Grunt AWS Lambda task runner

 Footnotes

 

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s