Serverless Code with Amazons AWS and Claudia

I recently have been on a serverless kick. What is that you ask? Well, it means running code in the cloud without managing any server resources or having a server operating system that is always running. Even better, you only pay for resources when your code is executing. Hosting solutions have been around for a long time, but they typically run all the time, and you pay for that (also you might have to manage and update your server). Serverless also means you should break your code into smaller micro-services so it is easier for them to run independently of each other, and you can upgrade one component without affecting the whole.

Amazon Web Services (AWS) provides an excellent setup for doing this. They sell containerized services which are setup as elastic to automatically expand based on usage criteria. Their function as a service is called Lambda where you can write code and have it execute when needed. Lambda code can run for a bunch of different scenarios, here are two:

• In response to an S3 event (someone or a program uploads a file to your S3 bucket)
• An API Gateway call (AWS provide API services as well) for your application that needs to call a REST API, then call this Lambda function

Lambda also is billed in micro increments, but they give you a significant amount for free (a million milliseconds of processing I think).

Claudia is a tool that simplifies creating an API endpoint and setting up the Lambda function behind it. So if you want to setup domain.com/api/users which displays a list of users, this is the tool you can use to create all the AWS items.

The use case is if you are creating a React app and need to connect to an API for the back-end of your application. Typically the express library is used here, but this can replace express. You can also use Claudia to deploy express if you want and have Lambda run your express app on demand; you need to use Amazon’s aws-serverless-express library as a wrapper.

To start with you need to install Claudia (we assume here that you already have node 4. or 6.x installed with npm, Lambda is node 4.x currently):

npm install claudia -g

Then you need to configure your AWS credentials and create an account with proper IAM access. IAM is always the tricky part for AWS things, I have written about AWS credentials before. Let’s cover the important stuff. Here are Amazon’s docs that go into more details: setting-credentials-node

1. Login to our trusty IAM console: https://console.aws.amazon.com/iam/home
2. Create a new user, whatever name you like; you do not see the username much just the keys
3. We need to give it access to create Lambda functions, IAM Roles, and API Gateways. NOTE: we are giving it full permission to these areas, so if you have other production things running on this account, I suggest creating your policies and restricting the services down further. Better yet create a new AWS account only for these purposes (you get the free tier again too!). Don’t mix test and production environments; you have been warned.
   1. When creating the account, here are the roles you must add (these are built-in AWS policies, so just check the box next to them):
      1. AWSLambdaFullAccess
      2. IAMFullAccess
      3. AmazonAPIGatewayAdministrator
4. Once you create the account, write down the Access Key, and the Secret Keys AWS gives you
5. Now we have our keys; we need to update where are credentials are stored. There are a couple of places you can put them, in your home directory (on Unix) under a file called ~/.aws/credentials or in my opinion an easier way, environment variables. Recently environment variables have come under fire, so it is up to you if you think they are secure or not. Having a file read with the credentials is typically better.

If you want more in depth than just hello, Claudia has great tutorials on their site: claudiajs.com/tutorials

We are going to walk through a simple Hello World that uses Lambda and the API Gateway service to access that Lambda function. This way we can connect to an API and have our Lambda function return data without having to access it directly.

The first step, let’s initialize a new project with npm and install our claudia api builder package as a local development only dependency (-D):

mkdir claudia-test
cd claudia-test
npm init
npm install -D claudia-api-builder

Now let’s add the necessary code to see if this works (index.js). Open your favorite text editor and paste:

index.js
var ApiBuilder = require('claudia-api-builder');
var api = new ApiBuilder();

module.exports = api;

api.get('/hello', function () {
  return 'hello world';
});

If you want to use newer JavaScript syntax, then you need to transpile it with babel. This example does not need it, but if you start adding anything else in you are going to:

# here we are install babel as a global, you can also make it a local dependency and use -D instead
npm install babel -g
babel index.js --out-dir bin

Now let’s try and upload it to AWS! The first step is to export our environment variables, so claudia has access to our keys. There are a few ways to do this (like everything). If you like using the credentials file, then you do not need to worry about setting environment variables in this section.

1. You can export them in a shell script each time you run this, i.e. export xx or set xx
2. You can add them to your package.json file as part of an npm command
3. Have them added to your login script, for bash it is .bashrc, so when you create a shell they are already there and exported

Options 1 and 2 have the issue where you will most likely publish them in your git repo. If your repo is private, you are OK, but don’t make the mistake of posting your api credentials publicly!

For now, I am going with option 2, so we can have a handy npm command that does everything for us. Let’s edit your scripts portions of the package.json, add a line similar to this (it is very long so make sure to scroll over):

"scripts": {
    "transpile" "babel index.js --out-dir bin",
    "claudia": "npm run transpile && AWS_ACCESS_KEY_ID=XX AWS_SECRET_ACCESS_KEY=XX claudia create --timeout 30 --region us-west-1 --api-module bin/index",
    "claudia-update": "npm run transpile && AWS_ACCESS_KEY_ID=AXX AWS_SECRET_ACCESS_KEY=XX claudia update --timeout 30 --api-module bin/index",
    "claudia-destroy": "AWS_ACCESS_KEY_ID=AXX AWS_SECRET_ACCESS_KEY=XX claudia destroy"
  },

You can see I also added a transpile command to make that easier. Now we can run “npm run transpile” to create our files anytime. The rest of the commands run this automatically each time we create or upload to claudia.

So what is that second line doing? AWS_ACCESS_KEY_ID=XX  and  AWS_SECRET_ACCESS_KEY=XX  are setting environment variables that are only active when this particular shell is open. So when you run that command, your IDE or whatever you are using to run npm, set those. Claudia sees the variables and uses those keys to make its API gateway and Lambda functions.

The claudia command to create a new API gateway and Lambda combo is “create.” I am also specifying the timeout for the Lambda function at 30 seconds (the previous project I was playing with needed a longer timeout than the default 3 seconds). If your function takes longer than 3 seconds, it closes without returning anything. We are also telling it which AWS region to put it in, and lastly where the modules are stored and filename. They are in the bin directory since that is where babel outputs them. The filename is index (we do not need to specify the .js).

So let’s run it and see what happens:

npm run claudia
###################### - it will do a bunch of stuff, and if there are no errors, your API gateway should be shown:
saving configuration
{
  "lambda": {
    "role": "claudia-test-executor",
    "name": "claudia-test",
    "region": "us-west-1"
  },
  "api": {
    "id": "111111111111111",
    "module": "index",
    "url": "https://111111111111.execute-api.us-west-1.amazonaws.com/latest"
  }
}

It worked!

Now in this example, the URL we can check: https://111111111111.execute-api.us-west-1.amazonaws.com/latest/hello

NOTE: Make sure to add “/hello” to the URL. Otherwise, this returns the message “Missing Authentication Token” which is a cryptic way of saying there was no default URL for it to run. You can see from above, the API we registered was ‘/hello’ with the api.get function.

Head to your URL and make sure Hello World shows. If it is, success! That means we have a front-end API that is connecting to a Lambda function each time you hit it, exciting stuff!  AWS gives us extreme scale-ability, and it is free for quite some time based on usage.

The next command:

"claudia-update": "npm run transpile && AWS_ACCESS_KEY_ID=AXX AWS_SECRET_ACCESS_KEY=XX claudia update --timeout 30 --api-module bin/index",

This command updates our claudia functions after we have initially deployed them. On the first run, claudia creates a file called “claudia.json” which has information about which API module it has created. You should be OK saving this to git, the key and secret keys are what allow us access to modify the API and Lambda function.

The claudia.json file looks something like this:

{
  "lambda": {
    "role": "claudia-lambda-aws-executor",
    "name": "claudia-lambda-aws",
    "region": "us-west-1"
  },
  "api": {
    "id": "1111111111",
    "module": "bin/index"
  }
}

This configuration specifies the Lambda function that claudia creates (the function is called claudia-lambda-aws), and the id variable is your API endpoint internal ID. You can now login to the AWS console and take a look at your functions! There is a section for the API Gateway, which we created a simple GET URL, and there is the Lambda function.

Say your function is having issues, you can look at the output generated by Lambda pretty quickly. Add some console.log statements, and those get sent out to this as well.

In the AWS Console, open the Lambda section, then click on your function (claudia-lambda-aws). Click the monitoring tab.

Now click on “View logs in CloudWatch.” The new page is a list of all the times your Lambda function has run. You can then click and look at any log messages your function spat out. It tells you how long they ran, and what AWS is billing you as well.

The final command removes the API gateway and Lambda functions we have created:

npm run claudia-destroy

Now concur the serverless world!

• Amazon AWS