Apex Tutorial: Building development-friendly APIs with AWS Lambda and API Gateway

Hi, it’s 李孟 (Lee Meng) in Marketenterprise.

Today we will build a development-friendly RESTful API with AWS LambdaAPI Gateway and Apex.

For those who don’t familiar with Apex, it’s a tool let you build, deploy and manage AWS Lambda functions with ease. We will talk more about it later.

 

Target audiences

This tutorial is aimed to help developers who want to build and manage Lambda functions with ease and those who not only want to build naive APIs but APIs which are able to handle development and production request at the same time.

 

So.. what do you mean by “development-friendly”?

In short, development-friendly environment = version control-enabled environment.

In my opinion, in order to develop smoothly with AWS Lambda and API Gateway, the ideal developing environment should provide the following three features:

  1. Versioning, multiple versions of AWS Lambda function for both development and production usage.
  2. Isolation, every version of AWS Lambda function should possess its own environment by using different environment variables in order not to pollute other versions’ environment like database.
  3. Transparency.  API Gateway should be able to automatically invoke different version of AWS Lambda function according to its stage and unaware of the internal change of Lambda function.

Ideal development environment

After all these features are provided, one can imagine a development flow like this:

ideal-develop-environment-flow-with-aws-lambda-and-api-gateway

As showed above, developer(s) will first create / refine the application logic (step 1), and test the application using development API (step 2). After fully test and make sure the new features work and there exist not bug, developer may decide to publish the latest version of AWS Lambda into production (step 3). And finally, all these wonderful new features of the API is now available to the users and with them without noticing the internal change of the application (step 4).

This development flow could occur multiple times especially when the developer(s) are working on Agile software development like Scrum.

 

All right, then tell me how to achieve it?

In order to build the above development environment, we will need these tools / references:

As you will see, we are going to bring all these together step by step and build our version-control environment.

Although one can achieve what Apex can do by using AWS Lambda console or AWS-CLI directly, we will use Apex in this tutorial. Just a reminder, Apex is a tool allow you to deploy and manage AWS Lambda functions with ease. If you’re going to embrace server-less architecture, it’s a must-learn tool in my opinion. ( or tell me some better alternatives! )

 

Steps that we will walk through in this tutorial

  1. Create a Apex project with one naive Lambda function inside
  2. Create environment settings for different environments and access environment variables
  3. Create both production / development version of the Lambda function using different environment setting
  4. Integrate with API Gateway
  5. Have a tea and enjoy what you have done!

 

Before we start

If you want to use all the demo code showed below, be sure that you have ApexAWS Command line Interface and Python installed and of course, an valid AWS account. So let’s get started!

 

Get your hand dirty part 1 – Create a Apex project

For those who are not familiar with Apex, you can first create a new project folder and try

to initial a brand new apex project inside the folder.

Then you should be able to see this welcome message in your cli:

after enter your project name and description, your folder structure should look like this:

Apex manage your Lambda functions below “functions” folder and put each function’s source code in separate folder in their function name. As you can see, “hello” is the first Lambda function Apex created for us, and its source code is written in index.js.

Because I’m not familiar with Node.js, so I will replace index.js to lambda_function.py so the folder structure will become:

And the code for the Python file is as below:

 

In order to let Apex run your Python file as Lambda function, you also have to add runtime and handler setting in your project.json as below:

That’s it. And if you type

you should be able to see the function information, which indicate that is haven’t been deployed.

Now type

to deploy our Lambda function to AWS. (notice that Apex automatically create a version for every deploy we make)

After that, you should be able to invoke it and see the result.

You can also open AWS Lambda console to test your first Lambda function directly.

aws_lambda_console

 

Get your hand dirty part 2 – Create environment settings for different environments and access environment variables

After deploying our first Lambda function, we also want to apply different settings for production and develop environment to ensure that nothing will be ruined in production environment when we are developing. For example, you would not want to insert dummy user data into your production customer database when you’re developing. ( I guess )

environment_variables

In order to do that, one can edit the environment variables directly on the AWS Lambda Console thanks to recent release. But it would still be a pain if you have lots of variables to manage.

In alternative, Apex allow you to have multiple environment setting files as long as their names are as the form of project.ENV.json. For example, if you want to have a setting for production environment, you can name it as project.prod.json. And you can leave the default project.json as your develop environment setting file.

At this point, the content of project.prod.json is still the same as the project.json. But we’re going to add environment variables into them. First of all, open project.json and add a ENVIRONMENT variable with value “dev” under the environment key as below:

Then add the same variable in project.prod.json but with a different value:

Now, run the following command to deploy hello function using project.json (dev)

Open AWS Lambda Console. You should be able to see a new environment variable ENVIRONMENT is being created.

new_environment_variable

We have defined our own environment variables, but how do we access these variables in our Lambda function?

In Python, we can refine our hello function to someway like below. Notice that os module is included.

And now try to invoke our hello function:

Now we can access the environment variables, how can we deploy our hello function with another environment setting file?( that is, project.prod.json )

We can using the env flag

The following command will make Apex try to find a file name project.prod.json and use it to deploy our hello function. And as you can see,  the value of  ENVIRONMENT is changed to “prod”

environment_variable_prod

Again, try to invoke the function again:

Notice that, again, Apex will automatically publish a version with every deployment we make.

 

 

Get your hand dirty part 3 – Create versions of the Lambda function

Now we’re able to deploy our hello function with different setting and publish new versions, but it’s hard for us to remember all the difference between versions, so it’s a better idea to create AWS Lambda alias to manage our Lambda versions.

versions

An AWS Lambda alias is simply a pointer point to a specific version of the Lambda function. The common usage pattern will be setting your DEV alias to the latest version of Lambda function ($LATEST), and when you done developing, publish a new stable version and make a PROD alias point to that version. Version can’t not be changed after publish, so it remain unchanged.

Now let’s create a dev alias point to the $LATEST version of our hello function by using AWS-CLI:

And you should see the response like this:

Now type

You should be able to see that the dev alias is being created and is pointing to version $LATEST (which mean the latest version)

Now, suppose that we’re already satisfied with the current hello function and want to make it into production, what we need to do is to deploy it again but with project.prod.json setting

and make a prod alias point to that version. In my case, the version with production setting is version 5 so I make prod alias point to version 5

And you should be able to check the response.

Finally, to check what we have now, type

At this point, we already have two defined alias of our hello function named dev and prod, pointing to $LATEST and version 5, respectively.

 

Get your hand dirty part 4 – Integrate with API Gateway

In this part, we want to create a API that will invoke corresponding alias of our hello function in respect to API’s stage. So that if we access /dev/hello, it will invoke dev alias of hello function, and while we access /prod/hello, it will invoke prod alias of our function.

In order to do that, let’s create a new API called naive on API Gateway Console:

create_new_api

Next, click Actions button to create new Resource call “hello”:

hello

Next, create a new GET method and choose Integration type as “Lambda Function” and corresponding Lambda Region. And for the Lambda function name, enter

The syntax here is <lambda_function_name:${stageVariables.<stage_variable_name>}. “alias” is a API Gateway stageVariable key that we will create later. If someone access this API, API Gateway will check its “alias” key’s value to make sure which alias of hello function to invoke.

And you will be notified with this message. AWS is telling you that you have to manually give permission to API Gateway to invoke your Lambda alias. Copy the whole message for later use.

apigateway_notify2

Then click Actions button again to deploy the API. In our case, we need to deploy two stage and name them as dev and prod.

prod

Now go to Stages page and click Stage Variables tab to create “alias” key for both dev and prod stage.

Remember to use “prod” and “dev” as the value for prod stage and dev stage respectively.

stagevariables

We’re not done yet. Before access our Lambda alias using API Gateway, we have to grant API Gateway with the permissions to call both prod and dev alias. Cope the notification message showed before and change the Lambda alias part.

After that, we can test our API by click TEST button and enter both “dev” and “prod” for “alias” Stage Variables:

api_test_prod

As some of you may have noticed, the dev Lambda alias is returning the same result as prod Lambda alias. It’s because dev alias is pointing to the $LATEST version of our hello function and we just deployed the $LATEST version with production environment ( project.prod.json ).

In order to let dev alias point to development version, we need to deploy our hello function again with project.json.

And now try invoke dev Lambda alias again:

api_test_dev

Congratulation! You make a version-aware API using both AWS Lambda and API Gateway!

Although the example we provide here is naive, you can use this concept for a more wider application like make your Lambda alias access to different  database for development and production without messing thing up.

 

Note

In order to update your prod Lambda alias when you come up with something new, you can use

 

Conclusion

For me, this workflow is very attractive and has some benefits:

  • Keep developing without worrying about affecting the existing production version of AWS Lambda function. Existing users is unaffected by the development change.
  • Cleaner application logic. Developer save the time to write code to decide whether this Lambda function is invoked by Development API or Production API and change environment variables.
  • API Gateway will automatically find the corresponding Lambda versions so the change on the version can be tested and used immediately.

 

Left your comments below or contact me at leemeng@marketenterprise.co.jp.

Anyway, happy developing your application with version control 🙂