AWS API Gateway By Example

John Tucker

Published in

codeburst

13 min readApr 5, 2020

Explaining AWS API Gateway concepts through a concrete example.

Why This Article?

I learn best by example and I expect others do too. The example, however, also needs to be sufficiently robust and be explained in a way that I learn the core concepts. It is the latter that I found lacking in the AWS documentation; thus this article.

Why Use AWS API Gateway?

This is the second question in the AWS API Gateway — FAQs. The most compelling feature to me is metering; it is why I am using AWS API Gateway:

Metering. API Gateway helps you define plans that meter and restrict third-party developer access to your APIs. You can define a set of plans, configure throttling, and quota limits on a per API key basis. API Gateway automatically meters traffic to your APIs and lets you extract utilization data for each API key.

The Example’s Requirements

To keep things simple, we are going to create, you guessed it, a todos application with the following endpoints:

GET /todos: Returns an array of todos:

[{
  "Id": "016bdc99-ffcf-47fa-bcdc-e352456b52e2",
  "Name": "Eat lunch"
}, {
  "Id": "87b7576e-b639-4a6d-ac78-12a7a3008b38",
  "Name": "Go for walk"
}]

POST /todos: Provide a name:

{
  "Name": "Do sit ups"
}

It creates and returns a todo:

{
  "Id": "1f2be9ee-03b1-45fe-90d2-b73e27561e9b",
  "Name": "Do sit ups"
}

DELETE /todos/{id}: Provide a todo’s id; it deletes it and returns the id:

{
  "Id": "1f2be9ee-03b1-45fe-90d2-b73e27561e9b"
}

As for errors, they all can return a 500 (Internal Server Error). The POST endpoint can also return a 400 (Bad Request) if you do not properly supply the name. The DELETE endpoint can also return a 404 (Not Found) if there is no todo with the provided id.

Finally, we layer on the developer API key; each endpoint also requires a valid API key supplied on a x-api-key HTTP header. If not present or valid, the APIs will return a 403 (Forbidden). Also, the todos are stored such that each is associated with an API key, i.e., each developer gets their own set of todos.

The Backend

First we need to understand that AWS API Gateway does not implement the logic behind the endpoints; it simply maps them to backend elements that do. In our case, we will use three AWS Lambda functions (Todos, TodoCreate, and TodoDelete); each persisting the todos in a DynamoDB table (Todos).

The Todos table has the following structure:

We will store the developer API key in the IdentityId partition key and a random unique identifier in the Id sort key.

note: Remember, as DynamoDB is a NOSQL database, we do not specify the Name attribute in the table structure; we rather supply it through code.

The code for the three Lambda functions are available for download. While the specifics of the code is outside the scope of this article, here are the key aspects:

Each function requires an environment variable, APP_REGION, supplying the region, e.g., us-east-1, for the Todos table
Each function requires a key attribute on the event parameter; this key is used to form the unique IdentityId partition key to associate todos to each developer; here we prefix the IdentityId value with the string developer: followed by the key (to indicate that these are developer keys)
The TodoCreate function additionally requires a Name attribute on the event parameter
The TodoDelete function additionally requires an Id attribute on the event parameter

Additionally, each Lambda function is associated with an IAM role that requires additional permissions, e.g., supplied by an inline policy.

Todos

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "dynamodb:Query"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:143287522423:table/Todos"
        }
    ]
}

TodoCreate

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:143287522423:table/Todos"
        }
    ]
}

TodoDelete

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DeleteItem"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:143287522423:table/Todos"
        }
    ]
}

Creating the API (Gateway)

This, and the following sections, involve using the API Gateway feature in the AWS Console.

With the backend complete, we start by creating an API (Gateway); one of three options:

API Gateway REST API
A collection of HTTP resources and methods that are integrated with backend HTTP endpoints, Lambda functions, or other AWS services. You can deploy this collection in one or more stages. Typically, API resources are organized in a resource tree according to the application logic. Each API resource can expose one or more API methods that have unique HTTP verbs supported by API Gateway.
API Gateway HTTP API
A collection of routes and methods that are integrated with backend HTTP endpoints or Lambda functions. You can deploy this collection in one or more stages. Each route can expose one or more API methods that have unique HTTP verbs supported by API Gateway.
API Gateway WebSocket API
A collection of WebSocket routes and route keys that are integrated with backend HTTP endpoints, Lambda functions, or other AWS services. You can deploy this collection in one or more stages. API methods are invoked through frontend WebSocket connections that you can associate with a registered custom domain name.

— Amazon — Amazon API Gateway concepts

First, as we not using WebSockets, our choice is between the REST and HTTP options. Looking at the comparison, Choosing between HTTP APIs and REST APIs, it is clear that REST gives us more flexibility (including the Usage Plans feature that supports metering).

Next, we need to choose the Endpoint Type:

Edge-optimized API endpoint
The default hostname of an API Gateway API that is deployed to the specified Region while using a CloudFront distribution to facilitate client access typically from across AWS Regions. API requests are routed to the nearest CloudFront Point of Presence (POP), which typically improves connection time for geographically diverse clients.
Private API endpoint
An API endpoint that is exposed through interface VPC endpoints and allows a client to securely access private API resources inside a VPC. Private APIs are isolated from the public internet, and they can only be accessed using VPC endpoints for API Gateway that have been granted access.
Regional API endpoint
The host name of an API that is deployed to the specified Region and intended to serve clients, such as EC2 instances, in the same AWS Region. API requests are targeted directly to the Region-specific API Gateway API without going through any CloudFront distribution. For in-Region requests, a Regional endpoint bypasses the unnecessary round trip to a CloudFront distribution.

To keep things simple, we create a Regional API endpoint.

Creating the Models

We start by creating the API’s models (all of content type application/json):

A data schema specifying the data structure of a request or response payload. A model is required for generating a strongly typed SDK of an API. It is also used to validate payloads. A model is convenient for generating a sample mapping template to initiate creation of a production mapping template. Although useful, a model is not required for creating a mapping template.

— Amazon — Amazon API Gateway concepts

In order to support the GET /todos endpoint, we create the Todos model to define the response structure:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "Todos",
  "type": "array",
  "items": {
      "type": "object",
      "properties": {
          "Id": { 
              "minLength": 1,
              "type": "string"
          },
          "Name": {
              "minLength": 1,
              "type": "string" 
          }
      },
      "required": ["Id", "Name"]
    }
}

To support the POST /todos endpoint, we create the TodoCreate model to define the request structure:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "TodoCreate",
  "type": "object",
  "properties": {
      "Name": {
          "minLength": 1,
          "type": "string"
      }
  },
  "required": ["Name"]
}

and create the Todo model to define the response structure:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "Todo",
  "type": "object",
  "properties": {
      "Id": { 
          "minLength": 1,
          "type": "string"
      },
      "Name": {
          "minLength": 1,
          "type": "string" 
      }
  },
  "required": ["Id", "Name"]
}

To support the DELETE /todos/{id} endpoint we create the TodoDelete model to define the response structure:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "TodoDelete",
  "type": "object",
  "properties": {
      "Id": { 
          "minLength": 1,
          "type": "string"
      }
  },
  "required": ["Id"]
}

Create the Resources

Resources are essentially the URL paths of the API endpoints, e.g., /todos:

Typically, API resources are organized in a resource tree according to the application logic. Each API resource can expose one or more API methods that have unique HTTP verbs supported by API Gateway.

— Amazon — Amazon API Gateway concepts

In our case, we create two resources where we additionally select the Enable API Gateway CORS option to enable API Gateway to respond to CORS preflight requests:

/todos
/todos/{id}

note: The curly-braces indicate that id is a parameter.

Create the Methods

We next create methods, each with a HTTP verb, on the resources.

We create GET and POST methods on the /todos resource. We create a DELETE method on the /todos/{id} resource.

note: The OPTIONS methods are automatically provided because we selected the Enable API Gateway CORS option.

Method Backend

In addition to a HTTP verb, methods are associated to a backend. In our case, we associate them to the Lambda functions as follows (in each case we do not enable the Use Lambda Proxy Integration option):

GET /todos: Lambda function — Todos

POST/todos: Lambda function — TodoCreate

DELETE/todos/{id}: Lambda function — TodoDelete

Sidebar into the Use Lambda Proxy Integration Option

In our case, we chose to not use a Lambda Proxy Integration.

For Lambda proxy integration, API Gateway sends the entire request as input to a backend Lambda function. API Gateway then transforms the Lambda function output to a frontend HTTP response.

— Amazon — Amazon API Gateway concepts

If you recall from an earlier section, the Lambda functions we created simply use attributes, e.g., key, Name, and Id, on the event parameter. We also simply return Python objects, e.g., arrays and dictionaries. The important observation here is that the Lambda function has no “knowledge” that it is being called from a HTTP request; i.e., it is decoupled from how it is being used (a good thing).

On the other hand, if we chose to use a Lambda Proxy Integration, we would have had to do a lot more work in our Lambda function to extract and validate the relevant parameters, e.g., headers, path parameters, and body of a HTTP request object. We also would have had to return a structure that resembles a HTTP response object.

Method Execution

In addition to the method’s HTTP verb and backend, there are four additional configuration objects that define the method’s execution:

Method Request
Method Response
Integration Request
Integration Response

As visualized in the AWS Console:

Method Request

The public interface of a REST API method in API Gateway that defines the parameters and body that an app developer must send in requests to access the backend through the API.

— Amazon — Amazon API Gateway concepts

For each of the methods, we need to:

Set the Request Validator option to Validate body, query string parameters, and headers; we want API Gateway to validate all of the methods possible parameters
Enable the API Key Required option; remember we are using the metering feature to control access to the API. With this enabled, the API Gateway will automatically respond with the 403 (Forbidden) error
Add a x-api-key HTTP Request Header as Required; this key is used both by the metering feature as well as will be used as the key provided to our Lambda functions

Additionally for the POST /todos method, we need to set the Request Body to the TodoCreate model (and the application/json Content Type). With this if the request body does not match the model, API Gateway will automatically respond with the 400 (Bad Request) error

We can finally observe the DELETE /todos/{id} method already has a Request Path of id set because of the curly braces.

Method Response

The public interface of a REST API that defines the status codes, headers, and body models that an app developer should expect in responses from the API.

— Amazon — Amazon API Gateway concepts

For each of the methods, we need to:

Add the Header, Access-Control-Allow-Origin, to the 200 HTTP Status; this is because we will want to enable CORS for each of the methods
Add a 500 HTTP Status; each of the Lambda functions can have an unexpected error

Additionally for the GET /todos method, we need to set the Response Body of the 200 HTTP Status to the Todos model.

For the POST /todos method, we need to set the Response Body of the 200 HTTP Status to the Todo model.

For the DELETE /todos/{id} method, we need to set the Response Body of the 200 HTTP Status to the TodoDelete model.

For the DELETE /todos/{id} method, we also need to add a 404 HTTP Status; this is for when an id is provided that does not match an existing todo.

Integration Request

The internal interface of a WebSocket API route or REST API method in API Gateway, in which you map the body of a route request or the parameters and body of a method request to the formats required by the backend.

— Amazon — Amazon API Gateway concepts

For each of the methods, we need to:

Add an application/json Mapping Template (template values provided below per method)
Set the Request body passthrough option to When there are no templates defined (recommended)

This configuration is what maps the various parameters from the Method Request to the backend Lambda function event attributes.

Template for GET /todos:

#set($inputRoot = $input.path('$'))
{
  "key": "$input.params('x-api-key')"
}

Template for POST /todos:

#set($inputRoot = $input.path('$'))
{
  "key": "$input.params('x-api-key')",
  "Name": "$inputRoot.Name"
}

Template for DELETE /todos/{id}:

#set($inputRoot = $input.path('$'))
{
  "Id": "$input.params('id')",
  "key": "$input.params('x-api-key')"
}

Integration Response

The internal interface of a WebSocket API route or REST API method in API Gateway, in which you map the status codes, headers, and payload that are received from the backend to the response format that is returned to a client app.

— Amazon — Amazon API Gateway concepts

For each of the methods:

For the 200 Method response status, set the Access-Control-Allow-Origin Header Mapping to the value ‘*’ (includes the single quotes); this is what actually supplies the value of the header we specified in the Method Response above
Add an Integration response of status 500 (see the per-method values of Lambda Error Regex below); these are regular expressions that match the errorMessage of the response. For more information on this see Handle Lambda errors in API Gateway

GET /todos and POST /todos Lambda Error Regex:

.+

note: This matches any non-empty value.

DELETE /todos/{id} Lambda Error Regex:

^((?!ConditionalCheckFailedException).)+$

note: This matches any non-empty value that does not contain the string ConditionalCheckFailedException. This string is what the Python code returns when it does not find a matching todo (so should return a 404 error instead).

In addition, for the DELETE /todos/{id} method, we need to add an Integration response of status 404 with a Lambda Error Regex of:

.*ConditionalCheckFailedException.*

note: This matches any non-empty value that does contain the string ConditionalCheckFailedException.

Deploy API to Production Stage

Now that we have full constructed the resource tree (aka. the API) we now make it available to our customers by creating a deployment:

A point-in-time snapshot of your API Gateway API. To be available for clients to use, the deployment must be associated with one or more API stages.

— Amazon — Amazon API Gateway concepts

API stages are things like development, stage, and production; they can hold different versions (deployments) of the API and are available on different URLs to be used by different groups of people.

In our case, we can simply create and deploy to the production stage.

Create Usage Plan

A usage plan provides selected API clients with access to one or more deployed REST or WebSocket APIs. You can use a usage plan to configure throttling and quota limits, which are enforced on individual client API keys.

— Amazon — Amazon API Gateway concepts

Now, we can create a Usage Plan, e.g., Bronze with the following sample configuration:

Rate: 100 requests per second
Burst: 200 requests
Quota: 5,000 requests per month starting on the 1st day

We can then add the Develop / production stage the the plan.

Create API Key

An alphanumeric string that API Gateway uses to identify an app developer who uses your REST or WebSocket API. API Gateway can generate API keys on your behalf, or you can import them from a CSV file. You can use API keys together with Lambda authorizers or usage plans to control access to your APIs.

— Amazon — Amazon API Gateway concepts

We next create API keys for our customers. We then associate them with the Bronze Usage Plan.

Testing the Deployment

Now with an API key and the production stage URL, we can test our endpoints.

We first test the GET /todos endpoint: