Integrating HarperDB Authentication with AWS Cognito

‍Introduction

This is another article of the series of articles where we integrate HarperDB with different authentication providers. We already did it with oAuth2, Auth0 (Okta), and Azure Active Directory, now it’s time to let AWS Cognito users access HarperDB.

AWS Cognito: a Brief Introduction

AWS Cognito is a service that provides authentication, authorization, and user management for your web and mobile applications. It’s a fully managed service that scales to support hundreds of millions of users. It’s easy to integrate with your applications, and it’s free for up to 50,000 monthly active users.

Think of it as a user directory in the cloud. You can create users, groups, and roles, and assign permissions to them. You can also integrate it with other AWS services, such as API Gateway, Lambda, and S3. It’s Amazon’s version of Azure’s Active Directory.

One nice difference between the two services is that Cognito allows you to also authenticate and authorize users from other providers, such as Google, Facebook and others.

User Pools and Identity Pools

Cognito has two main components: User Pools and Identity Pools. The user pool is where you create users, groups, and roles. It’s also where you define the authentication methods that users can use to access your application.

The main use of a user pool is to authenticate external users, which is basically what we want to do. Amazon has a high level image that shows how it works:

Identity pools, on the other hand, authenticate and authorize users to access AWS resources. It’s a way to give users access to AWS resources without having to create IAM users for them. It’s not what we want to do here, but it’s good to know that it exists. We won’t talk about it much, but this is a another high level view of how it works:

Why Cognito

There are many reasons why you should use Cognito instead of creating your own authentication system. Here are some of them:

It’s free for up to 50,000 monthly active users.
It’s easy to integrate with your applications.
It’s secure.
It’s scalable.
It’s managed by Amazon, so you don’t have to worry about it.

Those are the same principles behind the use of managed identity pools like Azure AD and Okta. The idea is that we don’t want to manage users, groups, and roles ourselves. We want to focus on our application, not on the authentication system.

Cognito, as well as the others, have RBAC, which stands for Role Based Access Control. It’s a way to define what users can do in your application. You can define roles and assign permissions to them. Then you can assign users to those roles. It’s a very common way to manage access to applications.

Like we did in our previous article with AAD, we’ll also give the users permissions to access specific tables or resources in our database.

Pre-requisites

Like I say in all articles, to avoid repeating myself we are going to use our previous articles’ bases to create our HarperDB instance. For that you can read the “Creating the Infrastructure” section of our first article without any modifications. This should give you a HarperDB instance running on a Docker container.

When you start your Docker compose file using docker compose up -d you should have a HarperDB instance running on port 9925, with custom function enabled on port 9926. Also, the custom functions volume will be created on the local directory, which means you will also have a harperdb folder on your project root. That’s where we will put all the files related to our server.

The final code for this application can be found in the GitHub repository at https://github.com/HarperDB-Add-Ons/harperdb-cognito-integration. You can clone the repository and follow along with the code, or you can just copy and paste the code from the repository.

I’m also assuming you already have an AWS account. If you don’t, you can create one for free at https://aws.amazon.com/free.

Creating the Cognito User Pool

The first thing we need to do when working with Cognito is to create an user pool. The user pool is the place where we create users, groups, and roles. It’s also where we define the authentication methods that users can use to access our application.

To create a user pool, go to the AWS console and search for Cognito. You should reach the Cognito main screen.

Click on Create user pool, you will be redirected to the creation screen. Make sure that User name is checked in the Authentication Providers section and click next:

Then you’ll be prompted to set the password rules. You can set the ones you want or leave the default ones. I’ll set a very simple one just for testing purposes.

DO NOT USE THIS PASSWORD RULES IN PRODUCTION

Then you’ll be prompted to set the MFA (Multi Factor Authentication) rules. You can set the ones you want or leave the default ones. I’ll remove them altogether so we don’t have to deal with extra steps when creating users, at least for now.

Click Next to configure the sign-up part. This is a very important part because it’s where we’ll define who can be a user of our system. In this example I’m not allowing external users, so I’ll remove the Self-Registration option. We won’t use the email options either, but removing them is not necessary.

The Attribute verification and user account confirmation step will remain default.

Then we will scroll all the way down to the Custom Attributes and add one called roles. This is the attribute that we’ll use to define the roles of our users. We’ll use it to define what tables they can access in HarperDB. This attribute is going to be presented to us as custom:roles in Cognito.

In the next step, we’ll define Cognito as the default email provider, this way we don’t have to deal with email configuration for now.

The last step is where we’ll define the pool’s name and all of our app attributes, it’s the most important step. I’m calling my pool harperdb-cognito. Also, we’ll use the Cognito hosted UI so we can change the user’s password easier. Let’s enable it and choose a domain name for it, I’ll just use harperdb as the domain name.

The most important part is on the Initial app client section. We’ll define it as a Confidential Client which means we’ll be the ones doing the authentication flow, this will give us a Client ID and Secret that we’ll need to use later on. Make sure the Generate client secret option is checked. It’ll ask you for a callback URL, but we won’t use it, so you can just put any URL there. I’ll use https://harperdb.io just for the sake of it.

Next, scroll down to Advanced app client settings and enable the Username and Password authentication flow. This will allow us to authenticate users using their username and password. You don’t need to remove the SRP authorization flow, but we won’t use it, so you can remove it if you want.

Click Next, review your settings and finish. This will give you a user pool that we can work with.

Create the first user

Now that we have a user pool, we can create our first user. Click the user pool you just created, go to Users and then Create User. We’ll create Bob, without any email and a simple password that we can remember.

When created, click Bob’s name on the list and open his details. Then go to User Attributes and add a new attribute, the one we created before. It’ll be presented as custom:roles, the value of it will be a mapping of schema.table.permission where schema is the schema of the table, table is the table name and permission is the permission the user has on that table. In this example, we’ll give Bob access to the breeds table in the dogs schema with read and write permission. So the value will be dogs.breeds.read,dogs.breeds.write.

When saving, Cognito will ask you to fill in the user’s email, we’re not actively using it, so you can just put any email there. And with this we finish our user, which should be like this in the end:

To finish up, we need to change the user’s password. Go back to the dashboard of your user pool, click on the App Integration tab and copy the Cognito domain.

Then navigate to that domain and log in as Bob (remember that the username is case sensitive). You’ll be prompted to change the password, so do it.

Troubleshooting: The Cognito Domain doesn’t work

In some cases, the Cognito URL and domain will not open or not work, it might take several minutes for it to get up to speed with all the changes, but if you don’t want to wait, you can use the AWS SDK to send a request to change the user’s password permanently.

Be aware that you’ll need your user pool ID, which is in the dashboard of the user pool, and the AWS credentials of your user. Which you can find by clicking on your name on the top right corner of the AWS console and then clicking on Security Credentials, then proceed to the Access keys section and create a new one. Save the client ID and secret somewhere safe, you’ll need them later.

To do it, make sure you install the @aws-sdk/client-cognito-identity-provider package in your project. Then you can use the following code to change the user’s password:

const {
  CognitoIdentityProviderClient,
  AdminSetUserPasswordCommand
} = require('@aws-sdk/client-cognito-identity-provider')
  
const client = new CognitoIdentityProviderClient({
  region: 'us-east-1', // or the region you created your user pool
  credentials: {
    accessKeyId: 'your key id',
    secretAccessKey: 'your access key'
  }
})
  
const command = new AdminSetUserPasswordCommand({
  UserPoolId: 'your user pool id',
  Username: 'Bob',
  Password: 'new password',
  Permanent: true
})
  
client
  .send(command)
  .then(() => {
    console.log('Password changed')
  })
  .catch((err) => {
    console.log(err)
  })

If it works, when you reload the user list, Bob’s confirmation status will be set as Confirmed:

Creating the HarperDB Custom Function

Now we need to create our custom function in HarperDB. This function will be responsible for authenticating the user in the Cognito user pool. It’ll receive the username and password and return a JWT token if the user is authenticated, we’ll then use this JWT token to verify the user’s permissions in the database.

To start, make sure that you have HarperDB installed and running, and that you can access the Studio on https://studion.harperdb.io, if you followed the steps in the Pre-requisites section, you should also have a harperdb directory inside the same place you spun up HarperDB’s container using Docker.

Inside the harperdb directory, create a new directory called api, then two more directories inside it, one called helpers and another one called routes.

In the harperdb/api root directory, run npm init -y to create your package.json file, and then use the following command to install the required dependencies:

npm install dotenv @aws-sdk/client-cognito-identity-provider

Then create a .env file in the harperdb/api root directory and add the following variables:

AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEY
COGNITO_CLIENT_ID=YOUR_COGNITO_CLIENT_ID
COGNITO_CLIENT_SECRET=YOUR_COGNITO_CLIENT_SECRET

You can find your credentials by clicking on your name on the top right corner of the AWS console and then clicking on Security Credentials, then proceed to the Access keys section and create a new one. The Cognito client ID and secret can be found by clicking the App integration tab of your user pool and then clicking on the App client list section on the bottom, then click your client.

Then, remember to click the Show client secret button to see the secret.

Copy both of them and paste them in your .env file.

In the routes file, we’ll create our base routes to read and write data from the database. Create a file called index.js inside the harperdb/api/routes directory and add the following code:

const path = require('path')
require('dotenv').config({ path: path.resolve(__dirname, '../.env') })
const cognito = require('../helpers/cognito')
  
module.exports = async (server, { hdbCore, logger }) => {
  // CREATE A DATA RECORD
  server.route({
    url: '/:schema/:table',
    preValidation: (request, response, next) => cognito.validate(request, response, next, hdbCore, logger),
    method: 'POST',
    handler: (request) => {
      const { schema, table } = request.params
      const hasPermission = request.body?.hdb_user?.role?.permission[schema]?.tables[table]?.insert === true
  
      if (!hasPermission) return response.code(403).send('Forbidden')
  
      const { records } = request.body
      request.body = {
        operation: 'insert',
        schema,
        table,
        records,
        hdb_user: request.body.hdb_user
      }
  
      return hdbCore.request(request)
    }
  })
  
  // READ DATA RECORDS
  server.route({
    url: '/:schema/:table/:hash',
    preValidation: (request, response, next) => cognito.validate(request, response, next, hdbCore, logger),
    method: 'POST',
    handler: (request, response) => {
      const { schema, table, hash } = request.params
      const hasPermission = request.body?.hdb_user?.role?.permission[schema]?.tables[table]?.read === true
  
      if (!hasPermission) return response.code(403).send('Forbidden')
  
      request.body = {
        operation: 'search_by_hash',
        schema,
        table,
        hash_values: [Number(hash)],
        hdb_user: request.body.hdb_user,
        get_attributes: ['name', 'id']
      }
  
      return hdbCore.request(request)
    }
  })
}

What we’re doing here is creating two routes, one to create a new record in the database and another one to read a record from the database. Both of them will receive the schema and table name as parameters, and the read route will also receive the hash of the record to be read. But first, on the preValidation we’ll call our cognito.validate function, which will validate the user’s JWT token and check if the user has the required permissions to perform the operation.

Now, let’s create the cognito.validate function. Create a file called cognito.js inside the harperdb/api/helpers directory. First we’ll set all the data we need, this code is not going to be the prettiest ever, but it’s going to be easy to understand, so let’s create some objects that will hold the information we need to access AWS:

const { CognitoIdentityProviderClient, InitiateAuthCommand } = require('@aws-sdk/client-cognito-identity-provider')
const { createHmac } = require('crypto')
  
const COGNITO_CONFIG = {
  region: process.env.COGNITO_REGION || 'us-east-1',
  clientId: process.env.COGNITO_CLIENT_ID,
  clientSecret: process.env.COGNITO_CLIENT_SECRET,
  awsAccessKeyId: process.env.AWS_ACCESS_KEY_ID,
  awsSecretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
}
  
const cognitoClient = new CognitoIdentityProviderClient({
  region: COGNITO_CONFIG.region,
  credentials: {
    accessKeyId: COGNITO_CONFIG.awsAccessKeyId,
    secretAccessKey: COGNITO_CONFIG.awsSecretAccessKey
  }
})
  
const PERMISSION_MAP = {
  read: 'read',
  write: 'insert'
}

Then let’s create a function that will extract the token data from the request and validate it. By default, all Cognito’s authentication results have three tokens:

the access token, which is the one the user can use to perform other queries to Cognito, like getting the user’s information;
the ID token, which is the one we’ll use to validate the user’s permissions in the database since it has all the user’s information;
and the refresh token, which is the one to use to refresh the user’s session.

They’re all JWTs, which means that they’re composed of a head, a payload and a signature. We won’t validate the signature of it or the head, we’re just interested in the payload:

function getTokenData(token) {
  const tokenData = token.split('.')[1]
  return JSON.parse(Buffer.from(tokenData, 'base64').toString())
}

For the main validation function, we’ll get the user’s data from the request and perform the authentication flow in Cognito:

async function validate(request, response, next, hdbCore, logger) {
  const userData = {
    username: request.body.username,
    password: request.body.password
  }
  
  try {
    const authCommand = new InitiateAuthCommand({
      AuthFlow: 'USER_PASSWORD_AUTH',
      ClientId: COGNITO_CONFIG.clientId,
      AuthParameters: {
        USERNAME: userData.username,
        PASSWORD: userData.password,
        SECRET_HASH: createHmac('sha256', COGNITO_CONFIG.clientSecret)
          .update(userData.username + COGNITO_CONFIG.clientId)
          .digest('base64')
      }
    })
  
    const result = await cognitoClient.send(authCommand)
    if (!result.AuthenticationResult?.IdToken) {
      throw new Error('Invalid credentials')
    }
  } catch (error) {
    console.log('error', error)
    return response.code(500).send('Cognito Error')
  }
}

Then we’ll get the user’s data from the token and check if the user has the required permissions to perform the operation:

async function validate(request, response, next, hdbCore, logger) {
  const userData = {
    username: request.body.username,
    password: request.body.password
  }
  
  try {
    const authCommand = new InitiateAuthCommand({
      AuthFlow: 'USER_PASSWORD_AUTH',
      ClientId: COGNITO_CONFIG.clientId,
      AuthParameters: {
        USERNAME: userData.username,
        PASSWORD: userData.password,
        SECRET_HASH: createHmac('sha256', COGNITO_CONFIG.clientSecret)
          .update(userData.username + COGNITO_CONFIG.clientId)
          .digest('base64')
      }
    })
  
    const result = await cognitoClient.send(authCommand)
    if (!result.AuthenticationResult?.IdToken) {
      throw new Error('Invalid credentials')
    }
  
    const tokenData = getTokenData(result.AuthenticationResult.IdToken)
    if (!tokenData['custom:roles']) {
      throw new Error('User has no roles')
    }
  } catch (error) {
    console.log('error', error)
    return response.code(500).send('Cognito Error')
  }
}

In the end, we’ll build a map of the user’s permissions by adding a new object in the request body which we’re already checking in the routes:

async function validate(request, response, next, hdbCore, logger) {
  const userData = {
    username: request.body.username,
    password: request.body.password
  }
  
  try {
    const authCommand = new InitiateAuthCommand({
      AuthFlow: 'USER_PASSWORD_AUTH',
      ClientId: COGNITO_CONFIG.clientId,
      AuthParameters: {
        USERNAME: userData.username,
        PASSWORD: userData.password,
        SECRET_HASH: createHmac('sha256', COGNITO_CONFIG.clientSecret)
          .update(userData.username + COGNITO_CONFIG.clientId)
          .digest('base64')
      }
    })
  
    const result = await cognitoClient.send(authCommand)
    if (!result.AuthenticationResult?.IdToken) {
      throw new Error('Invalid credentials')
    }
  
    const tokenData = getTokenData(result.AuthenticationResult.IdToken)
    if (!tokenData['custom:roles']) {
      throw new Error('User has no roles')
    }
  
    /* POPULATE USER ROLES IN REQUEST BODY */
    request.body.hdb_user = { role: { permission: {} } }
    tokenData['custom:roles'].split(',').forEach((role) => {
      const [schema, table, operation] = role.split('.')
      if (!request.body.hdb_user.role.permission[schema]) {
        request.body.hdb_user.role.permission[schema] = { tables: {} }
      }
      if (!request.body.hdb_user.role.permission[schema].tables[table]) {
        request.body.hdb_user.role.permission[schema].tables[table] = {
          read: false,
          insert: false,
          update: false,
          delete: false,
          attribute_permissions: []
        }
      }
      const permission = PERMISSION_MAP[operation]
      request.body.hdb_user.role.permission[schema].tables[table][permission] = true
    })
  } catch (error) {
    console.log('error', error)
    return response.code(500).send('Cognito Error')
  }
}

Don’t forget to export the function:

module.exports = {
  validate
}

Create the schema and table

Now the last step is to create the schema and the table that we will use to store the data. To do that, go to the HarperDB Studio and, in the browser tab, create the schema dogs and the table breed.

Testing the application

To test our application, you can open any request tool like Postman or Insomnia. We will use Thunder Client for this tutorial.

Let’s use Bob’s credentials to create a new record. To do that, we will use the POST method to the /dogs/breed endpoint. The authentication is sent on the body of the request.

{
  "username": "Bob",
  "password": "123456789",
  "records": [
    {
      "id": 1,
      "name": "corgi"
    }
  ]
}

You should get a response like this.

If we try to read, we’ll also get the data back from the /dogs/breed/1 endpoint.

Now, if we either create another user or try to read another table using the POST method. We will get a 403 status code.

Conclusion

In this tutorial, we learned how to use HarperDB’s custom authentication to integrate with AWS Cognito. We also learned how to use the custom authentication to create a role-based access control system. If you want to learn more about AWS Cognito, you can check the documentation.

‍

Dev Center

Our Story