Routing Traffic to HarperDB using Fastly Compute@Edge

Our Goal

The purpose of this tutorial is to use the Fastly network to route traffic to HarperDB based on geo proximity.

Fastly and Compute@Edge

Fastly is a content delivery network (CDN) and an edge cloud platform. Compute@Edge is a Fastly offering that is designed to allow developers to run custom code at the edge of the Fastly network, closer to users.

Requirements

This tutorial requires a few things to be set up.

A HarperDB account
A multiple node HarperDB cluster
Your own domain with DNS records pointing to your nodes that have TLS certificates on them. *NOTE: If you do not have DNS or TLS certificates you can config Fastly to ignore certificate validation, however this is not recommended.

HarperDB Nodes / Instances

If you don’t already have the above requirements, start here to get an account with HarperDB. From there you’ll need to deploy at least two internet accessible HarperDB nodes (also referred to as instances) that are serving HarperDB via https.

HarperDB Multi Node Cluster

Then cluster your HarperDB nodes together in your configuration of choice. Here is a great video resource for using HarperDB Clustering, and this written tutorial provides a common use case scenario.

Fastly Getting Started

Create your Fastly account at https://www.fastly.com/signup, and follow the instructions to log in.
‍

Next, navigate to the “Getting Started Guide.” The instructions will direct you to:

Generate an API token
Download/install the Fastly CLI
Configure the Fastly CLI
Install local dependencies (as needed)
Create a new Compute@Edge project
Choose a language
Compile the project to a Wasm binary
Deploy the project to a new Fastly service

‍

Generate an API token

Navigate to “Account” by clicking your account name in the top right corner, then click “API Tokens” from the lower left menu. Click “Create Token”. Use the following options:

Submit the form. Next, save your token somewhere securely and click “Okay.” You’ll use this when you config the CLI.

‍

Download/install the Fastly CLI

Install the fastly CLI https://developer.fastly.com/learning/compute.

If you do not have NodeJS set up, follow the Javascript setup instructions at the link above.

‍

Configure The Fastly CLI

Next, set up the Fastly CLI to use our API token. Run fastly profile create <yourproject> and answer the following prompts.

‍

Install Local Dependencies

You will need a recent version of Node.js and npm if you don’t have them already.

Create a new Compute@Edge project

Now you can use the Fastly CLI to initialize your Compute@Edge code project. The command is fastly compute init. It’s here that you’ll be able to choose your language and choose a starter if wanted. In this example, I’ve gone with an empty starter for JavaScript.

This will result in a shell project for your Fastly Compute@Edge code.

‍

Compile the Project to a Wasm Binary

Next up, we’ll use fastly compute build to compile into a WebAssembly module.

‍

Deploy the Project to a New Fastly Service

To deploy, run fastly compute deploy. At this point, Fastly starts a Compute@Edge free trial. At the prompts, you can accept an auto created domain name, or choose your own. To use Fastly’s free TLS your domain should be <name>.global.ssl.fastly.net, replacing <name> with a single word that claims the domain you're creating. You can also now access the Compute Console.

‍

Fastly Compute Setup

Next we need to add Hosts (your HarperDB nodes).

‍

From the Compute Console, select your domain (or version associated with that domain), and then select “Hosts” from the left side menu.

‍

Add all of your hosts to this list. You may use up to five hosts for free.

‍

These host references can be by DNS or by IP. If you use IP, you will need to click the pencil icon and edit the TLS settings. You will also need to do this if you decide not to use your own DNS and TLS certificates on your HarperDB nodes.

‍

Next you need to edit each host to give them unique names. To do this click on the pencil icon next to the host.

‍

Next you need to set up a lookup table using the Dictionary tab. Click on the “Dictionary” button on the left side of the screen. Click “Create your first dictionary”, name it “backends” and click “Add.”

‍

Here you will create a mapping of Fastly points of presence (PoP) to your HarperDB nodes. You can find a complete list of Fastly PoPs by using the Fastly api. You should at least use the Fastly PoP that is closest to you so that you can test the routing.

‍

The following screenshot shows an example of a mapping of a few PoPs to your HarperDB hosts by using the name of the host defined in the step above.

‍

‍

This lookup table can be modified to update your Fastly PoP to HarperDB node routing without having to modify any of your Compute code, which we will discuss in more detail later in the tutorial. You should also create a key called “default” pointing to one of your nodes. This will help in cases that a PoP is used that is not in your lookup table.

‍

With Compute service set up in the console, it’s a good time to install dependencies then switch over to the IDE to add some code.

‍

To install dependencies run npm install in your project.

‍

Now, open src/index.js and replace the existing code with the following:

//! Default Compute@Edge template program.


/// 
import { Dictionary } from "fastly:dictionary";
import { env } from "fastly:env";


// The entry point for your application.
//
// Use this fetch event listener to define your main request handling logic. It could be
// used to route based on the request properties (such as method or path), send
// the request to a backend, make completely new requests, and/or generate
// synthetic responses.


addEventListener("fetch", (event) => event.respondWith(handleRequest(event)));


async function handleRequest(event) {
 // Grab the information from the lookup dictionary from the Compute Service
 const exampleDictionary = new Dictionary("backends");
 // Get the Fastly PoP the user was connected to
 var fastlyPop = env("FASTLY_POP");


 // Find the host name of the HarperDB node to connect to via the lookup table
 var hdbServer = exampleDictionary.get(fastlyPop)


 // If the pop isn't found in the lookup table use a record in the table named "default"
 if (!hdbServer) {
   hdbServer = exampleDictionary.get("default")
 }


 // Proxy the request to the HarperDB node
 return fetch(event.request, {
   backend: hdbServer
 });
}

‍

Now we’re ready to publish our service!

‍

Run fastly compute publish.

‍

Once this is done, your Compute service in the console will show up as “Active” and you will be able to access your HarperDB Cluster via the domain you set earlier, <name>.global.ssl.fastly.net. In the following screenshot I’m using a curl command via Postman to hit the “registration_info” operation using the username/password authentication for the node my traffic is routed to.

‍

‍

Additional Considerations

There are a couple more things that should be considered for a production environment.

‍

Health Checks to redirect traffic when a HarperDB node is offline
A way to automatically update your lookup table when Fastly adds new PoPs

‍

Conclusion

Deploying a Fastly Compute service is quick and easy while providing a way to route traffic to HarperDB nodes by using the client’s fastest connection to the Fastly network via Fastly PoPs. It also provides us a way to change our traffic routing on the fly with no code changes using Fastly’s dictionary.

‍

Dev Center

Our Story