21 KiB

Raw Blame History

layout	description	permalink
storage	Storing user data with Blockstack	/:collection/:path.html

Configure a hub on Amazon EC2

{:.no_toc}

This teaches you how to run a Gaia hub on Amazon EC2. Amazon EC2 is an affordable and convenient cloud computing provider. This example uses Amazon EC2 instance together with an EBS disk for file storage.

TOC {:toc}

Is this tutorial for you?

This documentation is appropriate for advanced power users who are familiar with command line tools, ssh, and basic editing configuration files.

If you are planning on running an open-membership hub or an application-specific hub, see the section on Hub Operation.

Prerequisites you need

This procedure uses Amazon AWS to choose and configure an Amazon Machine Image (AMI) running a Gaia service. For this reason, you should have an AWS account on the Amazon AWS free tier, personal account, or corporate account. These instructions assume you are using a free tier account.

These instructions assume you have already created a free domain through the freenom service. If you have another domain, you can use that instead.

Finally, setting up the SSL certificates on your EC2 instance requires you to use the terminal command line on your workstation. Make sure you have the watch command installed using the which command.

$ which watch
/usr/local/bin/watch

If watch is not located, install it on your workstation.

Task 1: Launch an EC2 instance

Visit the AWS Free Tier page and choose Sign in to the Console.
Make sure your region is set to one close to you.
Under Build a solution choose Launch a virtual machine.

The system opens the EC2 dashboard.
Enter blockstack-gaia in the search bar.

The system finds AMIs in the Marketplace and the Community.
Choose Community AMIs.

The system displays the available Gaia hub images.

Each image name has this format:

blockstack-gaia_hub-STORAGETYPE-VERSION-hvm - ami-BUILDTAG

You can choose an image that uses ephemeral or EBS storage. The ephemeral storage is very small but free. Only choose this if you plan to test or use a personal hub. Otherwise, choose the AMI for elastic block storage (EBS) which provides a persistent data store on a separate disk backed by EBS.

So, the blockstack-gaia_hub-ephemeral-2.5.3-hvm - ami-0c8fc48c10a42737e image uses ephemeral storage, is at version 2.5.3 and has the 0c8fc48c10a42737e tag.
Select the most recent version image with the storage you want. The images are not sorted; The most recent images is not necessarily at the top of the list.

After you select an image, the system displays Step 2: Choose an Instance Type page.

Select t2.micro and choose Next: Configure Instance Details.

To configure instance details, do the following:

Select a VPC.

A default VPC is created with a free tier account. You can use this default VPC. Or you can choose another VPC. If you choose another VPC, ensure the Subnet value is set to a subnet reachable by a public IP.

Important: If you're using a private subnet, you should attach an elastic IP (EIP) to the VM. This EIP allows you to reboot the instance without worrying whether the address will reset. To attach an IP, press allocate new address and follow the instructions to attach the EIP to your new EC2 instance.
Set Protect against accidental termination.

If you terminate a Gaia instance, you lose all the data associated with it. Protection adds an extra step to terminating your Gaia instance.
Open the Advanced Details.

At this point, you are going to configure environment variables for your instance.

Paste the following into the Advanced Details.

  <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  
  {
     "ignition": { "version": "2.2.0" },
     "storage": {
        "files": [{
        "filesystem": "root",
        "path": "/etc/environment",
        "mode": 420,
        "contents": {
           "source": "data:application/octet-stream,API_KEY%3DKEYPHRASE%0ADOMAIN%3DNAME_OF_DOMAIN%0ASTAGING%3DSTAGING_VALUE"
        }
        }]
     }
  }
  </code></pre></div>        </div>

Replace the following values in the JSON.

Value Description

<KEYPHRASE> A phrase to pass when using the hub admin. For example, hubba is a fun key phrase.

<NAME_OF_DOMAIN> Your hub's domain name. For example, maryhub.ml is the domain name in this example.

Value	Description
`<KEYPHRASE>`	A phrase to pass when using the hub admin. For example, `hubba` is a fun key phrase.
`<NAME_OF_DOMAIN>`	Your hub's domain name. For example, `maryhub.ml` is the domain name in this example.
`<STAGING_VALUE>`	Indicates what type of SSL to create, testing (`1`) or production (`0`). Set testing if you want to test without worrying about rate limiting. A testing cerificate is not secure. For this tutorial, use production (`0`).

<STAGING_VALUE>

Indicates what type of SSL to create, testing (`1`) or production (`0`). Set testing if you want to test without worrying about rate limiting. A testing cerificate is not secure.

For this tutorial, use production (`0`).

Check your Advanced Details they should look similar to the following:

        <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  {
     "ignition": { "version": "2.2.0" },
     "storage": {
        "files": [{
        "filesystem": "root",
        "path": "/etc/environment",
        "mode": 420,
        "contents": {
           "source": "data:application/octet-stream,API_KEY%3Dhubba%0ADOMAIN%3Dmaryhub.ml%0ASTAGING%3D0"
        }
        }]
     }
  }
  </code></pre></div>        </div>

At this point, you have configured your instance details.

Choose Next: Add Storage.

The storage is set according to the AMI you selected.
Choose Next: Add tags.
Optionally, add the following tags:

The tags are not required, they just apply searchable labels to an instance on an EC2 console.
- Key of Purpose with the Value gaia
- Key of Name with the Value gaia-hub
- Key of Version with the Value 2.5.3 (This value is an example, your version may be different.)
Choose Next: Configure Security Group.

Create a security group with the following three types:

Type	Protocol	Port Range	Source	Description
SSH	TCP	22	My IP	optional
HTTP	TCP	80	Anywhere	optional
HTTPS	TCP	443	Anywhere	optional

Choose Review and Launch.

The system may warn you that the selection is not free tier eligible. You can ignore this for now.
Press Launch.

The system prompts you for a key pair.
Select Create a new keypair or Choose an existing key pair.
Select Launch Instances.

The system launches your instance.

During the launch process the machine starts and runs some initial setup processes. These processes take a few minutes depending on the network, typically launching does not take more than 10 minutes. While this is happening the instance Status Checks reflect the Initializing status.

Task 2: Connect your Gaia server to your domain

Now, you are ready to test your Gaia server. This procedure ensures the Gaia services started correctly and they are configured to the domain name you provided in Advanced Details above.

Visit the AWS Free Tier page and choose Sign in to the Console.
Choose All services > EC2.

The system displays the EC2 Dashboard.
Select Running Instances.

The system displays your running instances.
Locate your recently launched Gaia server.

Make sure the instance shows as running and Status Checks are complete. Completed status checks ensures the Gaia processes and service were started.
Select the Description tab.
Locate the IPv4 Public IP value.

The public IP must match the DNS configured for the domain you entered in Advanced Details in the previous procedure.

Copy the IP and paste it in your browser.

If the response is	Do this...
	You should see a message that your connection is not private. Everything is fine, continue to the next step, step 8.
	Check that your domain's DNS configuration matches the public IP address of your instance. Update the DNS site's configuration. Restart your EC2 instance as per the Restart and reload certificates procedure on this page. Continue with next step, step 8.

Press Advanced.
Choose to proceed.
Extend the IP with the PUBLIC_IP/hub_info tag like so.

You should see a response from your Gaia hub!

At this point, you should see a Not secure message in the browser. That's because you haven't yet enabled SSL certification. While HTTPS is not required simple to run the hub services, Blockstack will only connect to a hub and write to its storage over a valid HTTPS connection.

Task 3: Configure a domain name

At this point, you can point a domain to your Gaia hub. Although it's not required, it is highly recommended. If you use a domain, you can migrate your instance to a different server (or even provider such as Azure or Dropbox) at any time, and still access it through the domain URL. Just point your domain at the IP address for your EC2 instance. Use an A Record DNS type.

These instructions assume you have already created a free domain through the freenom service. To point this freenom domain to your Gaia hub, do the following:

Log into your freenom account.
Choose the Manage Freenom Domain tab.
Add an A record leave the Name field blank.

This points your entire domain to the hub IP.
Save your changes.
Create a CNAME record.

For example, you can use the prefix www with your domain name. When you are done, your
Save your changes.

At this point, your DNS management should look similar to the following except that with your domain rather than the maryhub.ga domain.
After your changes propagate, visit your new domain at the hub_info page.

If you receive another Your connection is not private dialogs, take the option to proceed to your domain. The Not secure message should no longer appear in the browser bar. If the message does appear, try waiting a few minutes for your recent changes to propagate across the net domain servers. Then, refresh the page.
Check the SSL certificate for your hub.

Each browser has its own check procedure, for example, Chrome:

At this point, you have the following. An EC2 instance running Gaia and a DNS record pointing your domain to this instance.

AWS hub tips and tricks

Once your Gaia storage hub is up and running on AWS, you may occassionally need to troubleshoot. This section contains some useful information for interacting with your EC2 instance.

SSH into the Host

To SSH to the EC2 host directly:

ssh -t -i <your keyfile.pem> core@<public ip address>

Displaying the docker services

Your EC2 instance is running several docker services that support the Gaia hub. You can list these services using the docker ps command.

{% raw %}

$ docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Command}}\t{{.Names}}"
CONTAINER ID        IMAGE                                   COMMAND                  NAMES
6b170ce9b0d6        nginx:alpine                            "nginx -g 'daemon of…"   nginx
91c5ff651586        quay.io/blockstack/gaia-hub:v2.5.3      "docker-entrypoint.s…"   gaia-hub
16b229a20320        quay.io/blockstack/gaia-reader:v2.5.3   "node lib/index.js"      gaia-reader
89739e338573        quay.io/blockstack/gaia-admin:v2.5.3    "docker-entrypoint.s…"   gaia-admin

{% endraw %}

Each service plays a particular role in running your Gaia hub.

Service	Description
`certbot`	This service runs every 12 hours so you may not see it in the output. The service runs Let's Encrypt `certbot` client to support SSL. Certbot renews your certificates and reloads Nginx to pick up the changes. This service will run 2x per day checking if the certificate needs to be renewed.
`nginx`	Runs an Nginx proxy in front of the Gaia Hub. This service does things like rate-limiting, SSL termination, and redirects to HTTPS. Your nginx service relies on your hub's `readURL` to make requests. Changes to a hub's `readURL` must be reflected in the `nginx` service configuration in `/gaia/nginx/conf.d/default.conf`
`gaia-admin`	A simple administrative service that allows you to administer the Gaia hub. Use REST calls with this service to get and set hub configuration values.
`gaia-reader`	The Gaia read side-car services get file requests on URLs that start with your Gaia hub's `readURL`. You can determine your Gaia hub's read URL by either looking for the `readURL` key in your Gaia hub's config file. This value is or by looking for the `read_url_prefix` field in the data returned by a `HUB_URL/hub_info` page on your Gaia hub.
`gaia-hub`	The Gaia hub service.

Locations of key files

File or Directory	Description
`/etc/systemd/system`	Contains systemd unit-files for managing your Gaia hub.
`/etc/environment`	Contains the `DOMAIN` and `STAGING` variables you entered when creating your EC2 instance.
`/gaia/gaia.env`	Contains the environment variables used by the Gaia `systemd` unit-files.
`/etc/systemd/system/reset-ssl-certs.service`	A service that removes all existing certificates and restarts all the Gaia hub services. Use this sparingly, since the Lets Encrypt service will throttle too many requests for certificates.
`/gaia/hub-config`	Configuration for the Gaia Hub service.
`/gaia/admin-config`	Configuration for the Gaia Hub admin service.
`/gaia/reader-config`	Configuration for the Gaia Hub reader service.
`/gaia/nginx/conf.d`	Configuration files for the Nginx service.
`/gaia/nginx/certbot/conf`	Lets Encrypt SSL certificates/configs.
`/gaia/scripts`	Scripts run by the systemd services on startup.

You can cat the various services to see what settings they are using.

$ cat /etc/systemd/system/reset-ssl-certs.service
# reset-ssl-certs.service
[Unit]
Description=Reset Gaia to first boot
ConditionFileIsExecutable=/gaia/scripts/reset-certs.sh

[Service]
Type=oneshot
RemainAfterExit=no
EnvironmentFile=/gaia/gaia.env
EnvironmentFile=/etc/environment
ExecStart=/bin/bash /gaia/scripts/reset-certs.sh

[Install]
WantedBy=multi-user.target

Restart services and reload certificates

This procedures requires you to interact from a workstation command line with your running EC2 instance.

Open a terminal on your local workstation.
Confirm the hub DNS is set correctly with the following command:
```
watch -n 2 -t -g -x host <domain>
```
Substitute your domain name for the <domain> variable. For example:
```
watch -n 2 -t -g -x host maryhub.ga
maryhub.ml has address 34.219.71.143
```
If the command returns the correct IP, the same as appears on your EC2 dashboard, stop the process with a CTRL-C on your keyboard.
Change the permissions on your downloaded .pem file.

For example, this
```
chmod 400 <location-of-pem>
```
SSH from your workstation and restart Gaia Hub:

This process requires that you know the location of the .pem file you downloaded when you created the keypair.
```
ssh -t -i <your keyfile.pem> -A core@<public ip address> "sudo systemctl restart gaia.service"
```
For example:
```
$ ssh -t -i /Users/manthony/gaia.pem -A core@34.219.71.143 "sudo systemctl restart gaia.service"
Connection to 34.219.71.143 closed.
```
This will restart all services required for running a Gaia Hub (nginx, hub, reader, admin, certbot)
SSH from your workstation to reset back to first boot:

** This process will stop Gaia Hub, Nginx and remove any existing SSL certificates. It will then start the process of retrieving certificates and setting up the services again. This will not affect any existing data stored on the server.

This process requires that you know the location of the .pem file you downloaded when you created the keypair.
```
ssh -t -i <your keyfile.pem> -A core@<public ip address> "sudo systemctl restart reset-ssl-certs.service"
```
For example:
```
$ ssh -t -i /Users/manthony/gaia.pem -A core@34.219.71.143 "sudo systemctl restart reset-ssl-certs.service"
Connection to 34.219.71.143 closed.
```
After a few minutes, all Gaia Hub services will restart automatically and will retrieve a new SSL certificate.

21 KiB Raw Blame History