20 KiB

Raw Blame History

layout	permalink
storage	/:collection/:path.html

Configure a hub on DigitalOcean

{:.no_toc}

This teaches you how to run a Gaia storage hub on DigitalOcean (DO). DigitalOcean is an affordable and convenient cloud computing provider. This example uses DigitalOcean Spaces for file storage. DigitalOcean Spaces is equivalent to AWS's S3 file storage solution.

DigitalOcean provides you with a compute machines known as a Droplets and storage called a Spaces. You need both to run a Gaia hub. The Gaia hub setup you create here, requires get a Digital Droplet with Docker pre-installed and a 250 GB Space. Each run for $5/month or a total of $10/month.

TOC {:toc}

Is this tutorial for you?

This documentation is appropriate for advanced power users who are familiar with command line tools, editing configuration files, and basic configuration of services of DNS or Nginx.

If you are planning on running an open-membership hub or an application-specific hub, you'll should see the section on Hub Operation

Prerequisites you need

You use DigitalOcean choose and configure assets for running droplets and spaces. To enable this, you must be sure to complete a number of prerequisites.

Required prerequisites on DigitalOcean

You must create an account on DigitalOcean. DigitalOcean requires you to supply a credit card to create an account. You are only charged for the services you use the Gaia hub as of this writing should cost $10 USD a month.

Optional prerequisites for SSH users

The easiest way to interact with your droplet is the DigitalOcean Console. Users who are comfortable using the secure shell (SSH) and private keys may prefer to open a local terminal on their home machine instead. To enable this, you should ensure you have the following prerequisites completed.

Locate an existing SSH key pair on your Mac or create a new SSH key pair. Your key should have a passphrase, do not use a key pair without one.
Add the SSH from your local machine to DigitalOcean.
Create a personal access token in DigitalOcean.
Install doctl which is the DigitalOcean command line tool. For information on installing these, see which is the DigitalOcean command line utility. Check out their installation instructions to see how to install it on your computer.

Task 1: Create a DigitalOcean space

In this task you create a Space which is where Gaia stores your files.

Choose Create > Spaces from the dashboard menu.
Choose a datacenter region section.

{% include important.html content="Choose a region that is both geographically close to you and that supports spaces. Currently, San Francisco, New York, Amsterdam, and Singapore support spaces." %}

The geographical location of your server impacts latency for storing data. You select a region close to you so that when you use Blockstack apps, storing data is quicker.
Scroll down to Finalize and Create.
Choose a unique name.

This name is used when reading files that you've stored through Gaia. You'll need to remember this name when you set up your Gaia server later on.
Click Create a Space.

After a moment, your Space is up and running.

Task 2: Create a DigitalOcean droplet

In this task, you add a droplet to your account. The droplet is a droplet is a cloud-based server you can use as a compute resource. This server is where you will run the Gaia Storage System service. The droplet you create will be an Ubuntu server with Docker pre-installed.

Log into DigitalOcean.
Go to your DigitalOcean dashboard.
Click the Create > Droplets button in the top right.
Select the One-click apps tab.
Select the Docker app from the options presented.

This will give you
Scroll down to the Choose a size section and use the left arrow to display and select the $5/mo image.

This size gives you plenty of memory and disk space to run a personal hub.
Scroll down to the Choose a datacenter region section.

{% include important.html content="Choose a region that is both geographically close to you and that supports spaces. Currently, San Francisco, New York, Amsterdam, and Singapore support spaces." %}

The geographical location of your server impacts latency for storing data. You select a region close to you so that when you use Blockstack apps, storing data is quicker.
If you are using SSH, scroll to the Add your SSH key section and choose an SSH key to use. Otherwise,
Scroll down to the Finalize and create section.
Choose a hostname for you Give your Droplet such as moxiegirl-storage-hub.
Review your choices then click Create to start your droplet running.

At this point, your new droplet should appear in the list of resources on your DigitalOcean dashboard.

Task 3: Open a console on your Droplet

A droplet console emulates the access you would have if you were sitting down with a keyboard and monitor attached to the actual server. In this section, you open a console on your droplet.

{% include note.html content="If you are an SSH user and have completed the prerequisites, skip this section. Instead, use the DigitalOcean instructions for connecting with doctl." %}

From the DigitalOcean dashboard, select Droplets.

You should see the droplet you just created.
Click on the droplet name to open the control panel.
Choose Access from the control panel.
Select Reset Root Password to have DigitalOcean send you the root password.

DigitalOcean sends a temporary password to the email attached to your account. It takes a couple of minutes to reset the root password on your droplet.
Open your email and copy the password.
Switch back to the droplet control panel and choose Launch Console.

A new window with the console appears.
Enter root for the login.
Paste the password copied from your email.

The system displays a message telling you to change the root password.

And prompts you for the current password.
Past the password copied from your email again.

The system prompts you for a new password and ask you to enter it again.

Provide and confirm a new password.

The system logins you in and gives you a welcome message. At the conclusion of the message, you are at the console prompt.

Welcome to DigitalOcean's One-Click Docker Droplet.
To keep this Droplet secure, the UFW firewall is enabled.
All ports are BLOCKED except 22 (SSH), 2375 (Docker) and 2376 (Docker).
*	The Docker One-Click Quickstart guide is available at: https ://do.co/docker1804#start
*	You can SSH to this Droplet in a terminal as root: ssh root@138.68.28.100
*	Docker is installed and configured per Docker's recommendations: https://docs.docker.com/install/linux/docker-ce/ubuntu/
*	Docker Compose is installed and configured per Docker's recommendations: https://docs.docker.eom/compose/install/#install-compose
For help and more information, visit http://do.co/dockerl804

To delete this message of the day: rm -rf /etc/update-motd.d/99-one-click root@moxiegirl:~#

Useful tips for the console

If you run into problems using the console, see the notes on this page in the DigitalOcean documentation.

If you find the output from ls difficult to read, try enter the following to change the console colors from the command line: LS_COLORS="di=1;31" You can also edit your console .bashrc. file permanently, of course.

Task 4: Create a space key

In the DigitalOcean dashboard, go to the API page.
Scroll to the Spaces Access Keys section.
Click Generate New Key.

The system prompts you to give the key a name.
Enter a name for the key.

It is helpful to choose descriptive name like gai-hub-key.
Press the check mark.

The system creates your key and displays both the key and its secret.
Save your secrete in a secure password manager.

You should never share your secret.
Leave the page up with your key and secret and go to your open console.

Task 5: Get the Gaia code and configure your server

You should have the console open as root on your Droplet. In this section, you get the Gaia code and configure the Gaia service.

Copy the Gaia code into your droplet using the git clone command.

root@moxiegirl:~# git clone https://github.com/blockstack/gaia.git

Successful output from this command looks like the following.

Cloning into 'gaia'...
remote: Enumerating objects: 63, done.
remote: Counting objects: 100% (63/63), done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 4206 (delta 27), reused 35 (delta 17), pack-reused 4143
Receiving objects: 100% (4206/4206), 17.40 MiB | 9.89 MiB/s, done.
Resolving deltas: 100% (2700/2700), done.
root@moxiegirl:~#

This command creates a gaia subdirectory.

Change to hub directory in the gaia code.
```
cd gaia/hub
```
Copy the configuration sample to a new config.json file.
```
cp config.sample.json config.json
```
Edit your new config.json file with vi or vim.
```
vi config.json
```
You now need to edit this JSON file to have it store files on your DigitalOcean space.
Set the driver to aws.

The DigitalOcean space API exactly mimics the S3 API. Since Gaia doesn't have a DigitalOcean driver, you can just use the aws driver with some special configuration.
Set the bucket to the name of the DigitalOcean space you just created.

If your space is called moxiegirl-hub-space, the bucket value is moxiegirl-hub-space.
Set the readURL to the URL of the DigitalOcean space you just created.

If your space URL called https://moxiegirl-hub-space.sfo2.digitaloceanspaces.com , the readURL name is https://moxiegirl-hub-space.sfo2.digitaloceanspaces.com.

Add an endpoint value to the awsCredentials section.

"awsCredentials": {
   "accessKeyId": "",
   "secretAccessKey": "",
   "endpoint": ""
},

Go back to your DigitalOcean dashboard open to your space key.
Copy the Key and paste it into the accessKeyId value in the config.json file.
Copy the Secret and paste it into the secretAccessKey value in the config.json file.
In the DigitalOcean dashboard, choose the Spaces page.
Copy the section of your space URL that follows the name.

In this example, you would copy the sfo2.digitaloceanspaces.com section.
Paste the string you copied into the endpoint value.

Change the proofsRequired value to the number 0 (zero).

This will allow Blockstack user to write to your Gaia hub, without any social proofs required. You can change this later on, and do other things to lock-down this Gaia hub to just yourself, but that is outside the scope of this document.

At this point, the json.config file should be completed and appear similar to the following &—; but with your values.

{
  "servername": "",
  "port": 3000,
  "driver": "aws",
  "bucket": "moxiegirl-hub-space",
  "readURL": "https://moxiegirl-hub-space.sfo2.digitaloceanspaces.com",
  "awsCredentials": {
    "accessKeyId": "W7GBQGIUWDWKA6KAGL56",
    "secretAccessKey": "O6hBYRPCeRmE0d9lr3Frtc345QsWt3l+mrDgvrVT9oE",
    "endpoint": "sfo2.digitaloceanspaces.com "
  },

  "proofsConfig": {
    "proofsRequired" : 0
  },

  "azCredentials": {
    "accountName": "",
    "accountKey": ""
  },

  "gcCredentials": {
    "projectId": ""
  },

  "argsTransport": {
    "level": "warn",
    "handleExceptions": true,
    "stringify": true,
    "timestamp": true,
    "colorize": false,
    "json": true
  }
}

Save and close the vim editor.

The system returns you back to the prompt.

Task 6: Run the Gaia image with Docker

While your console is still in the the gaia/hub folder, build the gaia.hub image.

Enter the following docker command at the console command line.

docker build -t gaia.hub .

This build users the Dockerfile already in the gaia/hub folder. The output of the command is similar to the following:

....

npm WARN gaia-hub@2.3.4 No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 877 packages from 540 contributors and audited 3671 packages in 38.122s
found 0 vulnerabilities

Removing intermediate container b0aef024879f
---> 5fd126019708
Step 5/5 : CMD ["npm", "run", "start"]
 ---> Running in ae459cc0865b
Removing intermediate container ae459cc0865b
---> b1ced6c39784
Successfully built b1ced6c39784
Successfully tagged gaia.hub:latest

Run your Gaia hub image.

docker run --restart=always -v ~/gaia/hub/config.json:/src/hub/config.json -p 3000:3000 -e CONFIG_PATH=/src/hub/config.json gaia.hub

This runs your Gaia hub on port 3000. If everything runs successfully, the last line outputted from this command should be:

Successfully compiled 13 files with Babel.
{"level":"warn","message":"Listening on port 3000 in development mode","timestamp":"2019-01-23T16:35:05.216Z"}

If your command did run successfully, stop the service using the hotkey ctrl-c.

Run the the image again with this new command.

docker run --restart=always -v ~/gaia/hub/config.json:/src/hub/config.json -p 3000:3000 -e CONFIG_PATH=/src/hub/config.json -d gaia.hub

This command includes -d option to docker run. This runs Docker in detached mode, so that it runs in the background. You can run docker ps to see your running docker images, and get the id of your Gaia server.

root@moxiegirl:~/gaia/hub# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES
aeca7eea4a86        gaia.hub            "npm run start"     11 seconds ago      Up 10 seconds       0.0.0.0:3000->3000/tcp   musing_payne

At this point, your Gaia service is up and running. You can run docker logs MY_CONTAINER_ID with your running image's ID to see the logs of this server at any time.

Task 7: Set up an Nginx reverse proxy

In this task, you set up a simple Nginx reverse proxy to serve your Docker container through a public URL. You do this from the droplet console command line.

Install nginx into the droplet.
```
sudo apt-get install nginx
```
Enter y to confirm the installation.
Edit the nginx default configuration file.
```
vi /etc/nginx/sites-available/default
```

Inside the location / block (line 48), enter the following configuration:

location / {
       proxy_pass http://localhost:3000;
       proxy_http_version 1.1;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection 'upgrade';
       proxy_set_header Host $host;
       proxy_cache_bypass $http_upgrade;
 }

This simple configuration passes all requests through to your Gaia hub running at port 3000.

Save and close the file.

Run nginx -t to make sure you have no syntax errors.

root@moxiegirl:~/gaia/hub# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Restart nginx with your new configuration.
```
systemctl restart nginx
```
Allow access to your Gaia hub by exposing port 80 to the public.
```
ufw allow 80
```

Task 8: Test your Gaia server

Now, you are ready to test your Gaia server and make sure it is up and running.

Click on Droplets in the sidebar.
Find your Droplet running Gaia.
Copy the IP address for it.
In your browser, visit the page MY_DROPLET_IP/hub_info.

You should see a response from your Gaia hub!

The read_url_prefix should be combine from the bucket and endpoint create in your config.json file, for example, https://moxiegirl-hub-space.s3.amazonaws.com/.

Task 9: Configure a domain name

At this point, you can point a domain to your Gaia hub. Although it's not required, it is highly recommended. If you use a domain, you can migrate your Droplet to a different server (or even provider such as Azure or AWS) at any time, and still access it through the domain URL. Simply point your domain at the IP address for your Droplet. Use an A Record DNS type.

These instructions assume you have already created a free domain through the freenom service. To point this freenom domain to your Gaia hub, do the following:

Log into your freenom account.
Choose the Manage Freenom Domain tab.
Add an A record leave the Name field blank.

This record points your entire domain to the hub IP.
Save your changes.
Create a CNAME record.

For example, you can use the prefix www with your domain name. When you are done, your
Save your changes.

At this point, your DNS management should look similar to the following except that with your domain rather than the maryhub.ga domain.
After your changes propagate, visit your new domain at the hub_info page.

Task 10: Set up SSL

If you've configured a domain to point to your Gaia hub, then it's highly recommended that you set up SSL to connect to your hub securely. DigitalOcean provides how to setup SSL . Follow those instructions, to setup SSL. When completed, you'll be able to visit https://mygaiadomain.com/hub_info securely.

20 KiB Raw Blame History