Reworked EC2 tutorial, cleaned up admin article, and updated overview for storage hubs

.gitignore

@@ -1,6 +1,7 @@
 # OS or Editor folders
 .DS_Store
 node_modules
+.vscode
 
 # Jekyllg
 _site

src/common/navigation.yaml

@@ -98,7 +98,6 @@ sections:
       - path: /storage-hubs
         pages:
           - path: /overview
-          - path: /gaia-admin
         sections:
           - title: Tutorials
             usePageTitles: true

src/pages/storage-hubs/amazon-ec2-deploy.md

@@ -1,18 +1,154 @@
 ---
-title: Deploy on Amazon EC2
+title: Deploying Gaia Hub on Amazon EC2
 description: Use a template to deploy a Gaia hub on Amazon EC2
 ---
 
 ## Introduction
 
 The template provided on this page provides an easy way to deploy a Gaia hub directly to Amazon EC2. You can use this
-template to deploy your own Gaia hub to your Amazon Web Services account. Amazon EC2 is an affordable and convenient
+template to deploy your own Gaia hub to your Amazon Web Services (AWS) account. Amazon EC2 is an affordable and
-cloud computing provider. This template uses Amazon EC2 instance together with an S3 provider for file storage.
+convenient cloud computing provider. The template provides a one-click deploy for Amazon EC2 with an S3 provider for
+file storage.
 
-## Template
+## Prerequisites
+
+This procedure uses Amazon CloudFormation to configure an EC2 cloud compute provider to run the Gaia hub service with
+an S3 provider for file storage. You should have access to an AWS account either through your personal account or
+through a corporate account. This account should have permissions to create resources.
+
+Additionally, you must also own a domain name and be able to update the DNS records associated with that domain name.
+The procedure on this page uses a free domain created on [freenom][], generically the procedure used
+is similar on other domain name providers.
+
+## Launching the template
+
+Use a link in the table to launch the CloudFormation template in the AWS region that you wish to deploy a Gaia hub.
 
 | Template name            | Description                                                   | Launch                                                               |
 | ------------------------ | ------------------------------------------------------------- | -------------------------------------------------------------------- |
-| Gaia Hub EC2 (us-west-2) | Deploys a Gaia hub service on EC2 with an S3 storage provider | [![](/images/cloudformation-launch-stack-button.png)][ec2-us-west-2] |
+| Gaia Hub EC2 (us-east-1) | Deploys a Gaia hub service on EC2 with an S3 storage provider | [![](/images/cloudformation-launch-stack-button.png)][ec2-us-east-1] |
+
+## Task 1: Configure the CloudFormation template
+
+Before launching your Gaia hub, you must configure the template with the appropriate values for your hub and the domain
+it runs on.
+
+1. Launch the template using the **Launch stack** button in the preceding table.
+2. Review the **Prepare template** and **Template source** fields to ensure that they are populated.
+
+![CloudFormation specify template](/images/cloudformation-specify-template.png)
+
+3. Click **Next**.
+4. Specify configuration details for your Gaia hub:
+
+   i. Enter the **Stack name**. This name must be unique within your AWS account.
+
+   ![CloudFormation stack name](/images/cloudformation-stack-name.png)
+
+   ii. Enter the domain name on which the hub should run in the **DomainName** field. You must own this domain name and
+   be able to update the DNS records associated with it.
+
+   ![CloudFormation domain name](/images/cloudformation-domain-name.png)
+
+   iii. Enter an email address associated with the Gaia hub in the **EmailAddress** field. This should be a valid email
+   that you have access to.
+
+   ![CloudFormation email](/images/cloudformation-email.png)
+
+   iv. Enter the name of the S3 bucket to create for data storage in the **GaiaBucketName** field. The name will be
+   combined with the stack name to create a unique S3 bucket.
+
+   ![CloudFormation bucket name](/images/cloudformation-bucket.png)
+
+   v. Select an available **InstanceType** from the drop-down. The default value is `t2.micro`.
+
+   vi. In the **KeyName** drop-down, select an [EC2 KeyPair](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#KeyPairs:)
+   to enable SSH access to the EC2 instance. You should download the `.pem` keyfile for this pair from the EC2 console.
+
+   For more information see the [EC2 key pair documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#prepare-key-pair).
+
+   ![CloudFormation key name](/images/cloudformation-keyname.png)
+
+   vii. Leave the **SSHLocation** field with the default value of `0.0.0.0/0` to enable SSH access from any IP address.
+   If you wish to restrict SSH access to the EC2 instance to a certain IP, you can update this field.
+
+   viii. Select a subnet and virtual private cluster (VPC) from the **SubnetId** and **VpcId** drop-downs to designate
+   where to deploy the Gaia hub instance.
+
+   ![CloudFormation subnet and VPC](/images/cloudformation-subnet.png)
+
+5. Click **Next**.
+6. Configure stack options for your Gaia hub:
+
+   i. Enter any key-value pairs you wish to include as tags for your Gaia hub. These are optional and display-only, they
+   have no effect on the behavior of the CloudFormation stack.
+
+   ii. Select an IAM role for the Gaia hub. This is optional, if you don't specify am IAM role, the container runs
+   with the same permissions as your AWS account.
+
+7. Click **Next**.
+8. Review the configuration of your Gaia hub, and make any changes necessary.
+9. At the bottom of the page, select the checkbox to acknowledge that AWS may create IAM resources with custom names.
+
+![CloudFormation IAM resources](/images/cloudformation-iam-resources.png)
+
+10. Click **Create stack** to launch your Gaia hub. The AWS console displays the summary of the stack.
+
+## Task 2: Retrieve the public IP of your Gaia hub
+
+Your stack can take several minutes to launch. You can monitor the **Events** tab of your hub to review the current
+progress of the launch. When the launch is complete, the **Outputs** tab displays information about the hub. Select
+the **PublicIP** and copy it to configure your domain name.
+
+![CloudFormation outputs](/images/cloudformation-details.png)
+
+## Task 3: Configure a domain name
+
+Connect your domain to the Gaia hub EC2 instance by creating an `A` record for the domain DNS entry, and enter the
+public IP from the previous step. In general, this procedure will be similar depending on your hostname provider.
+
+If you are using a free domain from [freenom], use the following procedure:
+
+1. Log in to your freenom account.
+2. Under **Services** > **My Domains**, click **Manage Domain** next to the domain corresponding to your Gaia hub.
+
+![My freenom domains](/images/freenom-my-domains.png)
+
+3. Click **Manage Freenom DNS** in the tab bar.
+4. In the **Add Records** table, select `A` from the **Type** drop-down, then paste the public IP of your Gaia hub EC2
+   instance in the **Target** field.
+5. Click **Save Changes** to update the DNS record.
+
+-> It can take up to 15 minutes for your DNS record changes to propagate. In a terminal, use the command
+`dig A +short <yourdomain.co>` to check if the changes have propagated. If the output of this command is the container
+public IP, the changes were successful.
+
+## Accessing your Gaia hub with SSH
+
+To SSH into your Gaia hub EC2 host directly, you must have the keyfile used in container creation. Access the host with
+the following command in your terminal:
+
+```bash
+ssh -i <your keyfile.pem> admin@<public_ip_address>
+```
+
+## Making changes to your Gaia hub
+
+If you want to make changes to your Gaia hub, there are two options. You can delete the entire CloudFormation stack
+using the **Delete** button in the CloudFormation dashboard. Once you have deleted the stack, you can re-create one and
+modify your DNS to point at the new public IP.
+
+![CloudFormation delete](/images/cloudformation-delete.png)
+
+To modify the instance in place, navigate to the [EC2 instances console][] and type the instance name into the
+**Filter instances** field. Select the instance from the search suggestion, then click the instance to select it.
+
+On the **Tags** tab, you can click **Manage tags** to update the relevant key value pairs for the instance. If you make
+changes to these tags, click **Save**, then select **Reboot instance** from the **Instance state** drop-down. Changes
+to the tags don't take effect until the instance is rebooted.
+
+![EC2 instances](/images/ec2-instances.png)
 
-[ec2-us-west-2]: https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=Gaia&templateURL=https://cf-templates-18jq0t04gve7c-us-west-2.s3-us-west-2.amazonaws.com/2021124ByT-cf.yaml
+[ec2-us-east-1]: https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=Gaia&templateURL=https://cf-templates-18jq0t04gve7c-us-east-1.s3.amazonaws.com/cloudformation.yaml
+[freenom]: https://freenom.com
+[ec2 instances console]: https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances

src/pages/storage-hubs/gaia-admin.md

@@ -5,23 +5,29 @@ description: 'Storing user data with Stacks'
 
 ## Introduction
 
-A Gaia service can run a simple administrative service co-located with your Gaia hub. This service allows you to administer the Gaia hub with the help of an API key. Gaia hubs installed using the Gaia Amazon Machine Image (AMI) have this service integrated automatically.
+A Gaia service can run a simple administrative service co-located with your Gaia hub. This service allows you to
+administer the Gaia hub with the help of an API key. Gaia hubs installed using the Gaia Amazon Machine Image (AMI) have
+this service integrated automatically.
 
-In this section, you learn how to use the Gaia admin service with your Gaia hub.
+In this section, you learn how to use the Gaia administrator service with your Gaia hub.
 
--> The examples in this section assume that Gaia and the admin service were installed through the Configure a hub on Amazon EC2.
+-> The examples in this section assume that you installed the Gaia service using the
+[Deploy on Amazon EC2](/storage-hubs/amazon-ec2-deploy) tutorial.
 
 ## Understand the configuration files
 
-The admin service relies on two configuration files, the hub's configuration and the configuration of the admin service itself. The hub's configuration is mounted `/tmp/hub-config/config.json` in the `docker_admin_1` container. Your EC2 instance has the admin service configuration in the `/gaia/docker/admin-config/config.json` file.
+The administrator service relies on two configuration files, the hub's configuration and the configuration of the
+administrator service itself. You can find the hub's configration in `/tmp/hub-config/config.json` in the
+`docker_admin_1` container. Your EC2 instance has the administrator service configuration in the
+`/gaia/docker/admin-config/config.json` file.
 
-The admin service needs to know the following:
+The administrator service requires the following information:
 
-- where the Gaia hub config file is located
+- The location of the Gaia hub configuration file
 - which API keys to use when authenticating administrative requests
 - which commands to run to restart the Gaia hub on a config change
 
-The following is the standard admin service config installed with your EC2 instance.
+The following is the standard administrator service config installed with your EC2 instance.
 
 ```json
 {
@@ -40,7 +46,7 @@ The following is the standard admin service config installed with your EC2 insta
   },
   "reloadSettings": {
     "command": "/bin/sh",
-    "argv": ["-c", "docker restart docker_hub_1 &"],
+    "argv": ["-c", "docker restart gaia_hub_1 &"],
     "env": {},
     "setuid": 1000,
     "setgid": 1000
@@ -48,15 +54,17 @@ The following is the standard admin service config installed with your EC2 insta
 }
 ```
 
-The `port` is the port where Gaia is running. The `apiKeys` field is key used for making calls to the hub. The `gaiaSettings`
+The `port` is the port where Gaia is running. The `apiKeys` field is key used for making calls to the hub. The
+`gaiaSettings` field specifies the location of the Gaia hub configuration file.
 
-The `argsTransport` section configures the hub logging. The service uses the `winston` logging service. Refer to their documentation for full details on the [logging configuration options](https://github.com/winstonjs/winston).
+The `argsTransport` section configures the hub logging. The service uses the `winston` logging service. Refer to their
+documentation for full details on the [logging configuration options](https://github.com/winstonjs/winston).
 
 | Field           | Description                                                              |
 | --------------- | ------------------------------------------------------------------------ |
-| level           | Lowest level this transport will log. (default: `info`)                  |
+| level           | Lowest level this transport logs (default: `info`)                       |
 | handleException | Set to true to have this transport handle exceptions. (default: `false`) |
-| timestamp       | The timestamp when the message was received.                             |
+| timestamp       | The timestamp when of the message                                        |
 | stringify       | Converts the output to a JSON string.                                    |
 | colorize        | Colorizes the standard logging level                                     |
 | json            | Log format.                                                              |
@@ -68,35 +76,35 @@ The `reloadSettings` configure the command that is used to reload your Gaia hub.
 | command | A command which reloads the Gaia hub service.                                                                       |
 | argv    | An array containing the command arguments.                                                                          |
 | env     | This is a key/value list of any environment variables that need to be set for the command to run. This is optional. |
-| setuid  | This is the UID under which the command will be run. This is optional.                                              |
+| setuid  | This is the UID under which the command runs. This is optional.                                                     |
-| setgid  | This is the GID under which the command will run. This is optional.                                                 |
+| setgid  | This is the GID under which the command runs. This is optional.                                                     |
 
--> Please review the [JSON config schema](#optional-understand-the-configuration-file-schema) for a list of all available parameters and possible values.
+-> Review the [JSON config schema](#optional-understand-the-configuration-file-schema) for a list of all
+available parameters and possible values.
 
-## Using the admin service APIs
+## Using the administrator service APIs
 
-You use the admin service APIs to manage the hub. Administrating a hub requires
+You use the administrator service APIs to manage the hub. Administrating a hub requires that you make calls from a
-that you make calls from a terminal on your hub service server. To execute admin
+terminal on your hub service server. To execute administrator functions on a Gaia hub created with AWS, you `ssh` into
-functions on a Gaia hub created with AWS, you `ssh` into your instance as follows:
+your instance as follows:
 
 ```bash
-ssh -t -i <your keyfile.pem> -A core@<public ip address>
+ssh -i <your keyfile.pem> admin@<public_ip_address>
 ```
 
 You must also set the `API_KEY` in your environment:
 
 ```bash
-export API_KEY="hello"
+export API_KEY="<API_KEY>"
 ```
 
 You may find it useful to install a JSON processor such as `jq` to process the
-output of the admin commands.
+output of the administrator commands.
 
 ### Restart the Gaia Hub (`POST /v1/admin/reload`)
 
-The admin service will make changes to the Gaia hub's config file, but the
+The administrator service can make changes to the Gaia hub's config file, but the changes only take effect when the Gaia
-changes will only take effect when the Gaia hub is reloaded. You can do this
+hub reboots. You can do this as follows:
-as follows:
 
 ```bash
 export API_KEY="hello"
@@ -107,24 +115,21 @@ curl -H "Authorization: bearer $API_KEY" -X POST http://localhost:8009/v1/admin/
 { "result": "OK" }
 ```
 
-When you `POST` to this endpoint, the admin service runs the command described
+When you `POST` to this endpoint, the administrator service runs the command described in the `reloadSettings` section
-in the `reloadSettings` section of the config file. It attempts to spawn a
+of the config file. It attempts to spawn a subprocess from the given `reloadSettings.command` binary, and pass it the
-subprocess from the given `reloadSettings.command` binary, and pass it the
+arguments given in `reloadSettings.argv`. Note that the subprocess doesn't run in a user-accesssible shell.
-arguments given in `reloadSettings.argv`. Note that the subprocess will _NOT_
-be run in a shell.
 
 #### Errors
 
-If you do not supply a valid API key, this method fails with HTTP 403.
+If you don't supply a valid API key, this method fails with HTTP 403.
 
-This endpoint returns HTTP 500 if the reload command fails. If this
+This endpoint returns HTTP 500 if the reload command fails. If this happens, the return value contains the command's
-happens, you will get back the command's exit code and possibly the signal that
+exit code and the signal that killed.
-killed it.
 
 ### Get the hub configuration (`GET /v1/admin/config`)
 
-This endpoint is used to read and write a Gaia hub's non-driver-related
+This endpoint can to read and write a Gaia hub's non-driver-related settings. These include the port it listens on, and
-settings. These include the port it listens on, and its proof-checking
+its proof-checking
 settings.
 
 To read the Gaia hub settings, you would run the following:
@@ -140,8 +145,7 @@ curl -H "Authorization: bearer $API_KEY" http://localhost:8009/v1/admin/config
 
 ### Set the hub configuration (`POST /v1/admin/config`)
 
-To set Gaia hub settings, you simply `POST` the changed JSON fields to this
+To set Gaia hub settings, `POST` the changed JSON fields to this endpoint.
-endpoint.
 
 ```bash
 export API_KEY="hello"
@@ -152,7 +156,8 @@ curl -H "Authorization: bearer $API_KEY" -H 'Content-Type: application/json' -X
 { "message": "Config updated -- you should reload your Gaia hub now." }
 ```
 
-If the settings were successfully applied, the method returns a message to reload your Gaia hub. You can set multiple drivers' settings with a single call. For example, you can set:
+If the settings were successfully applied, the method returns a message to reload your Gaia hub. You can set multiple
+drivers' settings with a single call. For example, you can set:
 
 - The driver to use (`driver`)
 - The Gaia's read URL endpoint (`readURL`)
@@ -213,18 +218,19 @@ const GAIA_CONFIG_SCHEMA = {
 }
 ```
 
-The same fields are returned on `GET` within a `config` object.
+When performing a `GET` within a `config` object the return value contains the same fields.
 
 #### Errors
 
-If you do not supply a valid API key, both the `GET` and `POST` method fail with HTTP 403.
+If you don't supply a valid API key, both the `GET` and `POST` method fail with HTTP 403.
 
-Only relevant Gaia hub config fields are set. If you `POST` invalid settings
+In general, you should only set the relevent Gaia hub config fields. If you `POST` invalid settings values, you get an
-values, you get an HTTP 400 error.
+HTTP 400 error.
 
 ## Example: Read and write driver settings
 
-Use the `/v1/admin/config` endpoint to read and write storage driver settings. To get the current driver settings, you would run:
+Use the `/v1/admin/config` endpoint to read and write storage driver settings. To get the current driver settings, run
+the following commands in a terminal:
 
 ```bash
 export API_KEY="hello"
@@ -242,7 +248,7 @@ curl -H "Authorization: bearer $API_KEY" http://localhost:8009/v1/admin/config
 }
 ```
 
-To update the driver settings, you would run:
+To update the driver settings, run the following commands in a terminal:
 
 ```bash
 export API_KEY="hello"
@@ -259,7 +265,7 @@ curl -H "Authorization: bearer $API_KEY" -H 'Content-Type: application/json' -X
 
 This endpoint lets you read and write the `whitelist` section of a Gaia hub, to control who can write to it and list its files.
 
-To get the current whitelist, you would run the following:
+To get the current whitelist, run the following commands in a terminal:
 
 ```bash
 export API_KEY="hello"
@@ -270,7 +276,7 @@ curl -H "Authorization: bearer $API_KEY" http://localhost:8009/v1/admin/config
 { "config": { "whitelist": ["15hUKXg1URbQsmaEHKFV2vP9kCeCsT8gUu"] } }
 ```
 
-To set the whitelist, you must set the _entire_ whitelist. To set the list, pass a command such as the following:
+To set the whitelist, you must set the _entire_ whitelist. To set the list, run the following command in a terminal:
 
 ```bash
 export API_KEY="hello"
@@ -281,9 +287,10 @@ curl -H "Authorization: bearer $API_KEY" -H 'Content-Type: application/json' -X
 { "message": "Config updated -- you should reload your Gaia hub now." }
 ```
 
-## View logs for the hub or admin service
+## View logs for the hub or administrator service
 
-The logs for each Gaia service are maintained by their respective Docker containers. To view the log for a particular service, use the `docker logs` command. For example, to get the logs for the hub:
+The Docker container for each Gaia service contain the logs for that service. To view the log for a particular service,
+use the `docker logs` command. For example, to get the logs for the hub:
 
 ```bash
 docker logs docker_hub_1
@@ -487,4 +494,4 @@ The following JSON schema details the possible parameters for a hub configuratio
 }
 ```
 
--> A full list of examples can be found in [the Gaia repository on GitHub](https://github.com/blockstack/gaia/tree/master/hub)
+-> A full list of examples are in [the Gaia repository on GitHub](https://github.com/blockstack/gaia/tree/master/hub)

src/pages/storage-hubs/overview.md

@@ -1,5 +1,6 @@
 ---
 title: Storage hubs overview
+description: Securely store application and user data off-chain
 ---
 
 ## Introduction
@@ -13,64 +14,3 @@ A [Gaia hub](/build-apps/references/gaia#user-control-or-how-is-gaia-decentraliz
 resource, generally hosted on the same cloud compute provider. The hub service requires an authentication token from a
 storage requestor, and writes key-value pairs to the associated storage resource. Storage requestors can choose a Gaia
 hub provider. This documentation provides an overview of how to set up and operate a Gaia hub.
-
-## Creating a configuration file
-
-You should store a JSON configuration file either in the top-level directory of the hub server. Alternatively, you can
-specify a file location using the `CONFIG_PATH` environment variable. The following is an example configuration file for
-Amazon S3:
-
-```json
-{
-  "servername": "localhost",
-  "port": 4000,
-  "driver": "aws",
-  "readURL": "https://YOUR_BUCKET_NAME.s3.amazonaws.com/",
-  "pageSize": 20,
-  "bucket": "YOUR_BUCKET_NAME",
-  "awsCredentials": {
-    "accessKeyID": "YOUR_ACCESS_KEY",
-    "secretAccessKey": "YOUR_SECRET_KEY"
-  },
-  "argsTransport": {
-    "level": "debug",
-    "handleExceptions": true,
-    "stringify": true,
-    "timestamp": true,
-    "colorize": false,
-    "json": true
-  }
-}
-```
-
-You can specify the logging level, the backend driver, the credentials for that backend driver, and the `readURL` of the
-hub. Typically, this is the URL for the compute resource on the cloud computing provider, where the hub service is
-running.
-
-### Requiring the correct hub URL
-
-Enabling the `requireCorrectHubUrl` option in your `config.json` file, requires that authentication requests correctly
-include the `hubURL` for the hub the request is trying to connect with. Use this option to prevent a malicious Gaia hub
-from using an authentication token for itself on other Gaia hubs.
-
-By default, the Gaia hub validates that the supplied URL matches `https://${config.servername}`, but if there are
-multiple valid URLs for clients to reach the hub at, you can include a list in your `config.json`:
-
-```json
-{
-  ...
-  "servername": "normalserver.com",
-  "validHubUrls": [ "https://specialserver.com/",
-                  "https://legacyurl.info" ],
-  ...
-}
-```
-
-### Configuring read URLs
-
-By default, hub drivers return read URLs that point directly at the written content. For example, an S3 driver would
-return the URL directly to the S3 file. If you configure a CDN or domain to point at that same bucket, you can use the
-`readURL` parameter to tell the hub to read files from a given URL. For example, the configuration of the
-`hub.blockstack.org` Gaia Hub results in a read URL that looks like `https://gaia.blockstack.org/hub/`.
-
-Unset the `readURL` parameter if you don't intend to deploy any caching.