mirror of https://github.com/lukechilds/docs.git
Browse Source
* Adding in new Gaia material Closes #104, Closes #72, Closes #80, Closes #81,Closes #111, Closes blockstack/gaia#184, Closes blockstack/gaia#191 Entering review comments Signed-off-by: Mary Anthony <mary@blockstack.com> * Updating with more comments from this morning Signed-off-by: Mary Anthony <mary@blockstack.com> * Updating with last minute comments Signed-off-by: Mary Anthony <mary@blockstack.com>feat/clarity-updates
Moxiegirl
6 years ago
committed by
GitHub
GitHub
70 changed files with 2302 additions and 349 deletions
@ -1,156 +0,0 @@ |
|||
# Glossary |
|||
|
|||
Commonly used terms and jargon in Blockstack |
|||
|
|||
## Account |
|||
|
|||
A field in a profile that links the name to an existing service, like Twitter or OpenBazaar. They are listed under the `accounts` listing in a profile. |
|||
|
|||
Some accounts serve as social proofs, but they can contain any data the user wants. |
|||
|
|||
## Atlas |
|||
|
|||
A peer-to-peer network maintained by Blockstack Core nodes that stores each |
|||
name's zone files and immutable data. See [this document]({{ site.baseurl }}/atlas/overview.html) for |
|||
details. |
|||
|
|||
## Blockstack ID |
|||
|
|||
(Also called a "name"). |
|||
|
|||
A human-readable name in Blockstack. It is comprised only of upper and lower-case ASCII characters, numbers, as well as `-`, `_`, and `.`. It must end with a `.`, followed by a namespace ID. It has at least 3 characters, and at most 37 (including the `.` and the namespace ID). |
|||
|
|||
Anyone can register a Blockstack ID, such as through the [Blockstack Browser](https://github.com/blockstack/blockstack-browser) |
|||
|
|||
## Blockstack Core |
|||
|
|||
A server that reads a blockchain with [virtualchain](https://github.com/blockstack/blockstack-virtualchain), filters out transactions that represent name operations, and builds up a database of (name, public key, state value) triples. |
|||
|
|||
## Blockstack Naming Service (BNS) |
|||
|
|||
This is the naming protocol that Blockstack Core implements. See [this |
|||
document]({{ site.baseurl }}/core/naming/introduction.html) for details. |
|||
|
|||
## Consensus Hash |
|||
|
|||
A cryptographic hash that represents a proof-of-computation by a Blockstack Core node. Two Blockstack Core nodes have seen and processed the same name operations up to block `n` if and only if they each calculate the same consensus hash at height `n`. |
|||
|
|||
A Blockstack Core node only accepts a name operation if it has a previously-calculated but recent consensus hash. Blockstack clients obtain a consensus hash from a Blockstack Core node in order to construct a name operation. |
|||
|
|||
## Gaia |
|||
|
|||
This is Blockstack's storage system. Gaia hosts all of your app-specific data. |
|||
|
|||
## Gaia Hub |
|||
|
|||
This is a publicly-routable server that serves as an entry point for Gaia data. |
|||
Anyone can stand up and run a Gaia hub by following [these |
|||
instructions](https://github.com/blockstack/gaia). |
|||
Blockstack provides a [default Gaia hub](https://gaia.blockstack.org). |
|||
|
|||
## Immutable Data |
|||
|
|||
This is the general term for chunks of data whose hash is cryptographically |
|||
bound to a blockchain transaction. This includes all data stored in the Atlas |
|||
network (such as your Blockstack ID's zone file), |
|||
as well as any data whose hash is stored in the Atlas network. |
|||
|
|||
## mnemonic |
|||
|
|||
A human-friendly representation of a seed. |
|||
|
|||
## Mutable Data |
|||
|
|||
This is the general term for data that is (1) signed by your Blockstack ID, and |
|||
(2) can be looked up using your Blockstack ID. This includes all your Gaia |
|||
data, as well as your profile. |
|||
|
|||
## Name |
|||
|
|||
See Blockstack ID. |
|||
|
|||
## Name Database |
|||
|
|||
The set of (name, public key, name state) triples that the Blockstack Core node generates by reading the blockchain. The name state is usually the hash of a DNS zone file stored in Atlas. |
|||
|
|||
## Name Operation |
|||
|
|||
A specially-crafted transaction in the underlying blockchain that, when processed, will change each Blockstack Core's name database. Examples include `NAME_PREORDER` (preorders a name), `NAME_REGISTRATION` (registers a name), `NAME_UPDATE` (changes a name's zonefile hash), `NAME_TRANSFER` (changes a name's public key), and `NAME_REVOKE` (locks everyone out of a name until it expires). |
|||
|
|||
Name operations are encoded on Bitcoin as `OP_RETURN` outputs that start with `id`, followed by a 1-byte character that identifies the particular operation. |
|||
|
|||
See the [wire format](wire-format.md) document for details. |
|||
|
|||
## Namespace |
|||
|
|||
Analogous to a DNS TLD, it represents a grouping of names. All names under the same namespace have the same pricing and lifetime rules. |
|||
|
|||
Anyone can create a namespace, but doing so is expensive by design. See the |
|||
[namespace creation]({{ site.baseurl }}/core/naming/tutorial_creation.html) tutorial for details. |
|||
|
|||
## Preorder |
|||
|
|||
The first of two steps to acquire a name. This operation writes the hash of both the name and the address that will own it. |
|||
|
|||
## Profile |
|||
|
|||
A signed JSON web token that describes a [Person](https://schema.org/Person), which describes the name's owner. You can put anything you want into your profile. |
|||
|
|||
Additionally, profiles hold lists of social verifications and pointers to your Gaia data. |
|||
|
|||
## Register |
|||
|
|||
(1) The act of acquiring a name in Blockstack. |
|||
|
|||
(2) The second of two steps to create a new name entry in the name database. Reveals the name as plaintext to the world. Must match a recent preorder to be accepted. |
|||
|
|||
## Registrar |
|||
|
|||
An online service that lets you sign up for and manage the profiles of Blockstack IDs. |
|||
|
|||
## Resolver |
|||
|
|||
An online service that displays zonefile and profile data for a Blockstack ID. [The Blockstack Explorer](https://explorer.blockstack.org) is a resolver. |
|||
|
|||
## Seed |
|||
|
|||
A 128 or 256 bits of random data used to generate a master public and privte key pair. |
|||
|
|||
## Social proof |
|||
|
|||
A post in an account on an existing Web service like Twitter, Facebook, or GitHub that points back to a Blockstack ID. Used to provide some evidence that the person who owns the Blockstack ID is also the person who owns the Web service account. |
|||
|
|||
Social proofs are listed in your profile. |
|||
|
|||
## Storage Provider |
|||
|
|||
This is any service that can serve your zone file, profile, or data. In all cases, the data is signed by one of your wallet's keys, so you can use any provider without having to worry about it trying to change the data. |
|||
|
|||
Storage providers are accessed through a Gaia hub. Gaia hubs ship with drivers |
|||
that allow them to treat storage providers as dumb hard drives, which store |
|||
signed encrypted data on the hub's behalf. |
|||
|
|||
Not all storage providers support writes--some of them are read-only. |
|||
|
|||
Supported storage providers today include: |
|||
* Amazon S3 |
|||
* Dropbox |
|||
* Your harddrive |
|||
* Any HTTP/HTTPS/FTP server (read-only) |
|||
* Any public-use Gaia hub |
|||
* IPFS |
|||
|
|||
Support is being added for: |
|||
* Google Drive |
|||
* Microsoft OneDrive |
|||
* Box.com |
|||
* BitTorrent |
|||
|
|||
If you have a preferred storage provider, and you're a developer, please consider sending us a pull request to add support for it! |
|||
|
|||
## Zone file |
|||
|
|||
A specially-formatted file that stores routing information for a Blockstack ID. |
|||
Blockstack clients use your zone file to find out where your preferred Gaia |
|||
hub(s) are. Ever Blockstack Core node stores a copy of every zone file for |
|||
every Blockstack ID by participating in the Atlas network. |
|||
|
After Width: | Height: | Size: 89 KiB |
|
Can't render this file because it contains an unexpected character in line 14 and column 0.
|
@ -1,18 +0,0 @@ |
|||
- title: About |
|||
docs: |
|||
- gaia/overview |
|||
- gaia/control |
|||
- gaia/writeread |
|||
- gaia/access |
|||
|
|||
- title: Tutorials |
|||
docs: |
|||
- gaia/hello-gaia |
|||
- gaia/media |
|||
- gaia/sharing |
|||
|
|||
- title: How to |
|||
docs: |
|||
- gaia/run-hub |
|||
- gaia/configure |
|||
- gaia/test |
|||
@ -0,0 +1,19 @@ |
|||
- title: Introduction to Gaia storage |
|||
docs: |
|||
- storage/overview |
|||
- storage/authentication |
|||
- storage/write-to-read |
|||
- storage/hub-choice |
|||
|
|||
- title: How to hub |
|||
docs: |
|||
- storage/hub-operation |
|||
- storage/amazon-s3-deploy |
|||
- storage/hello-hub-choice |
|||
- storage/gaia-admin |
|||
|
|||
- title: Reference |
|||
docs: |
|||
- storage/config-schema |
|||
- storage/cliDocs |
|||
- common/javascript_ref |
|||
@ -0,0 +1,22 @@ |
|||
|
|||
 |
|||
|
|||
Blockchains require consensus among large numbers of people, so they can be slow. Additionally, a blockchain is not designed to hold a lot of data. This means using a blockchain for every bit of data a user might write and store is expensive. For example, imagine if an application were storing every tweet in the chain. |
|||
|
|||
Blockstack addresses blockchain performance problems using a layered approach. At the base of the system is a blockchain and the Blockstack Naming System (BNS). The blockchain governs ownership of names (identities) in the system, names such as domain names, usernames, and application names. |
|||
|
|||
Names in Blockstack correspond to routing data in the OSI stack. The routing data is stored in the Atlas Peer Network, the second layer. Every core node that joins the Blockstack Network is able to obtain an entire copy of this routing data. Blockstack uses the routing data to associate names (usernames, domains, and application names) with a particular storage location. |
|||
|
|||
The final layer is the Gaia Storage System. A Gaia system consists of a _hub |
|||
service_ and storage resource on a cloud software provider such as Azure, |
|||
DigitalOcean, Amazon EC2, and so forth. Typically the compute resource and the |
|||
storage resource belong to same cloud vendor. Gaia currently has |
|||
driver support for S3 and Azure Blob Storage, but the driver model allows for |
|||
other backend support as well. |
|||
|
|||
Because Gaia stores application and user data off the blockchain, a Blockstack |
|||
DApp is typically more performant than DApps created on other blockchains. |
|||
Moreover, users choose where their data lives, and Gaia enables applications |
|||
to access that user data via a uniform API. When the user logs in, |
|||
the authentication process gives the application the URL of a Gaia hub, which |
|||
then writes to storage on behalf of that user. |
|||
@ -1,148 +0,0 @@ |
|||
|
|||
|
|||
Commonly used terms and jargon in Blockstack |
|||
|
|||
## Account |
|||
|
|||
A field in a profile that links the name to an existing service, like Twitter or OpenBazaar. They are listed under the `accounts` listing in a profile. |
|||
|
|||
Some accounts serve as social proofs, but they can contain any data the user wants. |
|||
|
|||
## Atlas |
|||
|
|||
A peer-to-peer network maintained by Blockstack Core nodes that stores each |
|||
name's zone files and immutable data. See [this document](atlas_network.md) for |
|||
details. |
|||
|
|||
## Blockstack ID |
|||
|
|||
(Also called a "name"). |
|||
|
|||
A human-readable name in Blockstack. It is comprised only of upper and lower-case ASCII characters, numbers, as well as `-`, `_`, and `.`. It must end with a `.`, followed by a namespace ID. It has at least 3 characters, and at most 37 (including the `.` and the namespace ID). |
|||
|
|||
Anyone can register a Blockstack ID, such as through the [Blockstack Browser](https://github.com/blockstack/blockstack-browser) |
|||
|
|||
## Blockstack Core |
|||
|
|||
A server that reads a blockchain with [virtualchain](https://github.com/blockstack/blockstack-virtualchain), filters out transactions that represent name operations, and builds up a database of (name, public key, state value) triples. |
|||
|
|||
## Blockstack Naming Service (BNS) |
|||
|
|||
This is the naming protocol that Blockstack Core implements. See [this |
|||
document](blockstack_naming_service.md) for details. |
|||
|
|||
## Consensus Hash |
|||
|
|||
A cryptographic hash that represents a proof-of-computation by a Blockstack Core node. Two Blockstack Core nodes have seen and processed the same name operations up to block `n` if and only if they each calculate the same consensus hash at height `n`. |
|||
|
|||
A Blockstack Core node only accepts a name operation if it has a previously-calculated but recent consensus hash. Blockstack clients obtain a consensus hash from a Blockstack Core node in order to construct a name operation. |
|||
|
|||
## Gaia |
|||
|
|||
This is Blockstack's storage system. Gaia hosts all of your app-specific data. |
|||
|
|||
## Gaia Hub |
|||
|
|||
This is a publicly-routable server that serves as an entry point for Gaia data. |
|||
Anyone can stand up and run a Gaia hub by following [these |
|||
instructions](https://github.com/blockstack/gaia). |
|||
Blockstack provides a [default Gaia hub](https://gaia.blockstack.org). |
|||
|
|||
## Immutable Data |
|||
|
|||
This is the general term for chunks of data whose hash is cryptographically |
|||
bound to a blockchain transaction. This includes all data stored in the Atlas |
|||
network (such as your Blockstack ID's zone file), |
|||
as well as any data whose hash is stored in the Atlas network. |
|||
|
|||
## Mutable Data |
|||
|
|||
This is the general term for data that is (1) signed by your Blockstack ID, and |
|||
(2) can be looked up using your Blockstack ID. This includes all your Gaia |
|||
data, as well as your profile. |
|||
|
|||
## Name |
|||
|
|||
See Blockstack ID. |
|||
|
|||
## Name Database |
|||
|
|||
The set of (name, public key, name state) triples that the Blockstack Core node generates by reading the blockchain. The name state is usually the hash of a DNS zone file stored in Atlas. |
|||
|
|||
## Name Operation |
|||
|
|||
A specially-crafted transaction in the underlying blockchain that, when processed, will change each Blockstack Core's name database. Examples include `NAME_PREORDER` (preorders a name), `NAME_REGISTRATION` (registers a name), `NAME_UPDATE` (changes a name's zonefile hash), `NAME_TRANSFER` (changes a name's public key), and `NAME_REVOKE` (locks everyone out of a name until it expires). |
|||
|
|||
Name operations are encoded on Bitcoin as `OP_RETURN` outputs that start with `id`, followed by a 1-byte character that identifies the particular operation. |
|||
|
|||
See the [wire format](wire-format.md) document for details. |
|||
|
|||
## Namespace |
|||
|
|||
Analogous to a DNS TLD, it represents a grouping of names. All names under the same namespace have the same pricing and lifetime rules. |
|||
|
|||
Anyone can create a namespace, but doing so is expensive by design. See the |
|||
[namespace creation](namespace_creation.md) tutorial for details. |
|||
|
|||
## Preorder |
|||
|
|||
The first of two steps to acquire a name. This operation writes the hash of both the name and the address that will own it. |
|||
|
|||
## Profile |
|||
|
|||
A signed JSON web token that describes a [Person](https://schema.org/Person), which describes the name's owner. You can put anything you want into your profile. |
|||
|
|||
Additionally, profiles hold lists of social verifications and pointers to your Gaia data. |
|||
|
|||
## Register |
|||
|
|||
(1) The act of acquiring a name in Blockstack. |
|||
|
|||
(2) The second of two steps to create a new name entry in the name database. Reveals the name as plaintext to the world. Must match a recent preorder to be accepted. |
|||
|
|||
## Registrar |
|||
|
|||
An online service that lets you sign up for and manage the profiles of Blockstack IDs. |
|||
|
|||
## Resolver |
|||
|
|||
An online service that displays zonefile and profile data for a Blockstack ID. [The Blockstack Explorer](https://explorer.blockstack.org) is a resolver. |
|||
|
|||
## Social proof |
|||
|
|||
A post in an account on an existing Web service like Twitter, Facebook, or GitHub that points back to a Blockstack ID. Used to provide some evidence that the person who owns the Blockstack ID is also the person who owns the Web service account. |
|||
|
|||
Social proofs are listed in your profile. |
|||
|
|||
## Storage Provider |
|||
|
|||
This is any service that can serve your zone file, profile, or data. In all cases, the data is signed by one of your wallet's keys, so you can use any provider without having to worry about it trying to change the data. |
|||
|
|||
Storage providers are accessed through a Gaia hub. Gaia hubs ship with drivers |
|||
that allow them to treat storage providers as dumb hard drives, which store |
|||
signed encrypted data on the hub's behalf. |
|||
|
|||
Not all storage providers support writes--some of them are read-only. |
|||
|
|||
Supported storage providers today include: |
|||
* Amazon S3 |
|||
* Dropbox |
|||
* Your harddrive |
|||
* Any HTTP/HTTPS/FTP server (read-only) |
|||
* Any public-use Gaia hub |
|||
* IPFS |
|||
|
|||
Support is being added for: |
|||
* Google Drive |
|||
* Microsoft OneDrive |
|||
* Box.com |
|||
* BitTorrent |
|||
|
|||
If you have a preferred storage provider, and you're a developer, please consider sending us a pull request to add support for it! |
|||
|
|||
## Zone file |
|||
|
|||
A specially-formatted file that stores routing information for a Blockstack ID. |
|||
Blockstack clients use your zone file to find out where your preferred Gaia |
|||
hub(s) are. Ever Blockstack Core node stores a copy of every zone file for |
|||
every Blockstack ID by participating in the Atlas network. |
|||
@ -0,0 +1,480 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Configure a hub on Amazon EC2 |
|||
{:.no_toc} |
|||
|
|||
This teaches you how to run a Gaia hub on Amazon EC2. Amazon EC2 is an affordable and convenient cloud computing provider. This example uses Amazon EC2 together with an EB3 instance for file storage. |
|||
|
|||
* TOC |
|||
{:toc} |
|||
|
|||
<div class="uk-card uk-card-default uk-card-body"> |
|||
<h5>Is this tutorial for you?</h5> |
|||
|
|||
<p>This documentation is appropriate for advanced power users who are familiar with command line tools, <code>ssh</code>, and basic editing configuration files.</p> |
|||
|
|||
<p>If you are planning on running an <emphasis>open-membership hub</emphasis> or an <emphasis>application-specific hub</emphasis>, see <a href="hub-operation.html">the section on Hub Operation</a>.</p> |
|||
|
|||
</div> |
|||
|
|||
## Prerequisites you need |
|||
|
|||
This procedure uses Amazon AWS to choose and configure an Amazon Machine Image |
|||
(AMI) running a Gaia service. For this reason, you should have an AWS account |
|||
on the <a href="https://aws.amazon.com/free/" target="\_blank">Amazon AWS free |
|||
tier</a>, personal account, or corporate account. These instructions assume you |
|||
are using a free tier account. |
|||
|
|||
These instructions assume you have already created a free <a |
|||
href="https://www.freenom.com" target="\_blank">domain through the freenom |
|||
service</a>. If you have another domain, you can use that instead. |
|||
|
|||
Finally, setting up the SSL certificates on your EC2 instance requires you to use the terminal command line on your workstation. Make sure you have the `watch` command installed using the `which` command. |
|||
|
|||
``` |
|||
$ which watch |
|||
/usr/local/bin/watch |
|||
``` |
|||
|
|||
If `watch` is not located, install it on your workstation. |
|||
|
|||
## Task 1: Launch an EC2 instance |
|||
|
|||
1. Visit the <a href="https://aws.amazon.com/free/" target="\_blank">AWS Free Tier page</a> and choose **Sign in to the Console**. |
|||
|
|||
 |
|||
|
|||
2. Make sure your region is set to one close to you. |
|||
|
|||
 |
|||
|
|||
3. Under **Build a solution** choose **Launch a virtual machine**. |
|||
|
|||
The system opens the EC2 dashboard. |
|||
|
|||
4. Enter `Blockstack Gaia` in the search bar. |
|||
|
|||
The system finds AMIs in the Marketplace and the Community. |
|||
|
|||
5. Choose **Community AMIs**. |
|||
|
|||
The system displays the available Gaia hub images. |
|||
|
|||
 |
|||
|
|||
6. Select the most recent version of the image. |
|||
|
|||
Each image name has this format: |
|||
|
|||
`blockstack-gaia_hub-STORAGETYPE-VERSION-hvm - ami-BUILDTAG` |
|||
|
|||
So, the `blockstack-gaia_hub-ephemeral-0001.0.1-hvm - ami-0425cf8c91bb2d331` image uses ephemeral storage, is at version `0001.0.1` and has the `0425cf8c91bb2d331` tag. |
|||
|
|||
You can choose an image that uses ephemeral or EBS storage. The ephemeral |
|||
storage is very small but free. Only choose this if you plan to test or use |
|||
a personal hub. Otherwise, choose the AMI for elastic block storage (EBS). |
|||
|
|||
After you select an image, the system displays **Step 2: Choose an Instance Type** page. |
|||
|
|||
 |
|||
|
|||
7. Select **t2.micro** and choose **Next: Configure Instance Details**. |
|||
|
|||
<hr> |
|||
To configure instance details, do the following: |
|||
|
|||
1. Select a VPC. |
|||
|
|||
A default VPC is created with a free tier account. You can use this |
|||
default VPC. Or you can choose another VPC. If you choose another VPC, |
|||
ensure the `Subnet` value is set to a subnet reachable by a public IP. |
|||
|
|||
{% include important.html content="If you're using a private subnet, you |
|||
should attach an elastic IP (EIP) to the VM. This EIP allows you to |
|||
reboot the instance without worrying whether the address will reset. To |
|||
attach an IP, <strong>press allocate new address</strong> and follow the |
|||
instructions to attach the EIP to your new EC2 instance." %} |
|||
|
|||
2. Set **Protect against accidental termination**. |
|||
|
|||
If you terminate a Gaia instance, you lose all the data associated with it. Protection adds an extra step to terminating your Gaia instance. |
|||
|
|||
3. Open the **Advanced Details**. |
|||
|
|||
At this point, you are going to configure environment variables for your instance. |
|||
|
|||
4. Paste the following into the **Advanced Details**. |
|||
|
|||
```json |
|||
{ |
|||
"ignition": { "version": "2.2.0" }, |
|||
"storage": { |
|||
"files": [{ |
|||
"filesystem": "root", |
|||
"path": "/etc/environment", |
|||
"mode": 420, |
|||
"contents": { |
|||
"source": "data:application/octet-stream,API_KEY%3D<KEYPHRASE>%0ADOMAIN%3D<DOMAIN_NAME_VALUE>%0ASTAGING%3D<STAGING_VALUE>" |
|||
} |
|||
}] |
|||
} |
|||
} |
|||
``` |
|||
|
|||
5. Replace the following values in the JSON. |
|||
|
|||
<table class="uk-table uk-table-small uk-table-divider"> |
|||
<tr> |
|||
<th>Value</th> |
|||
<th>Description</th> |
|||
</tr> |
|||
<tr> |
|||
<td><code><KEYPHRASE></code></td> |
|||
<td>A phrase to pass when using the hub admin. For example, <code>hubba</code> is a fun key phrase.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code><DOMAIN_NAME_VALUE></code></td> |
|||
<td>Your hub's domain name. For example, <code>maryhub.ml</code> is the domain name in this example.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code><STAGING_VALUE></code></td> |
|||
<td><p>Indicates what type of SSL to create, testing (`1`) or production (`0`). Set testing if you want to test without worrying about rate limiting. A testing cerificate is not secure.</p> |
|||
|
|||
<p>For this tutorial, use production (`0`).</p> |
|||
</td> |
|||
</tr> |
|||
</table> |
|||
|
|||
6. Check your **Advanced Details** they should look similar to the following: |
|||
|
|||
``` |
|||
{ |
|||
"ignition": { "version": "2.2.0" }, |
|||
"storage": { |
|||
"files": [{ |
|||
"filesystem": "root", |
|||
"path": "/etc/environment", |
|||
"mode": 420, |
|||
"contents": { |
|||
"source": "data:application/octet-stream,API_KEY%3Dhubba%0ADOMAIN%3Dmaryhub.ml%0ASTAGING%3D0" |
|||
} |
|||
}] |
|||
} |
|||
} |
|||
``` |
|||
|
|||
At this point, you have configured your instance details. |
|||
<hr> |
|||
|
|||
8. Choose **Next: Add Storage**. |
|||
|
|||
 |
|||
|
|||
The storage is set according to the AMI you selected. |
|||
|
|||
9. Choose **Next: Add tags**. |
|||
10. Add a **Key** of `purpose` with the **Value** `gaia`. |
|||
|
|||
 |
|||
|
|||
11. Choose **Next: Configure Security Group**. |
|||
12. Create a security group with the following three types: |
|||
|
|||
<table class="uk-table uk-table-small uk-table-divider"> |
|||
<tr> |
|||
<th>Type</th> |
|||
<th>Protocol</th> |
|||
<th>Port Range</th> |
|||
<th>Source</th> |
|||
<th>Description</th> |
|||
</tr> |
|||
<tr> |
|||
<td>SSH</td> |
|||
<td>TCP</td> |
|||
<td>22</td> |
|||
<td>My IP</td> |
|||
<td>optional</td> |
|||
</tr> |
|||
<tr> |
|||
<td>HTTP</td> |
|||
<td>TCP</td> |
|||
<td>80</td> |
|||
<td>Anywhere</td> |
|||
<td>optional</td> |
|||
</tr> |
|||
<tr> |
|||
<td>HTTPS</td> |
|||
<td>TCP</td> |
|||
<td>443</td> |
|||
<td>Anywhere</td> |
|||
<td>optional</td> |
|||
</tr> |
|||
</table> |
|||
|
|||
13. Choose **Review and Launch**. |
|||
|
|||
The system may warn you that the selection is not free tier eligible. You can ignore this for now. |
|||
|
|||
14. Press **Launch**. |
|||
|
|||
The system prompts you for a key pair. |
|||
|
|||
15. Select **Create a new keypair** or **Choose an existing key pair**. |
|||
16. Select **Launch Instances**. |
|||
|
|||
The system launches your instance. |
|||
|
|||
 |
|||
|
|||
During the launch process the machine starts and runs some initial setup processes. These processes take a few minutes depending on the network, typically launching does not take more than 10 minutes. While this is happening the instance **Status Checks** reflect the **Initializing** status. |
|||
|
|||
 |
|||
|
|||
|
|||
## Task 2: Test your Gaia server |
|||
|
|||
Now, you are ready to test your Gaia server and make sure it is up and running. |
|||
|
|||
1. Visit the <a href="https://aws.amazon.com/free/" target="\_blank">AWS Free Tier page</a> and choose **Sign in to the Console**. |
|||
|
|||
 |
|||
|
|||
2. Choose **All services > EC2**. |
|||
|
|||
The system displays the **EC2 Dashboard**. |
|||
|
|||
 |
|||
|
|||
3. Select **Running Instances**. |
|||
|
|||
The system displays your running instances. |
|||
|
|||
4. Locate your recently launched Gaia server. |
|||
|
|||
Make sure the instance shows as running and **Status Checks** are complete. |
|||
Completed status checks ensures the Gaia processes and service were started. |
|||
|
|||
5. Select the **Description** tab. |
|||
|
|||
 |
|||
|
|||
6. Locate the **IPv4 Public IP** value. |
|||
7. Copy the IP and paste it in your browser. |
|||
|
|||
You should see a message that your connection is not private. |
|||
|
|||
8. Press **Advanced**. |
|||
|
|||
 |
|||
|
|||
9. Choose to proceed. |
|||
10. Extend the IP with the `PUBLIC_IP/hub_info` tag like so. |
|||
|
|||
You should see a response from your Gaia hub! |
|||
|
|||
 |
|||
|
|||
At this point, you should see a **Not secure** message in the browser. |
|||
That's because you haven't yet enabled SSL certification. While `HTTPS` is |
|||
not required simple to run the hub services, Blockstack will only connect to |
|||
a hub and write to its storage over a valid `HTTPS` connection. |
|||
|
|||
|
|||
## Task 3: Configure a domain name |
|||
|
|||
At this point, you can point a domain to your Gaia hub. Although it's not required, it is highly recommended. If you use a domain, you can migrate your instance to a different server (or even provider such as Azure or Dropbox) at any time, and still access it through the domain URL. Just point your domain at the IP address for your EC2 instance. Use an `A Record` DNS type. |
|||
|
|||
These instructions assume you have already created a free <a href="https://www.freenom.com" target="\_blank">domain through the freenom service</a>. To point this freenom domain to your Gaia hub, do the following: |
|||
|
|||
1. Log into your freenom account. |
|||
2. Choose the **Manage Freenom Domain** tab. |
|||
3. Add an **A** record leave the **Name** field blank. |
|||
|
|||
This points your entire domain to the hub IP. |
|||
|
|||
4. Save your changes. |
|||
|
|||
5. Create a CNAME record. |
|||
|
|||
For example, you can use the prefix `www` with your domain name. When you are done, your |
|||
|
|||
6. Save your changes. |
|||
|
|||
At this point, your DNS management should look similar to the following except that with your domain rather than the `maryhub.ga` domain. |
|||
|
|||
 |
|||
|
|||
7. After your changes propagate, visit your new domain at the `hub_info` page. |
|||
|
|||
 |
|||
|
|||
If you receive another **Your connection is not private** dialogs, take the option to proceed to your domain. The *Not secure* message should no longer appear in the browser bar. If the message does appear, try waiting a few minutes for your recent changes to propagate across the net domain servers. Then, refresh the page. |
|||
|
|||
9. Check the SSL certificate for your hub. |
|||
|
|||
Each browser has its own check procedure, for example, Chrome: |
|||
|
|||
 |
|||
|
|||
At this point, you have the following. An EC2 instance running Gaia and a DNS |
|||
record pointing your domain to this instance. |
|||
|
|||
|
|||
## AWS hub tips and tricks |
|||
|
|||
Once your Gaia storage hub is up and running on AWS, you may occassionally need to troubleshoot. This section contains some useful information for interacting with your EC2 instance. |
|||
|
|||
### SSH into the Host |
|||
|
|||
To SSH to the EC2 host directly: |
|||
|
|||
```bash |
|||
ssh -t -i <your keyfile.pem> core@<public ip address> |
|||
``` |
|||
|
|||
### Displaying the docker services |
|||
|
|||
Your EC2 instance is running several `docker` services that support the Gaia hub. You can list these services using the `docker ps` command. |
|||
|
|||
{% raw %} |
|||
```bash |
|||
$ docker ps --format "table {{.ID}}\t{{.Command}}\t{{.Names}}" |
|||
CONTAINER ID COMMAND NAMES |
|||
b371234dc741 "/bin/sh -c 'trap ex…" docker_certbot_1 |
|||
597866815f42 "/bin/sh -c 'envsubs…" docker_nginx_1 |
|||
1d559bc51699 "npm run start" docker_admin_1 |
|||
46d410a1dce5 "npm run start" docker_reader_1 |
|||
f83fb8d044f5 "npm run start" docker_hub_1 |
|||
``` |
|||
{% endraw %} |
|||
|
|||
Each service plays a particular role in running your Gaia hub. |
|||
|
|||
<table class="uk-table uk-table-small uk-table-divider"> |
|||
<thead> |
|||
<tr> |
|||
<th>Service</th> |
|||
<th>Description</th> |
|||
</tr> |
|||
</thead> |
|||
<tbody> |
|||
<tr> |
|||
<td><code>certbot</code></td> |
|||
<td>Service running Let's Encrypt `certbot` client to support SSL. Certbot renews your certificates and reloads Nginx to pick up the changes.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>nginx</code></td> |
|||
<td>Runs an Nginx proxy in front of the <code>reader</code> side-car. This service does things like rate-limiting and SSL termination. Your |
|||
that nginx service relies on your hub's <code>readURL</code> to make requests. Changes to a hub's <code>readURL</code> must be reflected in the <code>nginx</code> service configuration.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>admin</code></td> |
|||
<td>A simple administrative service that allows you to administer the Gaia hub. Use REST calls with this service to get and set hub configuration values.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>reader</code></td> |
|||
<td>The Gaia read side-car services get file requests on URLs that start with |
|||
your Gaia hub's <code>readURL</code>. You can determine your Gaia hub's read URL by either |
|||
looking for the <code>readURL</code> key in your Gaia hub's config file. This value is or by looking for |
|||
the <code>read_url_prefix</code> field in the data returned by a <code>HUB_URL/hub_info</code> page on your |
|||
Gaia hub.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>hub</code></td> |
|||
<td>The Gaia hub service.</td> |
|||
</tr> |
|||
</tbody> |
|||
</table> |
|||
|
|||
|
|||
### Locations of key files |
|||
|
|||
<table> |
|||
<thead> |
|||
<tr> |
|||
<th>File or Directory</th> |
|||
<th>Description</th> |
|||
</tr> |
|||
</thead> |
|||
<tbody> |
|||
<tr> |
|||
<td><code>/etc/systemd/system</code></td> |
|||
<td>Contains services for managing your Gaia hub.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>/etc/environment</code></td> |
|||
<td>Contains the <code>DOMAIN</code> and <code>STAGING</code> variables you entered when creating your EC2 instance. |
|||
</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>reset-ssl-certs.service</code></td> |
|||
<td>Restarts all the Gaia hub services.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>/gaia/docker/admin-config</code></td> |
|||
<td>Configuration for the hub admin service.</td> |
|||
</tr> |
|||
</tbody> |
|||
</table> |
|||
|
|||
You can `cat` the various services to see what settings they are using. |
|||
|
|||
``` |
|||
$ cat /etc/systemd/system/reset-ssl-certs.service |
|||
[Unit] |
|||
Description=Reset Gaia to first boot |
|||
|
|||
[Service] |
|||
Type=oneshot |
|||
RemainAfterExit=no |
|||
EnvironmentFile=/gaia/docker/.env |
|||
EnvironmentFile=/etc/environment |
|||
ExecStart=/bin/bash -x /gaia/docker/nginx/certbot/reset-certs.sh |
|||
|
|||
[Install] |
|||
``` |
|||
|
|||
### Restart services and reload certificates |
|||
|
|||
This procedures requires you to interact from a workstation command line with your running EC2 instance. |
|||
|
|||
1. Open a terminal on your local workstation. |
|||
2. Confirm the hub DNS is set correctly with the following command: |
|||
|
|||
```bash |
|||
watch -n 2 -t -g -x host <domain> |
|||
``` |
|||
|
|||
Substitute your domain name for the `<domain>` variable. For example: |
|||
|
|||
```bash |
|||
watch -n 2 -t -g -x host maryhub.ga |
|||
maryhub.ml has address 34.219.71.143 |
|||
``` |
|||
|
|||
If the command returns the correct IP, the same as appears on your EC2 dashboard, stop the process with a` CTRL-C` on your keyboard. |
|||
|
|||
3. Change the permissions on your downloaded `.pem` file. |
|||
|
|||
For example, this |
|||
|
|||
``` |
|||
chmod 400 <location-of-pem> |
|||
``` |
|||
|
|||
4. SSH from your workstation and restart it. |
|||
|
|||
This process requires that you know the location of the `.pem` file you downloaded when you created the keypair. |
|||
|
|||
``` |
|||
ssh -t -i <your keyfile.pem> -A core@<public ip address> "sudo systemctl restart reset-ssl-certs.service" |
|||
``` |
|||
|
|||
For example: |
|||
|
|||
``` |
|||
$ ssh -t -i /Users/manthony/gaia.pem -A core@34.219.71.143 "sudo systemctl restart reset-ssl-certs.service" |
|||
Connection to 34.219.71.143 closed. |
|||
``` |
|||
@ -0,0 +1,34 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Authentication and Gaia |
|||
|
|||
Blockstack authentication is a bearer token-based authentication system. From an app user's perspective, login similar to third-party authentication techniques that they're familiar with. For an app developer, the flow is unlike the typical client-server flow of centralized sign-in services such as OAuth. With Blockstack the authentication flow happens entirely client-side. |
|||
|
|||
In this section, you get an overview of the authentication system and learn how Gaia fits into it. |
|||
|
|||
## Authentication and Gaia |
|||
|
|||
A decentralized application (DApp) and the Blockstack Browser communicate during |
|||
the authentication flow by passing back and forth two tokens. The requesting |
|||
application sends the Blockstack Browser an `authRequest` token. Once a user |
|||
approves a sign-in, the Blockstack Browser responds to the application with an |
|||
`authResponse` token. These tokens are <a href="https://jwt.io/" target="\_blank">JSON Web Tokens</a>, and they are passed via |
|||
URL query strings. |
|||
|
|||
When a user chooses to "Sign in with Blockstack" on your DApp, the `redirectToSignIn()` method sends the user to the Blockstack Browser. The browser responds with an authentication token and an _app private key_. |
|||
|
|||
 |
|||
|
|||
The app private key is application-specific. It is generated from the user's identity address private key using the `appDomain` as input. This key is deterministic, meaning that for a given Blockstack ID and domain name, the same private key is generated each time. The app private key is securely shared with the app on each authentication and encrypted by the Blockstack Browser. The key serves three functions, it: |
|||
|
|||
* is used to create the credentials that give an app access to the Gaia hub storage bucket for that specific app |
|||
* is used in the end-to-end encryption of files stored for the app on the user's Gaia hub |
|||
* serves as a cryptographic secret that apps can use to perform other cryptographic functions |
|||
|
|||
When an application writes to a Gaia hub, the authentication token, key, and the data are passed to the Gaia hub. |
|||
|
|||
 |
|||
|
|||
The token ensures the DApp has the authorization to write to the hub on the user's behalf. |
|||
@ -0,0 +1,7 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# blockstack_cli reference |
|||
|
|||
{% include commandline.md %} |
|||
@ -0,0 +1,106 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Hub configuration parameters |
|||
|
|||
```json |
|||
{ |
|||
"$schema": "http://json-schema.org/draft-04/schema#", |
|||
"type": "object", |
|||
"properties": { |
|||
"servername": { "type": "string" }, |
|||
"port": { "type": "integer" }, |
|||
"driver": { "type": "string" }, |
|||
"bucket": { "type": "string" }, |
|||
"readURL": { "type": "string" }, |
|||
"requireCorrectHubUrl": { "type": "string" }, |
|||
"awsCredentials": { |
|||
"type": "object", |
|||
"properties": { |
|||
"endpoint": { "type": "string"}, |
|||
"accessKeyId": { "type": "string"}, |
|||
"secretAccessKey": { "type": "string"} |
|||
}, |
|||
"required": [ |
|||
"endpoint", |
|||
"accessKeyId", |
|||
"secretAccessKey" |
|||
] |
|||
}, |
|||
"proofsConfig": { |
|||
"type": "object", |
|||
"properties": { |
|||
"proofsRequired": { "type": "string"} |
|||
}, |
|||
"required": [ |
|||
"proofsRequired" |
|||
] |
|||
}, |
|||
"azCredentials": { |
|||
"type": "object", |
|||
"properties": { |
|||
"accountName": { "type": "string"}, |
|||
"accountKey": { "type": "string"} |
|||
}, |
|||
"required": [ |
|||
"accountName", |
|||
"accountKey" |
|||
] |
|||
}, |
|||
"gcCredentials": { |
|||
"type": "object", |
|||
"properties": { |
|||
"keyFilename": { "type": "string"} |
|||
}, |
|||
"required": [ |
|||
"keyFilename" |
|||
] |
|||
}, |
|||
"diskSettings": { |
|||
"type": "object", |
|||
"properties": { |
|||
"storageRootDirectory": { "type": "string"} |
|||
}, |
|||
"required": [ |
|||
"storageRootDirectory" |
|||
] |
|||
}, |
|||
"pageSize": { "type": "integer"}, |
|||
"argsTransport": { |
|||
"type": "object", |
|||
"properties": { |
|||
"level": { "type": "string"}, |
|||
"handleExceptions": { "type": "boolean" }, |
|||
"stringify": { "type": "boolean" }, |
|||
"timestamp": { "type": "boolean" }, |
|||
"colorize": { "type": "boolean" }, |
|||
"json":{ "type": "boolean" } |
|||
}, |
|||
"required": [ |
|||
"level", |
|||
"handleExceptions", |
|||
"stringify", |
|||
"timestamp", |
|||
"colorize", |
|||
"json" |
|||
] |
|||
} |
|||
}, |
|||
"required": [ |
|||
"servername", |
|||
"port", |
|||
"driver", |
|||
"bucket", |
|||
"readURL", |
|||
"requireCorrectHubUrl", |
|||
"awsCredentials", |
|||
"proofsConfig", |
|||
"azCredentials", |
|||
"gcCredentials", |
|||
"diskSettings", |
|||
"pageSize", |
|||
"argsTransport" |
|||
] |
|||
} |
|||
``` |
|||
@ -0,0 +1,503 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Configure a hub on DigitalOcean |
|||
{:.no_toc} |
|||
|
|||
This teaches you how to run a Gaia storage hub on DigitalOcean (DO). DigitalOcean is an affordable and convenient cloud computing provider. This example uses DigitalOcean Spaces for file storage. DigitalOcean Spaces is equivalent to AWS's S3 file storage solution. |
|||
|
|||
DigitalOcean provides you with a compute machines known as a **Droplets** and storage called a **Spaces**. You need both to run a Gaia hub. The Gaia hub setup you create here, requires get a Digital Droplet with Docker pre-installed and a 250 GB Space. Each run for $5/month or a total of $10/month. |
|||
|
|||
* TOC |
|||
{:toc} |
|||
|
|||
<div class="uk-card uk-card-default uk-card-body"> |
|||
<h5>Is this tutorial for you?</h5> |
|||
|
|||
<p>This documentation is appropriate for advanced power users who are familiar with command line tools, editing configuration files, and basic configuration of services of DNS or Nginx. </p> |
|||
|
|||
<p>If you are planning on running an <emphasis>open-membership hub</emphasis> or an <emphasis>application-specific hub</emphasis>, you'll should see <a href="hub-operation.html">the section on Hub Operation</a></p>. |
|||
|
|||
</div> |
|||
|
|||
## Prerequisites you need |
|||
|
|||
You use DigitalOcean choose and configure assets for running droplets and spaces. To enable this, you must be sure to complete a number of prerequisites. |
|||
|
|||
### Required prerequisites on DigitalOcean |
|||
|
|||
You must create an account on <a href="https://digitalocean.com" target="\_blank">DigitalOcean</a>. DigitalOcean requires you to supply a credit card to create an account. You are only charged for the services you use the Gaia hub as of this writing should cost $10 USD a month. |
|||
|
|||
|
|||
### Optional prerequisites for SSH users |
|||
|
|||
The easiest way to interact with your droplet is the DigitalOcean Console. Users who are comfortable using the secure shell (SSH) and private keys may prefer to open a local terminal on their home machine instead. To enable this, you should ensure you have the following prerequisites completed. |
|||
|
|||
* Locate an existing SSH key pair on your Mac or <a href="https://help.dreamhost.com/hc/en-us/articles/115001736671-Creating-a-new-Key-pair-in-Mac-OS-X-or-Linux" target="\_blank">create a new SSH key pair</a>. Your key should have a passphrase, do not use a key pair without one. |
|||
|
|||
* Add the <a href="https://www.digitalocean.com/docs/droplets/how-to/add-ssh-keys/to-account/" target="\_blank">SSH from your local machine to DigitalOcean</a>. |
|||
|
|||
* Create a <a href="https://www.digitalocean.com/docs/api/create-personal-access-token/" target="\_blank">personal access token in DigitalOcean</a>. |
|||
|
|||
* Install `doctl` which is the DigitalOcean command line tool. For information on installing these, see which is the DigitalOcean command line utility. Check out their [installation instructions](https://github.com/digitalocean/doctl/blob/master/README.md#installing-doctl) to see how to install it on your computer. |
|||
|
|||
|
|||
## Task 1: Create a DigitalOcean space |
|||
|
|||
In this task you create a **Space** which is where Gaia stores your files. |
|||
|
|||
1. Choose **Create > Spaces** from the dashboard menu. |
|||
|
|||
 |
|||
|
|||
2. **Choose a datacenter region** section. |
|||
|
|||
{% include important.html content="Choose a region that is both |
|||
geographically close to you and that supports spaces. Currently, <strong>San |
|||
Francisco</strong>, <strong>New York</strong>, <strong>Amsterdam</strong>, |
|||
and <strong>Singapore</strong> support spaces." %} |
|||
|
|||
The geographical location of your server impacts latency for storing data. |
|||
You select a region close to you so that when you use Blockstack apps, |
|||
storing data is quicker. |
|||
|
|||
3. Scroll down to **Finalize and Create**. |
|||
|
|||
4. **Choose a unique name**. |
|||
|
|||
This name is used when reading files that you've stored through Gaia. You'll need to remember this name when you set up your Gaia server later on. |
|||
|
|||
5. Click **Create a Space**. |
|||
|
|||
After a moment, your Space is up and running. |
|||
|
|||
|
|||
## Task 2: Create a DigitalOcean droplet |
|||
|
|||
In this task, you add a droplet to your account. The droplet is a droplet is a cloud-based server you can use as a compute resource. This server is where you will run the Gaia Storage System service. The droplet you create will be an Ubuntu server with Docker pre-installed. |
|||
|
|||
1. Log into DigitalOcean. |
|||
2. Go to your DigitalOcean dashboard. |
|||
|
|||
 |
|||
|
|||
3. Click the **Create > Droplets** button in the top right. |
|||
|
|||
 |
|||
|
|||
4. Select the **One-click apps** tab. |
|||
|
|||
 |
|||
|
|||
5. Select the **Docker** app from the options presented. |
|||
|
|||
This will give you |
|||
|
|||
6. Scroll down to the **Choose a size** section and use the left arrow to display and select the **$5/mo** image. |
|||
|
|||
This size gives you plenty of memory and disk space to run a personal hub. |
|||
|
|||
7. Scroll down to the **Choose a datacenter region** section. |
|||
|
|||
{% include important.html content="Choose a region that is both geographically close to you and that supports spaces. Currently, <strong>San Francisco</strong>, <strong>New York</strong>, <strong>Amsterdam</strong>, and <strong>Singapore</strong> support spaces." %} |
|||
|
|||
The geographical location of your server impacts latency for storing data. You select a region close to you so that when you use Blockstack apps, storing data is quicker. |
|||
|
|||
8. If you are using SSH, scroll to the **Add your SSH key** section and choose an SSH key to use. Otherwise, |
|||
|
|||
9. Scroll down to the **Finalize and create** section. |
|||
|
|||
10. **Choose a hostname** for you Give your Droplet such as `moxiegirl-storage-hub`. |
|||
|
|||
11. Review your choices then click **Create** to start your droplet running. |
|||
|
|||
At this point, your new droplet should appear in the list of resources on your DigitalOcean dashboard. |
|||
|
|||
|
|||
## Task 3: Open a console on your Droplet |
|||
|
|||
A droplet console emulates the access you would have if you were sitting down with a keyboard and monitor attached to the actual server. In this section, you open a console on your droplet. |
|||
|
|||
{% include note.html content="If you are an SSH user and have completed the prerequisites, <strong>skip this section</strong>. Instead, use <a href='https://do.co/2S4HMk1' target='\_blank'>the DigitalOcean instructions for connecting with doctl</a>." %} |
|||
|
|||
1. From the DigitalOcean dashboard, select Droplets. |
|||
|
|||
You should see the droplet you just created. |
|||
|
|||
2. Click on the droplet name to open the control panel. |
|||
|
|||
 |
|||
|
|||
3. Choose **Access** from the control panel. |
|||
4. Select **Reset Root Password** to have DigitalOcean send you the root password. |
|||
|
|||
DigitalOcean sends a temporary password to the email attached to your account. It takes a couple of minutes to reset the root password on your droplet. |
|||
|
|||
5. Open your email and copy the password. |
|||
6. Switch back to the droplet control panel and choose **Launch Console**. |
|||
|
|||
A new window with the console appears. |
|||
|
|||
7. Enter `root` for the login. |
|||
8. Paste the password copied from your email. |
|||
|
|||
The system displays a message telling you to change the `root` password. |
|||
|
|||
 |
|||
|
|||
And prompts you for the current password. |
|||
|
|||
9. Past the password copied from your email again. |
|||
|
|||
The system prompts you for a new password and ask you to enter it again. |
|||
|
|||
10. Provide and confirm a new password. |
|||
|
|||
The system logins you in and gives you a welcome message. At the conclusion of the message, you are at the console prompt. |
|||
|
|||
``` |
|||
Welcome to DigitalOcean's One-Click Docker Droplet. |
|||
To keep this Droplet secure, the UFW firewall is enabled. |
|||
All ports are BLOCKED except 22 (SSH), 2375 (Docker) and 2376 (Docker). |
|||
* The Docker One-Click Quickstart guide is available at:
https ://do.co/docker1804#start |
|||
* You can SSH to this Droplet in a terminal as root: ssh root@138.68.28.100 |
|||
* Docker is installed and configured per Docker's recommendations:
https://docs.docker.com/install/linux/docker-ce/ubuntu/ |
|||
* Docker Compose is installed and configured per Docker's recommendations:
https://docs.docker.eom/compose/install/#install-compose |
|||
For help and more information, visit http://do.co/dockerl804 |
|||
|
|||
To delete this message of the day: rm -rf /etc/update-motd.d/99-one-click
root@moxiegirl:~# |
|||
``` |
|||
|
|||
<div class="uk-card uk-card-default uk-card-body"> |
|||
<h5>Useful tips for the console</h5> |
|||
<p>If you run into problems using the console, see <a href="https://www.digitalocean.com/docs/droplets/resources/console/" target="\_blank">the notes on this page in the DigitalOcean documentation</a>.</p> |
|||
<p>If you find the output from ls difficult to read, try enter the following to change the console colors from the command line: <code>LS_COLORS="di=1;31"</code> You can also edit your console <code>.bashrc</code>. file permanently, of course.</p> |
|||
</div> |
|||
|
|||
|
|||
## Task 4: Create a space key |
|||
|
|||
1. In the DigitalOcean dashboard, go to the **API** page. |
|||
2. Scroll to the **Spaces Access Keys** section. |
|||
3. Click **Generate New Key**. |
|||
|
|||
The system prompts you to give the key a name. |
|||
|
|||
4. Enter a name for the key. |
|||
|
|||
It is helpful to choose descriptive name like `gai-hub-key`. |
|||
|
|||
5. Press the check mark. |
|||
|
|||
The system creates your key and displays both the key and its secret. |
|||
|
|||
 |
|||
|
|||
6. Save your secrete in a secure password manager. |
|||
|
|||
You should never share your secret. |
|||
|
|||
7. Leave the page up with your key and secret and go to your open console. |
|||
|
|||
## Task 5: Get the Gaia code and configure your server |
|||
|
|||
You should have the console open as `root` on your Droplet. In this section, you get the Gaia code and configure the Gaia service. |
|||
|
|||
1. Copy the Gaia code into your droplet using the `git clone` command. |
|||
|
|||
``` |
|||
root@moxiegirl:~# git clone https://github.com/blockstack/gaia.git |
|||
``` |
|||
|
|||
Successful output from this command looks like the following. |
|||
|
|||
``` |
|||
Cloning into 'gaia'... |
|||
remote: Enumerating objects: 63, done. |
|||
remote: Counting objects: 100% (63/63), done. |
|||
remote: Compressing objects: 100% (46/46), done. |
|||
remote: Total 4206 (delta 27), reused 35 (delta 17), pack-reused 4143 |
|||
Receiving objects: 100% (4206/4206), 17.40 MiB | 9.89 MiB/s, done. |
|||
Resolving deltas: 100% (2700/2700), done. |
|||
root@moxiegirl:~# |
|||
``` |
|||
|
|||
This command creates a `gaia` subdirectory. |
|||
|
|||
2. Change to `hub` directory in the `gaia` code. |
|||
|
|||
``` |
|||
cd gaia/hub |
|||
``` |
|||
|
|||
3. Copy the configuration sample to a new `config.json` file. |
|||
|
|||
``` |
|||
cp config.sample.json config.json |
|||
``` |
|||
|
|||
4. Edit your new `config.json` file with `vi` or `vim`. |
|||
|
|||
``` |
|||
vi config.json |
|||
``` |
|||
You now need to edit this JSON file to have it store files on your DigitalOcean space. |
|||
|
|||
5. Set the `driver` to `aws`. |
|||
|
|||
The DigitalOcean space API exactly mimics the S3 API. Since Gaia doesn't have a DigitalOcean driver, you can just use the `aws` driver with some special configuration. |
|||
|
|||
6. Set the `bucket` to the name of the DigitalOcean space you just created. |
|||
|
|||
If your space is called `moxiegirl-hub-space`, the `bucket` value is `moxiegirl-hub-space`. |
|||
|
|||
6. Set the `readURL` to the URL of the DigitalOcean space you just created. |
|||
|
|||
If your space URL called `https://moxiegirl-hub-space.sfo2.digitaloceanspaces.com `, the `readURL` name is `https://moxiegirl-hub-space.sfo2.digitaloceanspaces.com`. |
|||
|
|||
|
|||
7. Add an `endpoint` value to the `awsCredentials` section. |
|||
|
|||
```json |
|||
"awsCredentials": { |
|||
"accessKeyId": "", |
|||
"secretAccessKey": "", |
|||
"endpoint": "" |
|||
}, |
|||
``` |
|||
8. Go back to your DigitalOcean dashboard open to your space key. |
|||
9. Copy the **Key** and paste it into the `accessKeyId` value in the `config.json` file. |
|||
10. Copy the **Secret** and paste it into the `secretAccessKey` value in the `config.json` file. |
|||
11. In the DigitalOcean dashboard, choose the Spaces page. |
|||
12. Copy the section of your space URL that follows the name. |
|||
|
|||
 |
|||
|
|||
In this example, you would copy the `sfo2.digitaloceanspaces.com` section. |
|||
|
|||
13. Paste the string you copied into the `endpoint` value. |
|||
|
|||
14. Change the `proofsRequired` value to the number `0` (zero). |
|||
|
|||
This will allow Blockstack user to write to your Gaia hub, without any social proofs required. You can change this later on, and do other things to lock-down this Gaia hub to just yourself, but that is outside the scope of this document. |
|||
|
|||
At this point, the json.config file should be completed and appear similar to the following &emdash; but with your values. |
|||
|
|||
```json |
|||
{ |
|||
"servername": "", |
|||
"port": 3000, |
|||
"driver": "aws", |
|||
"bucket": "moxiegirl-hub-space", |
|||
"readURL": "https://moxiegirl-hub-space.sfo2.digitaloceanspaces.com", |
|||
"awsCredentials": { |
|||
"accessKeyId": "W7GBQGIUWDWKA6KAGL56", |
|||
"secretAccessKey": "O6hBYRPCeRmE0d9lr3Frtc345QsWt3l+mrDgvrVT9oE", |
|||
"endpoint": "sfo2.digitaloceanspaces.com " |
|||
}, |
|||
|
|||
"proofsConfig": { |
|||
"proofsRequired" : 0 |
|||
}, |
|||
|
|||
"azCredentials": { |
|||
"accountName": "", |
|||
"accountKey": "" |
|||
}, |
|||
|
|||
"gcCredentials": { |
|||
"projectId": "" |
|||
}, |
|||
|
|||
"argsTransport": { |
|||
"level": "warn", |
|||
"handleExceptions": true, |
|||
"stringify": true, |
|||
"timestamp": true, |
|||
"colorize": false, |
|||
"json": true |
|||
} |
|||
} |
|||
``` |
|||
|
|||
15. Save and close the `vim` editor. |
|||
|
|||
The system returns you back to the prompt. |
|||
|
|||
## Task 6: Run the Gaia image with Docker |
|||
|
|||
While your console is still in the the `gaia/hub` folder, build the `gaia.hub` image. |
|||
|
|||
1. Enter the following `docker` command at the console command line. |
|||
|
|||
``` |
|||
docker build -t gaia.hub . |
|||
``` |
|||
This build users the `Dockerfile` already in the `gaia/hub` folder. The output of the command is similar to the following: |
|||
|
|||
``` |
|||
.... |
|||
|
|||
npm WARN gaia-hub@2.3.4 No license field. |
|||
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/fsevents): |
|||
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"}) |
|||
|
|||
added 877 packages from 540 contributors and audited 3671 packages in 38.122s |
|||
found 0 vulnerabilities |
|||
|
|||
Removing intermediate container b0aef024879f |
|||
---> 5fd126019708 |
|||
Step 5/5 : CMD ["npm", "run", "start"] |
|||
---> Running in ae459cc0865b |
|||
Removing intermediate container ae459cc0865b |
|||
---> b1ced6c39784 |
|||
Successfully built b1ced6c39784 |
|||
Successfully tagged gaia.hub:latest |
|||
``` |
|||
|
|||
2. Run your Gaia hub image. |
|||
|
|||
```bash |
|||
docker run --restart=always -v ~/gaia/hub/config.json:/src/hub/config.json -p 3000:3000 -e CONFIG_PATH=/src/hub/config.json gaia.hub |
|||
``` |
|||
|
|||
This runs your Gaia hub on port `3000`. If everything runs successfully, the last line outputted from this command should be: |
|||
|
|||
```bash |
|||
Successfully compiled 13 files with Babel. |
|||
{"level":"warn","message":"Listening on port 3000 in development mode","timestamp":"2019-01-23T16:35:05.216Z"} |
|||
``` |
|||
|
|||
3. If your command did run successfully, stop the service using the hotkey `ctrl-c`. |
|||
|
|||
4. Run the the image again with this new command. |
|||
|
|||
``` |
|||
docker run --restart=always -v ~/gaia/hub/config.json:/src/hub/config.json -p 3000:3000 -e CONFIG_PATH=/src/hub/config.json -d gaia.hub |
|||
``` |
|||
|
|||
This command includes `-d` option to `docker run`. This runs Docker in detached mode, so that it runs in the background. You can run `docker ps` to see your running docker images, and get the `id` of your Gaia server. |
|||
|
|||
|
|||
```bash |
|||
root@moxiegirl:~/gaia/hub# docker ps |
|||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES |
|||
aeca7eea4a86 gaia.hub "npm run start" 11 seconds ago Up 10 seconds 0.0.0.0:3000->3000/tcp musing_payne |
|||
``` |
|||
|
|||
At this point, your Gaia service is up and running. You can run `docker logs MY_CONTAINER_ID` with your running image's ID to see the logs of this server at any time. |
|||
|
|||
## Task 7: Set up an Nginx reverse proxy |
|||
|
|||
In this task, you set up a simple Nginx reverse proxy to serve your Docker container through a public URL. You do this from the droplet console command line. |
|||
|
|||
1. Install nginx into the droplet. |
|||
|
|||
``` |
|||
sudo apt-get install nginx |
|||
``` |
|||
|
|||
2. Enter `y` to confirm the installation. |
|||
3. Edit the nginx default configuration file. |
|||
|
|||
``` |
|||
vi /etc/nginx/sites-available/default |
|||
``` |
|||
|
|||
4. Inside the `location /` block (line 48), enter the following configuration: |
|||
|
|||
```nginx |
|||
location / { |
|||
proxy_pass http://localhost:3000; |
|||
proxy_http_version 1.1; |
|||
proxy_set_header Upgrade $http_upgrade; |
|||
proxy_set_header Connection 'upgrade'; |
|||
proxy_set_header Host $host; |
|||
proxy_cache_bypass $http_upgrade; |
|||
} |
|||
``` |
|||
|
|||
This simple configuration passes all requests through to your Gaia hub running at port `3000`. |
|||
|
|||
5. Save and close the file. |
|||
6. Run `nginx -t` to make sure you have no syntax errors. |
|||
|
|||
``` |
|||
root@moxiegirl:~/gaia/hub# nginx -t |
|||
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok |
|||
nginx: configuration file /etc/nginx/nginx.conf test is successful |
|||
``` |
|||
|
|||
7. Restart `nginx` with your new configuration. |
|||
|
|||
``` |
|||
systemctl restart nginx |
|||
``` |
|||
|
|||
8. Allow access to your Gaia hub by exposing port 80 to the public. |
|||
|
|||
``` |
|||
ufw allow 80 |
|||
``` |
|||
|
|||
## Task 8: Test your Gaia server |
|||
|
|||
Now, you are ready to test your Gaia server and make sure it is up and running. |
|||
|
|||
1. Click on **Droplets** in the sidebar. |
|||
2. Find your Droplet running Gaia. |
|||
|
|||
 |
|||
|
|||
3. Copy the IP address for it. |
|||
4. In your browser, visit the page `MY_DROPLET_IP/hub_info`. |
|||
|
|||
You should see a response from your Gaia hub! |
|||
|
|||
 |
|||
|
|||
The `read_url_prefix` should be combine from the bucket and endpoint create |
|||
in your `config.json` file, for example, |
|||
`https://moxiegirl-hub-space.s3.amazonaws.com/`. |
|||
|
|||
|
|||
## Task 9: Configure a domain name |
|||
|
|||
|
|||
At this point, you can point a domain to your Gaia hub. Although it's not required, it is highly recommended. If you use a domain, you can migrate your Droplet to a different server (or even provider such as Azure or AWS) at any time, and still access it through the domain URL. Simply point your domain at the IP address for your Droplet. Use an `A Record` DNS type. |
|||
|
|||
These instructions assume you have already created a free <a href="https://www.freenom.com" target="\_blank">domain through the freenom service</a>. To point this freenom domain to your Gaia hub, do the following: |
|||
|
|||
1. Log into your freenom account. |
|||
2. Choose the **Manage Freenom Domain** tab. |
|||
3. Add an **A** record leave the **Name** field blank. |
|||
|
|||
This record points your entire domain to the hub IP. |
|||
|
|||
4. Save your changes. |
|||
|
|||
5. Create a CNAME record. |
|||
|
|||
For example, you can use the prefix `www` with your domain name. When you are done, your |
|||
|
|||
6. Save your changes. |
|||
|
|||
At this point, your DNS management should look similar to the following except that with your domain rather than the `maryhub.ga` domain. |
|||
|
|||
 |
|||
|
|||
7. After your changes propagate, visit your new domain at the `hub_info` page. |
|||
|
|||
 |
|||
|
|||
|
|||
## Task 10: Set up SSL |
|||
|
|||
If you've configured a domain to point to your Gaia hub, then it's highly |
|||
recommended that you set up SSL to connect to your hub securely. DigitalOcean |
|||
provides <a |
|||
href="https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04" |
|||
target="\_blank">how to setup SSL </a>. Follow those instructions, to setup SSL. |
|||
When completed, you'll be able to visit `https://mygaiadomain.com/hub_info` |
|||
securely. |
|||
@ -0,0 +1,341 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Use the admin service |
|||
{:.no_toc} |
|||
|
|||
A Gaia service can run a simple administrative service co-located with your Gaia hub. This service allows you to administer the Gaia hub with the help of an API key. Gaia hubs installed using the Gaia Amazon Machine Image (AMI) have this service integrated automatically. |
|||
|
|||
In this section, you learn how to use the Gaia admin service with your Gaia hub. |
|||
|
|||
* TOC |
|||
{:toc} |
|||
|
|||
{% include note.html content=" |
|||
The examples in this section assume that Gaia and the admin service |
|||
were installed through the Configure a hub on Amazon EC2." %} |
|||
|
|||
|
|||
## Understand the configuration files |
|||
|
|||
The admin service relies on two configuration files, the hub's configuration and the configuration of the admin service itself. The hub's configuration is mounted `/tmp/hub-config/config.json` in the `docker_admin_1` container. Your EC2 instance has the admin service configuration in the `/gaia/docker/admin-config/config.json` file. |
|||
|
|||
The admin service needs to know the following: |
|||
|
|||
* where the Gaia hub config file is located |
|||
* which API key(s) to use when authenticating administrative requests |
|||
* which command(s) to run to restart the Gaia hub on a config change |
|||
|
|||
The following is the standard admin service config installed with your EC2 instance. |
|||
|
|||
|
|||
```json |
|||
{ |
|||
"argsTransport": { |
|||
"level": "debug", |
|||
"handleExceptions": true, |
|||
"timestamp": true, |
|||
"stringify": true, |
|||
"colorize": true, |
|||
"json": true |
|||
}, |
|||
"port": 8009, |
|||
"apiKeys": [ "hello" ], |
|||
"gaiaSettings": { |
|||
"configPath": "/tmp/hub-config/config.json" |
|||
}, |
|||
"reloadSettings": { |
|||
"command": "/bin/sh", |
|||
"argv": [ |
|||
"-c", |
|||
"docker restart docker_hub_1 &" |
|||
], |
|||
"env": {}, |
|||
"setuid": 1000, |
|||
"setgid": 1000 |
|||
} |
|||
} |
|||
``` |
|||
|
|||
The `port` is the port where Gaia is running. The `apiKeys` field is key used for making calls to the hub. The `gaiaSettings` |
|||
|
|||
|
|||
The `argsTransport` section configures the hub logging. The service uses the `winston` logging service. Refer to their documentation for full details on the [logging configuration options](https://github.com/winstonjs/winston). |
|||
|
|||
<table class="uk-table uk-table-small uk-table-divider"> |
|||
<thead> |
|||
<tr> |
|||
<th>Field</th> |
|||
<th>Description</th> |
|||
</tr> |
|||
</thead> |
|||
<tbody> |
|||
<tr> |
|||
<td><code>level</code></td> |
|||
<td>Lowest level this transport will log. (default: <code>info</code>)</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>handleException</code></td> |
|||
<td>Set to true to have this transport handle exceptions. (default: <code>false</code>)</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>timestamp</code></td> |
|||
<td>The timestamp when the message was received.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>stringify</code></td> |
|||
<td>Converts the output to a JSON string.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>colorize</code></td> |
|||
<td>Colorizes the standard logging level</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>json</code></td> |
|||
<td>Log format.</td> |
|||
</tr> |
|||
</tbody> |
|||
</table> |
|||
|
|||
The `reloadSettings` configure the command that is used to reload your Gaia hub. |
|||
|
|||
<table class="uk-table uk-table-small uk-table-divider"> |
|||
<thead> |
|||
<tr> |
|||
<th>Field</th> |
|||
<th>Description</th> |
|||
</tr> |
|||
</thead> |
|||
<tbody> |
|||
<tr> |
|||
<td><code>command</code></td> |
|||
<td>A command which reloads the Gaia hub service.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>argv</code></td> |
|||
<td>An array containing the command arguments.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>env</code></td> |
|||
<td>This is a key/value list of any environment variables |
|||
that need to be set for the command to run. This is optional.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>setuid</code></td> |
|||
<td>This is the UID under which the command will be run. This is optional.</td> |
|||
</tr> |
|||
<tr> |
|||
<td><code>setgid</code></td> |
|||
<td>This is the GID under which the command will run. This is optional.</td> |
|||
</tr> |
|||
</tbody> |
|||
</table> |
|||
|
|||
|
|||
## Using the admin service APIs |
|||
|
|||
You use the admin service APIs to manage the hub. Administrating a hub requires |
|||
that you make calls from a terminal on your hub service server. To execute admin |
|||
functions on a Gaia hub created with AWS, you `ssh` into your instance as follows: |
|||
|
|||
```bash |
|||
ssh -t -i <your keyfile.pem> -A core@<public ip address> |
|||
``` |
|||
|
|||
You must also set the `API_KEY` in your environment: |
|||
|
|||
```bash |
|||
export API_KEY="hello" |
|||
``` |
|||
|
|||
You may find it useful to install a JSON processor such as `jq` to process the |
|||
output of the admin commands. |
|||
|
|||
|
|||
### Restart the Gaia Hub (`POST /v1/admin/reload`) |
|||
|
|||
The admin service will make changes to the Gaia hub's config file, but the |
|||
changes will only take effect when the Gaia hub is reloaded. You can do this |
|||
as follows: |
|||
|
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ curl -H "Authorization: bearer $API_KEY" -X POST http://localhost:8009/v1/admin/reload |
|||
{"result":"OK"} |
|||
``` |
|||
|
|||
When you `POST` to this endpoint, the admin service runs the command described |
|||
in the `reloadSettings` section of the config file. It attempts to spawn a |
|||
subprocess from the given `reloadSettings.command` binary, and pass it the |
|||
arguments given in `reloadSettings.argv`. Note that the subprocess will *NOT* |
|||
be run in a shell. |
|||
|
|||
|
|||
#### Errors |
|||
{:.no_toc} |
|||
If you do not supply a valid API key, this method fails with HTTP 403. |
|||
|
|||
This endpoint returns HTTP 500 if the reload command fails. If this |
|||
happens, you will get back the command's exit code and possibly the signal that |
|||
killed it. |
|||
|
|||
### Get the hub configuration (`GET /v1/admin/config`) |
|||
|
|||
This endpoint is used to read and write a Gaia hub's non-driver-related |
|||
settings. These include the port it listens on, and its proof-checking |
|||
settings. |
|||
|
|||
To read the Gaia hub settings, you would run the following: |
|||
|
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ curl -H "Authorization: bearer $API_KEY" http://localhost:8009/v1/admin/config {"config":{"port":4000,"proofsConfig":{"proofsRequired":0}}} |
|||
``` |
|||
|
|||
### Set the hub configuration (`POST /v1/admin/config`) |
|||
|
|||
To set Gaia hub settings, you simply `POST` the changed JSON fields to this |
|||
endpoint. |
|||
|
|||
|
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ curl -H "Authorization: bearer $API_KEY" -H 'Content-Type: application/json' -X POST --data-raw '{"port": 3001}' http://localhost:8009/v1/admin/config |
|||
{"message":"Config updated -- you should reload your Gaia hub now."} |
|||
``` |
|||
|
|||
If the settings were successfully applied, the method returns a message to reload your Gaia hub. You can set multiple drivers' settings with a single call. For example, you can set: |
|||
|
|||
* The driver to use (`driver`) |
|||
* The Gaia's read URL endpoint (`readURL`) |
|||
* The number of items to return when listing files (`pageSize`) |
|||
* The driver-specific settings |
|||
|
|||
The data accepted on `POST` must contain a valid Hub configuration, for example: |
|||
|
|||
``` |
|||
const GAIA_CONFIG_SCHEMA = { |
|||
type: "object", |
|||
properties: { |
|||
validHubUrls: { |
|||
type: "array", |
|||
items: { type: "string", pattern: "^http://.+|https://.+$" }, |
|||
}, |
|||
requireCorrectHubUrl: { type: "boolean" }, |
|||
serverName: { type: "string", pattern: ".+" }, |
|||
port: { type: "integer", minimum: 1024, maximum: 65534 }, |
|||
proofsConfig: { type: "integer", minimum: 0 }, |
|||
whitelist: { |
|||
type: "array", |
|||
items: { |
|||
type: "string", |
|||
pattern: "^[123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz]+$" |
|||
} |
|||
}, |
|||
driver: { type: "string", pattern: ".+" }, |
|||
readURL: { type: "string", pattern: "^http://.+$|https://.+$" }, |
|||
pageSize: { type: "integer", minimum: 1 }, |
|||
bucket: { type: "string", pattern: ".+" }, |
|||
cacheControl: { type: "string", pattern: ".+" }, |
|||
azCredentials: { |
|||
accountName: { type: "string", pattern: ".+" }, |
|||
accountKey: { type: "string", pattern: ".+" }, |
|||
}, |
|||
diskSettings: { |
|||
storageRootDirectory: { type: "string" } |
|||
}, |
|||
gcCredentials: { |
|||
email: { type: "string" }, |
|||
projectId: { type: "string" }, |
|||
keyFilename: { type: "string" }, |
|||
credentials: { |
|||
type: "object", |
|||
properties: { |
|||
client_email: { type: "string" }, |
|||
private_key: { type: "string" } |
|||
} |
|||
}, |
|||
}, |
|||
awsCredentials: { |
|||
assessKeyId: { type: "string" }, |
|||
secretAccessKey: { type: "string" }, |
|||
sessionToken: { type: "string" } |
|||
} |
|||
} |
|||
} |
|||
``` |
|||
|
|||
The same fields are returned on `GET` within a `config` object. |
|||
|
|||
#### Errors |
|||
{:.no_toc} |
|||
If you do not supply a valid API key, both the `GET` and `POST` method fail with HTTP 403. |
|||
|
|||
Only relevant Gaia hub config fields are set. If you `POST` invalid settings |
|||
values, you get an HTTP 400 error. |
|||
|
|||
|
|||
## Example: Read and write driver settings |
|||
|
|||
Use the `/v1/admin/config` endpoint to read and write storage driver settings. To get the current driver settings, you would run: |
|||
|
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ curl -H "Authorization: bearer $API_KEY" http://localhost:8009/v1/admin/config |
|||
{"config":{"driver":"disk","readURL":"http://localhost:4001/","pageSize":20,"diskSettings":{"storageRootDirectory":"/tmp/gaia-disk"}}} |
|||
``` |
|||
|
|||
To update the driver settings, you would run: |
|||
|
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ export AWS_ACCESS_KEY="<hidden>" |
|||
$ export AWS_SECRET_KEY="<hidden>" |
|||
$ curl -H "Authorization: bearer $API_KEY" -H 'Content-Type: application/json' -X POST --data-raw "{\"driver\": \"aws\", \"awsCredentials\": {\"accessKeyId\": \"$AWS_ACCESS_KEY\", \"secretAccessKey\": \"$AWS_SECRET_KEY\"}}" http://localhost:8009/v1/admin/config |
|||
{"message":"Config updated -- you should reload your Gaia hub now."} |
|||
``` |
|||
|
|||
## Example: Read and write the whitelist |
|||
|
|||
This endpoint lets you read and write the `whitelist` section of a Gaia hub, to control who can write to it and list its files. |
|||
|
|||
To get the current whitelist, you would run the following: |
|||
|
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ curl -H "Authorization: bearer $API_KEY" http://localhost:8009/v1/admin/config |
|||
{"config":{"whitelist":["15hUKXg1URbQsmaEHKFV2vP9kCeCsT8gUu"]}} |
|||
``` |
|||
|
|||
To set the whitelist, you must set the *entire* whitelist. To set the list, pass a command such as the following: |
|||
|
|||
{% raw %} |
|||
```bash |
|||
$ export API_KEY="hello" |
|||
$ curl -H "Authorization: bearer $API_KEY" -H 'Content-Type: application/json' -X POST --data-raw '{"whitelist": ["1KDcaHsYJqD7pwHtpDn6sujCVQCY2e1ktw", "15hUKXg1URbQsmaEHKFV2vP9kCeCsT8gUu"]}' http://localhost:8009/v1/admin/config |
|||
{"message":"Config updated -- you should reload your Gaia hub now."} |
|||
``` |
|||
{% endraw %} |
|||
|
|||
## View logs for the hub or admin service |
|||
|
|||
The logs for each Gaia service are maintained by their respective Docker containers. To view the log for a particular service, use the `docker logs` command. For example, to get the logs for the hub: |
|||
|
|||
``` |
|||
$ docker logs docker_hub_1 |
|||
|
|||
> gaia-hub@2.3.4 start /src/hub |
|||
> npm run build && node lib/index.js |
|||
|
|||
|
|||
> gaia-hub@2.3.4 build /src/hub |
|||
> npm run lint && babel src -d lib && chmod +x lib/index.js |
|||
|
|||
|
|||
> gaia-hub@2.3.4 lint /src/hub |
|||
> eslint src |
|||
|
|||
Successfully compiled 13 files with Babel. |
|||
{"level":"warn","message":"Listening on port 3000 in development mode","timestamp":"2019-02-14T04:00:06.071Z"} |
|||
``` |
|||
@ -1,5 +0,0 @@ |
|||
--- |
|||
title: Storage with Gaia |
|||
layout: externalurl |
|||
redirect_url: https://github.com/blockstack/gaia |
|||
--- |
|||
@ -0,0 +1,276 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Hello hub choice tutorial |
|||
{:.no_toc} |
|||
|
|||
In this tutorial, you build on the <a href="{{ site.baseurl }}/browser/hello-blockstack.html" target="\_blank">Hello, Blockstack Tutorial</a>. You'll modify the authentication code so that it prompts users who have not yet created a Blockstack identity, to choose a hub URL. |
|||
|
|||
* TOC |
|||
{:toc} |
|||
|
|||
{% include note.html content="This tutorial was written on macOS High Sierra 10.13.4. If you use a Windows or Linux system, you can still follow along. However, you will need to \"translate\" appropriately for your operating system. Additionally, this tutorial assumes you are accessing the Blockstack Browser web application via Chrome. The application you build will also work with a local installation and/or with browsers other than Chrome. " %} |
|||
|
|||
## About this tutorial and the prerequisites you need |
|||
|
|||
This tutorial assumes you already set up your environment and tooling as specified in the <a href="{{ site.baseurl }}/browser/hello-blockstack.html" target="\_blank">Hello, Blockstack Tutorial</a>. You should also review that tutorial for basic information about |
|||
|
|||
## Task 1: Generate an initial Blockstack application |
|||
|
|||
In this section, you build an initial React.js application called `hello-hub-choice`. |
|||
|
|||
1. Create the `hello-hub-choice` directory. |
|||
|
|||
```bash |
|||
mkdir hello-hub-choice |
|||
``` |
|||
|
|||
2. Change into your new directory. |
|||
|
|||
```bash |
|||
cd hello-hub-choice |
|||
``` |
|||
|
|||
3. Use Yeoman and the Blockstack application generator to create your initial `hello-hub-choice` application. |
|||
|
|||
```bash |
|||
yo blockstack |
|||
``` |
|||
|
|||
You should see several interactive prompts. |
|||
|
|||
|
|||
{% raw %} |
|||
```bash |
|||
$ yo blockstack |
|||
|
|||
... |
|||
|
|||
? Are you ready to build a Blockstack app in React? (Y/n) |
|||
``` |
|||
{% endraw %} |
|||
|
|||
4. Respond to the prompts to populate the initial app. |
|||
|
|||
After the process completes successfully, you see a prompt similar to the following: |
|||
|
|||
{% raw %} |
|||
```bash |
|||
... |
|||
create public/icon-192x192.png |
|||
create public/index.html |
|||
create public/robots.txt |
|||
create public/manifest.json |
|||
|
|||
|
|||
Im all done. Running npm install for you to install the required dependencies. If this fails, try running the command yourself. |
|||
``` |
|||
{% endraw %} |
|||
|
|||
5. Verify that you have version `18.3.0` of blockstack.js or higher. |
|||
|
|||
``` |
|||
$ npm ls blockstack |
|||
hello-blockstack@0.0.0 /Users/manthony/sampleapps/hello-blockstack |
|||
└── blockstack@18.3.0 |
|||
``` |
|||
|
|||
If you don't have the correct version, install it. |
|||
|
|||
``` |
|||
npm install blockstack@18.3.0 |
|||
``` |
|||
|
|||
|
|||
Depending on your environment you may have some problems with the `npm` packages. Go ahead and fix these before continuing to the next section. |
|||
|
|||
## Task 2. Start the server and view the application |
|||
|
|||
When you start the server, it will create a Node.js server, start it locally, |
|||
and open your browser `http://localhost:5000`. From the root of your new application directory: |
|||
|
|||
1. Start the application server. |
|||
|
|||
```bash |
|||
npm start |
|||
``` |
|||
|
|||
2. Choose **Allow**. |
|||
|
|||
3. Open your browser to `http://localhost:8080`. |
|||
|
|||
You should see a simple application: |
|||
|
|||
 |
|||
|
|||
4. Leave your new application running and move onto the next section. |
|||
|
|||
## Task 3: Enable hub selection |
|||
|
|||
By default, the app generator assumes you want to use the default flow `redirectToSignIn()` method. In this section, you replace that method and use the `makeAuthRequest()` method instead. The `makeAuthRequest()` method takes the following parameters: |
|||
|
|||
<dl class="uk-description-list"> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>transitPrivateKey(String = generateAndStoreTransitKey())</code> |
|||
</dt> |
|||
<dd>A HEX encoded transit private key.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>redirectURI(String = `${window.location.origin}/`)</code> |
|||
</dt> |
|||
<dd>Location to redirect the user to after sign in approval.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>manifestURI(String = `${window.location.origin}/manifest.json`)</code> |
|||
</dt> |
|||
<dd> |
|||
Location of this app's manifest file. |
|||
</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>scopes (Array = DEFAULT_SCOPE)</code> |
|||
</dt> |
|||
<dd>The permissions this app is requesting.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>appDomain(String = window.location.origin)</code> |
|||
</dt> |
|||
<dd>The origin of this app.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>expiresAt(Number = nextHour().getTime())</code> |
|||
</dt> |
|||
<dd>The time at which this request is no longer valid.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>extraParams(Object = {})</code> |
|||
</dt> |
|||
<dd>Any extra parameters to pass to the authenticator. Use this to pass options that aren't part of the Blockstack authentication specification, but might be supported by special authenticators.</dd> |
|||
</dl> |
|||
|
|||
To replace the default login, do the following: |
|||
|
|||
1. Using your favorite editor, open the `public/app.js` file. |
|||
2. Locate the `redirectToSignIn()` method at line 4. |
|||
3. Replace `redirectToSignIn()` method with the `blockstack.redirectToSignInWithAuthRequest(authRequest)` method. |
|||
|
|||
The `authRequest` is the authentication request generated by `makeAuthRequest()` method. |
|||
|
|||
4. Immediately above the method you just added and below the `event.preventDefault()` method, construct a String `const` for the `authRequest`: |
|||
|
|||
``` |
|||
const authRequest = blockstack.makeAuthRequest( |
|||
blockstack.generateAndStoreTransitKey(), |
|||
'http://localhost:8080/', |
|||
'http://localhost:8080/manifest.json', |
|||
['store_write', 'publish_data'], |
|||
'http://localhost:8080/', |
|||
blockstack.nextHour().getTime(), { |
|||
solicitGaiaHubUrl: true |
|||
} // new options param |
|||
); |
|||
``` |
|||
|
|||
{% include note.html content="If your app is running a different port than <code>8080</code>, enter that port instead. " %} |
|||
|
|||
The extra `solicitGaiaHubUrl` parameter of `true` will cause the Blockstack Browser to prompt new identity creators for a storage hub URL. |
|||
|
|||
5. Save and close the `public/app.js` file. |
|||
6. Make sure your app rebuilds cleanly. |
|||
|
|||
|
|||
## Task 4: Try the new authentication flow |
|||
|
|||
Try your new authentication code. |
|||
|
|||
1. Refresh the client at `http://localhost:8080/`. |
|||
2. Click *Sign in with Blockstack*. |
|||
|
|||
The Blockstack Browser prompts you to sign in. I you are _not already authenticated_ with the browser, you should see the following: |
|||
|
|||
 |
|||
|
|||
If you are already authenticated with an existing ID, you can choose that ID |
|||
or **Deny**. Choosing an existing ID, signs you into the Hello Blockstack |
|||
application, without offering a hub choice. Instead, choose **Deny**. This |
|||
ends the authentication process. Then, reset the Blockstack Browser. |
|||
|
|||
3. Choose **Create new ID**. |
|||
|
|||
The system prompts you to enter a username. |
|||
|
|||
4. Enter a name and press **Check Availability**. |
|||
5. Press **Continue** after you find an available name you like. |
|||
6. Enter a password and chose **Register ID**. |
|||
|
|||
The system prompts you for a provider. |
|||
|
|||
 |
|||
|
|||
The default is to user Blockstacks' Gaia hub. |
|||
|
|||
7. Choose **Use a different provider**. |
|||
|
|||
The system prompts you for a URL. |
|||
|
|||
 |
|||
|
|||
8. Enter a Gaia hub URL. |
|||
|
|||
If you have one of your own enter it here. You can also just enter the Blockstack URL (`https://hub.blockstack.org`). |
|||
|
|||
9. Press **Continue**. |
|||
|
|||
The system takes a moment to check the hub is responsive and establish it in your new identity profile. Then, the system prompts you for an email. |
|||
|
|||
10. Enter an email and press **Next**. |
|||
|
|||
The system creates your user and prompts you to continue to your application. |
|||
|
|||
11. Choose **Go to Hello, Blockstack**. |
|||
|
|||
The system prompts you to allow Hello Blockstack access. |
|||
|
|||
12. Grant the access to the DApp. |
|||
|
|||
You user should be authenticated. |
|||
|
|||
## How to recommend a Gaia hub URL for new users |
|||
|
|||
If you want to create specific sign-up flows for your DApp, you can pass a preset Gaia storage hub URL also. For example, you might |
|||
do this if you have a corporate client whose employees would all like to use |
|||
your application with a company-run Gaia hub. |
|||
|
|||
To suggest a Gaia hub URL, provide an additional `recommendedGaiaHubUrl` value |
|||
alongside the `solicitGaiaHubUrl`, for example: |
|||
|
|||
```javascript |
|||
import { |
|||
makeAuthRequest, |
|||
redirectToSignInWithAuthRequest |
|||
} from 'blockstack'; |
|||
|
|||
const authRequest = makeAuthRequest(undefined, undefined, undefined, undefined, undefined, undefined, { |
|||
solicitGaiaHubUrl: true, |
|||
recommendedGaiaHubUrl: 'https://mygaiahub.com' |
|||
}); |
|||
|
|||
const authRequest = makeAuthRequest( |
|||
generateAndStoreTransitKey(), |
|||
'http://localhost:8080/', |
|||
'http://localhost:8080/manifest.json', |
|||
['store_write', 'publish_data'], |
|||
'http://localhost:8080/', |
|||
nextHour().getTime(), { |
|||
solicitGaiaHubUrl: true, //new options param |
|||
recommendedGaiaHubUrl: 'https://mygaiahub.com' // new options param |
|||
} |
|||
); |
|||
|
|||
redirectToSignInWithAuthRequest(authRequest); |
|||
``` |
|||
|
|||
Passing these parameters changes the storage hub URL prompt to the following: |
|||
|
|||
 |
|||
|
|||
|
|||
## Related information |
|||
{:.no_toc} |
|||
|
|||
[`makeAuthRequest()`](https://blockstack.github.io/blockstack.js/#makeauthrequest) method |
|||
@ -0,0 +1,111 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Hubs and user choice |
|||
|
|||
Blockstack operates a default Gaia storage hub at `https://hub.blockstack.org`. |
|||
Individuals or organizations may also run their own hubs, either as a for-profit |
|||
service or for other reasons. |
|||
|
|||
## Background |
|||
|
|||
Each user with an identity in the Blockstack Network has a Gaia hub |
|||
configured on their profile. New users that create identities with the |
|||
Blockstack Browser automatically are given storage on this default hub. For a |
|||
user to have a storage hub other than this default, the user must create an |
|||
identity through an application that enables storage hub selection. |
|||
|
|||
{% include note.html content="Users with existing identities cannot yet migrate |
|||
their data from one hub to another." %} |
|||
|
|||
## Enable the Gaia hub URL option for new users |
|||
|
|||
A major principle of Blockstack applications is that DApps do not store or |
|||
replicated user data. Through the Gaia feature, Blockstack seeks to empower |
|||
users to choose where to store their data and how they manage it. |
|||
|
|||
As a step towards fulfilling this principle, DApp developers can configure their |
|||
application to prompt new users to provide a Gaia hub URL. An application with |
|||
this feature enabled presents users creating an identity for the first time |
|||
with this prompt: |
|||
|
|||
 |
|||
|
|||
Users can choose to **Use Blockstack's official provider** or **Use a different provider**. Choosing the non-default option prompts the user with the following: |
|||
|
|||
 |
|||
|
|||
Users can enter the HTTPS address of a hub hosted by an entity other than |
|||
Blockstack. Once the user submits the above form, they continue through the |
|||
standard identity creation. As they use DApps on the Blockstack Network, all |
|||
their data is stored in this storage hub and not the default Blockstack hub. |
|||
|
|||
## Enabling hub selection in your DApp |
|||
|
|||
To enable this feature, you must ensure your DApp is using the latest version of the blockstack.js library. Instead of the default flow `redirectToSignIn()` method, you must use the `makeAuthRequest()` method. This method takes the following parameters: |
|||
|
|||
<dl class="uk-description-list"> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>transitPrivateKey(String = generateAndStoreTransitKey())</code> |
|||
</dt> |
|||
<dd>A HEX encoded transit private key.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>redirectURI(String = `${window.location.origin}/`)</code> |
|||
</dt> |
|||
<dd>Location to redirect the user to after sign in approval.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>manifestURI(String = `${window.location.origin}/manifest.json`)</code> |
|||
</dt> |
|||
<dd> |
|||
Location of this app's manifest file. |
|||
</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>scopes (Array = DEFAULT_SCOPE)</code> |
|||
</dt> |
|||
<dd>The permissions this app is requesting.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>appDomain(String = window.location.origin)</code> |
|||
</dt> |
|||
<dd>The origin of this app.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>expiresAt(Number = nextHour().getTime())</code> |
|||
</dt> |
|||
<dd>The time at which this request is no longer valid.</dd> |
|||
<dt class="uk-text-lowercase"> |
|||
<code>extraParams(Object = {})</code> |
|||
</dt> |
|||
<dd>Any extra parameters to pass to the authenticator. Use this to pass options that aren't part of the Blockstack authentication specification, but might be supported by special authenticators.</dd> |
|||
</dl> |
|||
|
|||
To ensure your DApps identity creation flow includes the Gaia URL, modify the `makeAuthRequest()` method to apply the `solicitGaiaHubUrl` parameter with value `true` when executing the method: |
|||
|
|||
```javascript |
|||
import { |
|||
makeAuthRequest, |
|||
redirectToSignInWithAuthRequest |
|||
} from 'blockstack'; |
|||
|
|||
const authRequest = makeAuthRequest( |
|||
generateAndStoreTransitKey(), |
|||
'http://localhost:8080/', |
|||
'http://localhost:8080/manifest.json', |
|||
['store_write', 'publish_data'], |
|||
'http://localhost:8080/', |
|||
nextHour().getTime(), { |
|||
solicitGaiaHubUrl: true |
|||
} // new options param |
|||
); |
|||
redirectToSignInWithAuthRequest(authRequest); |
|||
``` |
|||
|
|||
You can also pass a suggested Gaia storage hub URL also. For example, you might |
|||
do this if you have a corporate client whose employees would all like to use |
|||
your application with a company-run Gaia hub. To do this, you provide an |
|||
additional `recommendedGaiaHubUrl` value alongside the `solicitGaiaHubUrl` |
|||
|
|||
|
|||
## Related information |
|||
{:.no_toc} |
|||
|
|||
[Hello, Hub Choice Tutorial](hello-hub-choice.html) for a tutorial on using the [`makeAuthRequest()`](https://blockstack.github.io/blockstack.js/#makeauthrequest) method. |
|||
@ -0,0 +1,102 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Understand hub operation |
|||
{:.no_toc} |
|||
|
|||
This page describes the considerations hub operators must take into account when creating and operating a Gaia storage hub. |
|||
|
|||
* TOC |
|||
{:toc} |
|||
|
|||
|
|||
## Configuration files |
|||
|
|||
You should store a JSON configuration file either in the top-level directory of |
|||
the hub server. Alternatively, you can specify a file location using the |
|||
`CONFIG_PATH` environment variable. The following is an example configuration file for Amazon S3: |
|||
|
|||
```json |
|||
{ |
|||
"servername": "localhost", |
|||
"port": 4000, |
|||
"driver": "aws", |
|||
"readURL": "https://YOUR_BUCKET_NAME.s3.amazonaws.com/", |
|||
"pageSize": 20, |
|||
"bucket": "YOUR_BUCKET_NAME", |
|||
"awsCredentials": { |
|||
"accessKeyID": "YOUR_ACCESS_KEY", |
|||
"secretAccessKey": "YOUR_SECRET_KEY" |
|||
}, |
|||
"argsTransport": { |
|||
"level": "debug", |
|||
"handleExceptions": true, |
|||
"stringify": true, |
|||
"timestamp": true, |
|||
"colorize": false, |
|||
"json": true |
|||
} |
|||
} |
|||
``` |
|||
|
|||
You can specify the logging level, the backend driver, the credentials |
|||
for that backend driver, and the `readURL` of the hub. Typically, this is the URL for the compute resource on the cloud computing provider — where the hub service is running. |
|||
|
|||
### Require the correct hub URL |
|||
|
|||
If you enable on the `requireCorrectHubUrl` option in your `config.json` |
|||
file, your Gaia hub will require that authentication requests |
|||
correctly include the `hubURL` they are trying to connect with. Use this option to prevent a malicious Gaia hub from using an authentication |
|||
token for itself on other Gaia hubs. |
|||
|
|||
By default, the Gaia hub will validate that the supplied URL matches |
|||
`https://${config.servername}`, but if there are multiple valid URLs |
|||
for clients to reach the hub at, you can include a list in your `config.json`: |
|||
|
|||
```javascript |
|||
{ |
|||
.... |
|||
servername: "normalserver.com" |
|||
validHubUrls: [ "https://specialserver.com/", |
|||
"https://legacyurl.info" ] |
|||
.... |
|||
} |
|||
``` |
|||
|
|||
### The readURL parameter |
|||
|
|||
By default, hub drivers return read URLs that point directly at the written content. For example, an S3 driver would return the URL directly to the S3 file. If you configure a CDN or domain to point at that same bucket, you can use the `readURL` parameter to tell the hub that files can be read from a given URL. For example, the `hub.blockstack.org` Gaia Hub is configured to return a read URL that looks like `https://gaia.blockstack.org/hub/`. |
|||
|
|||
Unset the `readURL` parameter if you do not intend to deploy any caching. |
|||
|
|||
### The proofsRequired parameter (Deprecated) |
|||
|
|||
Past users could configure this setting as a crude spam-control mechanism. |
|||
However, for the smoothest operation of your Gaia hub, set the |
|||
`proofsConfig.proofsRequired` value to `0`. |
|||
|
|||
|
|||
## Open or private hubs |
|||
|
|||
You can configure an open-membership storage hub or a private storage hub. An open-membership hub, as it sounds, allows any user to use the hub service. A private hub limits the use of the service. In this section, you learn about configuring each type. |
|||
|
|||
### Open-membership hub |
|||
|
|||
An open-membership storage hub permits writes for _any_ address top-level |
|||
directory. Every request is validated such that write requests must provide |
|||
valid authentication tokens for that address. Operating in this mode is |
|||
recommended for service and identity providers who wish to support many |
|||
different users. |
|||
|
|||
### Private-user hub |
|||
|
|||
A private-user hub receives requests for a single user. Requests are controlled |
|||
via _whitelisting_ the addresses allowed to write files. Recall that each application uses a different app- and user-specific address. It follows, to |
|||
support application storage, your configuration must add to the whitelist each application you wish to use. |
|||
|
|||
Alternatively, the user's client can use the authentication scheme and generate |
|||
an association token for each app. The user should whitelist her address, and |
|||
use her associated private key to sign each app's association token. This |
|||
removes the need to whitelist each application, but with the caveat that the |
|||
user needs to take care that her association tokens do not get misused. |
|||
|
After Width: | Height: | Size: 137 KiB |
|
After Width: | Height: | Size: 27 KiB |
|
After Width: | Height: | Size: 82 KiB |
|
After Width: | Height: | Size: 64 KiB |
|
After Width: | Height: | Size: 32 KiB |
|
After Width: | Height: | Size: 102 KiB |
|
After Width: | Height: | Size: 126 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 51 KiB |
|
After Width: | Height: | Size: 107 KiB |
|
After Width: | Height: | Size: 46 KiB |
|
After Width: | Height: | Size: 74 KiB |
|
After Width: | Height: | Size: 61 KiB |
|
After Width: | Height: | Size: 35 KiB |
|
After Width: | Height: | Size: 14 KiB |
|
After Width: | Height: | Size: 65 KiB |
|
After Width: | Height: | Size: 62 KiB |
|
After Width: | Height: | Size: 46 KiB |
|
After Width: | Height: | Size: 60 KiB |
|
After Width: | Height: | Size: 74 KiB |
|
After Width: | Height: | Size: 122 KiB |
|
After Width: | Height: | Size: 212 KiB |
|
After Width: | Height: | Size: 147 KiB |
|
After Width: | Height: | Size: 132 KiB |
|
After Width: | Height: | Size: 19 KiB |
|
After Width: | Height: | Size: 12 KiB |
|
After Width: | Height: | Size: 48 KiB |
|
After Width: | Height: | Size: 25 KiB |
|
After Width: | Height: | Size: 138 KiB |
|
After Width: | Height: | Size: 103 KiB |
|
After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 108 KiB |
|
After Width: | Height: | Size: 146 KiB |
|
After Width: | Height: | Size: 27 KiB |
|
After Width: | Height: | Size: 30 KiB |
|
After Width: | Height: | Size: 83 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 102 KiB |
|
After Width: | Height: | Size: 147 KiB |
|
After Width: | Height: | Size: 54 KiB |
@ -0,0 +1,209 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# A decentralized storage architecture |
|||
|
|||
The Blockstack Network stores application data using a storage system called |
|||
Gaia. Transactional metadata is stored on the Blockstack blockchain and user |
|||
application data is stored in Gaia storage. Storing data off of the blockchain |
|||
ensures that Blockstack applications can provide users with high performance and |
|||
high availability for data reads and writes without introducing central trust |
|||
parties. |
|||
|
|||
* TOC |
|||
{:toc} |
|||
|
|||
## Understand Gaia in the Blockstack architecture |
|||
|
|||
The following diagram depicts the Blockstack architecture and Gaia's place in it: |
|||
|
|||
{% include architecture.md %} |
|||
|
|||
## User control or how is Gaia decentralized? |
|||
|
|||
A Gaia hub runs as a service which writes to data storage. The hub service |
|||
writes to data storage by requiring a valid authentication token from a requestor. Typically, the hub service runs on a compute resource and the storage itself on separate, dedicated storage resource. Typically, both resources belong to the same cloud computing provider. |
|||
|
|||
 |
|||
|
|||
Gaia's approach to decentralization focuses on user control of data and its |
|||
storage. If a user can choose which Gaia hub provider to use, then that choice |
|||
is all the decentralization required to enable user-controlled applications. |
|||
|
|||
The control of user data lies in the way that user data is accessed. |
|||
When an application fetches a file `data.txt` for a given user `alice.id`, the |
|||
lookup will follow these steps: |
|||
|
|||
1. Fetch the `zonefile` for `alice.id`. |
|||
2. Read her profile URL from her `zonefile`. |
|||
3. Fetch Alice's profile. |
|||
4. _Verify_ that the profile is signed by `alice.id`'s key |
|||
5. Read the `gaiaHubUrl` (e.g. `https://gaia.alice.org/`) out of the profile |
|||
6. Fetch the file from `https://gaia.alice.org/data.txt`. |
|||
|
|||
Because `alice.id` has access to her zonefile, she can change where her profile |
|||
is stored. For example, she may do this if the current profile's service or |
|||
storage is compromised. To change where her profile is stored, she changes her |
|||
Gaia hub URL to another Gaia hub URL from another hub provider. If Alice has |
|||
sufficient compute and storage resources herself, she may run her own Gaia |
|||
Storage System and bypass a commercial Gaia hub provider all together. |
|||
|
|||
{% include note.html content="Users with existing identities cannot yet migrate |
|||
their data from one hub to another." %} |
|||
|
|||
Applications writing directly on behalf of Alice do not need to perform a |
|||
lookup. Instead, the Blockstack <a |
|||
href="http://blockstack.github.io/blockstack.js/index.html" |
|||
target="\_blank">authentication flow</a> provides Alice's chosen application |
|||
root URL to the application. This authentication flow _is also_ within Alice's |
|||
control because Alice's browser _must_ generate the authentication response. |
|||
|
|||
|
|||
## Understand data storage |
|||
|
|||
A Gaia hub stores the written data _exactly_ as given. It offers minimal |
|||
guarantees about the data. It does not ensure that data is validly formatted, |
|||
contains valid signatures, or is encrypted. Rather, the design philosophy is |
|||
that these concerns are client-side concerns. |
|||
|
|||
Client libraries (such as `blockstack.js`) are capable of providing these |
|||
guarantees. Blockstack used a liberal definition of the <a href="https://en.wikipedia.org/wiki/End-to-end_principle" target="\_blank">end-to-end principle</a> to |
|||
guide this design decision. |
|||
|
|||
|
|||
## Gaia versus other storage systems |
|||
|
|||
Here's how Gaia stacks up against other decentralized storage systems. Features |
|||
that are common to all storage systems are omitted for brevity. |
|||
|
|||
<table class="uk-table uk-table-striped"> |
|||
<thead> |
|||
<tr> |
|||
<th>Features</th> |
|||
<th>Gaia</th> |
|||
<th><a href="https://sia.tech/" target="\_blank">Sia</a></th> |
|||
<th><a href="https://storj.io/" target="\_blank">Storj</a></th> |
|||
<th><a href="https://ipfs.io/" target="\_blank">IPFS</a></th> |
|||
<th><a href="https://datproject.org/" target="\_blank">DAT</a></th> |
|||
<th><a href="https://www.scuttlebutt.nz/" target="\_blank">SSB</a></th> |
|||
</tr> |
|||
</thead> |
|||
<tr> |
|||
<td>User controls where data is hosted</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data can be viewed in a normal Web browser</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data is read/write</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data can be deleted</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data can be listed</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
</tr> |
|||
<tr> |
|||
<td>Deleted data space is reclaimed</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data lookups have predictable performance</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Writes permission can be delegated</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Listing permission can be delegated</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Supports multiple backends natively</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data is globally addressable</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Needs a cryptocurrency to work</td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td></td> |
|||
<td></td> |
|||
<td></td> |
|||
</tr> |
|||
<tr> |
|||
<td>Data is content-addressed</td> |
|||
<td></td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
<td>X</td> |
|||
</tr> |
|||
</table> |
|||
@ -0,0 +1,62 @@ |
|||
--- |
|||
layout: storage |
|||
permalink: /:collection/:path.html |
|||
--- |
|||
# Storage write and read |
|||
|
|||
Once a user authenticates and a DApp obtains authentication, the application interacts with Gaia through the blockstack.js library. There are two simple methods for working with data in Gaia hub: the `putFile()` and `getFile()` methods. This section goes into greater detail about the methods, how they interact with a hub, and how to use them. |
|||
|
|||
|
|||
## Write-to and Read-from URL Guarantees |
|||
|
|||
Gaia is built on a driver model that supports many storage services. So, with |
|||
very few lines of code, you can interact with providers on Amazon S3, Dropbox, |
|||
and so forth. The simple `getFile` and `putFile` interfaces are kept simple |
|||
because Blockstack assumes and wants to encourage a community of |
|||
open-source-data-management libraries. |
|||
|
|||
The performance and simplicity-oriented guarantee of the Gaia specification is |
|||
that when an application submits a write-to |
|||
`https://myhub.service.org/store/foo/bar` URL, the application is guaranteed to |
|||
be able to read from the `https://myreads.com/foo/bar` URL. Note that, while the |
|||
prefix in the write-to url (for example,`myhub.service.org/store`) and the read-from URL |
|||
(`https://myreads.com`) are different, the `foo/bar` suffixes are the same. |
|||
|
|||
Consistent, identical suffixes allow an application to know _exactly_ where a |
|||
written file can be read from, given the read prefix. The Gaia service defines a `hub_info` endpoint to obtain that read prefix: |
|||
|
|||
``` |
|||
GET /hub_info/ |
|||
``` |
|||
|
|||
The endpoint returns a JSON object with a `read_url_prefix`, for example, if my service returns: |
|||
|
|||
```javascript |
|||
{ ..., |
|||
"read_url_prefix": "https://myservice.org/read/" |
|||
} |
|||
``` |
|||
|
|||
The data be read with this `getFile()` and this address: |
|||
|
|||
``` |
|||
https://myservice.org/read/1DHvWDj834zPAkwMhpXdYbCYh4PomwQfzz/0/profile.json |
|||
``` |
|||
|
|||
The application is guaranteed that the profile is written with `putFile()` this request address: |
|||
|
|||
``` |
|||
https://myservice.org/store/1DHvWDj834zPAkwMhpXdYbCYh4PomwQfzz/0/profile.json |
|||
``` |
|||
|
|||
When you use the `putFile()` method it takes the user data and POSTs it to the user's Gaia storage hub. The data POSTs directly to the hub, the blockchain is not used and no data is stored there. The limit on file upload is currently 25mb. |
|||
|
|||
|
|||
## Address-based access-control |
|||
|
|||
Access control in a Gaia storage hub is performed on a per-address basis. |
|||
Writes to URLs `/store/<address>/<file>` are allowed only if the writer can |
|||
demonstrate that they control _that_ address. This is achieved via the |
|||
authentication token which is a message _signed_ by the private key associated |
|||
with that address. The message itself is a challenge text, returned via the |
|||
`/hub_info/` endpoint. |
|||
Loading…
Reference in new issue