Make documentation more clear, add recommendations
This change makes many improvements to the documentation aimed at higher clarity and robustness. Aside from rewording and reordering various parts, it makes sure the documentation is up to date with the reality and best (security and cleanness) practices.
The list of changes:
* Changes the title of "Installation" section to make it clear it's manual and from source
* Points at more convenient options right on top
* Adds "Build dependencies" subtitle
* Makes "Build" a subtitle as it belongs under "Manual installation from source" section
* Recommends using Debain native packaging, which is more secure than `rustup`
* Suggests installation and use of `cfg_me`
* Moves Docker-based installation higher to be better visible
* Informs that native, stable packages are not available yet.
* Adds information about the experimental repository. While it can't be recommended for production use yet, the hope is to attract more contributors and deliver stable, secure and user-friendly experience sooner.
* Makes it clear that the documented configuration is manual, so that users using automated systems won't attempt to configure it manually
* Makes "Bitcoind configuration", "Usage" and "Configuration files" into subtitles
* Mentions the no-prune requirement upfront
* Makes it clear that `txindex` is not necessary, but also allowed.
* Recommends using cookie file
* Instead of telling people to wait for IBD before running `electrs` it accurately describes the behavior.
* Moves the section about `electrs` configuration before usage to be more visible. It's also the ordering of steps a user should take.
* Changes title of "configuration files and environment variables" to "Electrs configuration" to be consistent with "Bitcoind configuration"
* Recommends configuration files - they are clearer and saved across executions
* Clarifies the behavior of overriding files/arguments
* Documents `--conf` and `--conf-dir` arguments
* Explains the difference between `cookie` and `cookie_file`
* Renames "Usage" to "Electrs usage" to be more obvious
* Points out extra configuration suggestions, so they won't get lost after "Electrum client" section
* Notifies users of Debian repository to skip messing with Electrum configuration.
* Makes it clear that the script is provided in the repository (was not immediately obvious to me, LOL; providing `.desktop` file would be nicer, though)
* Added missing title "Extra configuration suggestions"
* Suggests to use more clear path in Tor configuration
* Hints at `tor-hs-config-patch`
* Warns users using the experimental Debian repository to not mess with systemd files
and [latest Electrum wallet](https://electrum.org/#download) (3.3+).
@ -12,7 +16,9 @@ $ sudo apt install clang cmake # for building 'rust-rocksdb'
Note for Raspberry Pi 4 owners: the old versions of OS/toolchains produce broken binaries. Make sure to use latest OS! (see #226)
## Build
Optionally, you may install [`cfg_me`](https://github.com/Kixunil/cfg_me) tool for generating the manual page. The easiest way is to run `cargo install cfg_me`.
### Build
First build should take ~20 minutes:
```bash
@ -21,18 +27,69 @@ $ cd electrs
$ cargo build --release
```
If you installed `cfg_me` to generate man page, you can run `cfg_me man` to see it right away or `cfg_me -o electrs.1 man` to save it into a file (`electrs.1`).
## Docker-based installation from source
```bash
$ docker build -t electrs-app .
$ docker run --network host \
--volume $HOME/.bitcoin:/home/user/.bitcoin:ro \
--volume $PWD:/home/user \
--rm -i -t electrs-app \
electrs -vvvv --timestamp --db-dir /home/user/db
```
## Native OS packages
There are currently no official/stable binary pckages.
However, there's an [**experimental** repository for Debian 10](https://deb.ln-ask.me) (should work on recent Ubuntu, but not tested well-enough). The repository provides several significant advantages:
* Everything is completely automatic - after installing `electrs` via `apt`, it's running and will automatically run on reboot, restart after crash... It also connects to bitoind out-of-the-bof, no messing with config files or anything else. It just works.
* Prebuilt binaries save you a lot of time. The binary installation of all the components is under 3 minutes on common hardware. Building from source is much longer.
* The repository contains some seurity hardening out-of-the-box - separate users for services, use of [btc-rpc-proxy](https://github.com/Kixunil/btc-rpc-proxy), etc.
And two significant disadvantages:
* It's currently impossible to independently verify the built packages, so you have to trust the author of the repository. This will hopefully change in the future.
* The repository is considered experimental and not well tested yet. The author of the repository is also a contributor to `electrs` and appreciates [bug reports](https://github.com/Kixunil/cryptoanarchy-deb-repo-builder/issues), [test reports](https://github.com/Kixunil/cryptoanarchy-deb-repo-builder/issues/61), and other contributions.
## Manual configuration
## Bitcoind configuration
This applies only if you do **not** use some other automated systems such as Debian packages. If you use automated systems, refer to their documentation first!
### Bitcoind configuration
Pruning must be turned **off** for `electrs` to work. `txindex` is allowed but unnecessary for `electrs`. However, you might still need it if you run other services (e.g.`eclair`)
The highly recommended way of authenticating `electrs` is using cookie file. It's the most secure and robust method. Set `rpccookiefile` option of `bitcoind` to a file within an existing directory which it can access. You can skip it if you're running both daemons under the same user and with the default directories.
`electrs` will wait for `bitcoind` to sync, however, you will be unabe to use it until the syncing is done.
Example command for running `bitcoind` (assuming same user, default dirs):
Allow Bitcoin daemon to sync before starting Electrum server:
```bash
$ bitcoind -server=1 -txindex=0 -prune=0
```
### Electrs configuration
If you are using `-rpcuser=USER` and `-rpcpassword=PASSWORD` for authentication, please use `cookie="USER:PASSWORD"` option in one of the [config files](https://github.com/romanz/electrs/blob/master/doc/usage.md#configuration-files-and-environment-variables).
Otherwise, [`~/.bitcoin/.cookie`](https://github.com/bitcoin/bitcoin/blob/0212187fc624ea4a02fc99bc57ebd413499a9ee1/contrib/debian/examples/bitcoin.conf#L70-L72) will be read, allowing this server to use bitcoind JSONRPC interface.
Electrs can be configured using command line, environment variables and configuration files (or their combination). It is highly recommended to use configuration files for any non-trivial setups since it's esier to manage. If you're setting password manually instead of cookie files, configuration file is the only way to set it due to security reasons.
## Usage
### Configuration files and priorities
The config files must be in the Toml format. These config files are (from lowest priority to highest): `/etc/electrs/config.toml`, `~/.electrs/config.toml`, `./electrs.toml`.
The options in highest-priority config files override options set in lowest-priority config files. Environment variables override options in config files and finally arguments override everythig else. There are two special arguments `--conf` which reads the specified file and `--conf-dir`, which read all the files in the specified directory. The options in those files override **everything that was set previously, including arguments that were passed before these arguments**. In general, later arguments override preious ones. It is a good practice to use these special arguments at the beginning of the command line in order to avoid confusion.
For each command line argument an environment variable of the same name with `ELECTRS_` prefix, upper case letters and underscores instead of hypens exists (e.g. you can use `ELECTRS_ELECTRUM_RPC_ADDR` instead of `--electrum-rpc-addr`). Similarly, for each such argument an option in config file exists with underscores instead o hypens (e.g. `electrum_rpc_addr`). In addition, config files support `cookie` option to specify cookie - this is not available using command line or environment variables for security reasons (other applications could read it otherwise). Note that this is different from using `cookie_path`, which points to a file containing the cookie instead of being the cookie itself.
Finally, you need to use a number in config file if you want to increase verbosity (e.g. `verbose = 3` is equivalent to `-vvv`) and `true` value in case of flags (e.g. `timestamp = true`)
If you are using `-rpcuser=USER` and `-rpcpassword=PASSWORD` of `bitcoind` for authentication, please use `cookie="USER:PASSWORD"` option in one of the [config files](https://github.com/romanz/electrs/blob/master/doc/usage.md#configuration-files-and-priorities).
Otherwise, [`~/.bitcoin/.cookie`](https://github.com/bitcoin/bitcoin/blob/0212187fc624ea4a02fc99bc57ebd413499a9ee1/contrib/debian/examples/bitcoin.conf#L70-L72) will be used as the default cookie file, allowing this server to use bitcoind JSONRPC interface.
### Electrs usage
First index sync should take ~1.5 hours (on a dual core Intel CPU @ 3.3 GHz, 8 GB RAM, 1TB WD Blue HDD):
2018-08-17T19:58:28 - DEBUG - applying 14 new headers from height 537205
2018-08-17T19:58:29 - INFO - RPC server running on 127.0.0.1:50001
```
You can specify options via command-line parameters, environment variables or using config files. See the documentation below.
You can specify options via command-line parameters, environment variables or using config files. See the documentation above.
Note that the final DB size should be ~20% of the `blk*.dat` files, but it may increase to ~35% at the end of the inital sync (just before the [full compaction is invoked](https://github.com/facebook/rocksdb/wiki/Manual-Compaction)).
@ -75,20 +132,15 @@ $ du db/
38G db/mainnet/
```
## Configuration files and environment variables
See below for [extra configuration suggestions](https://github.com/romanz/electrs/blob/master/doc/usage.md#extra-configuration-suggestions) that you might want to consider.
The config files must be in the Toml format. These config files are (from lowest priority to highest): `/etc/electrs/config.toml`, `~/.electrs/config.toml`, `./electrs.toml`.
The options in highest-priority config files override options set in lowest-priority config files. Environment variables override options in config files and finally arguments override everythig else.
For each argument an environment variable of the same name with `ELECTRS_` prefix, upper case letters and underscores instead of hypens exists (e.g. you can use `ELECTRS_ELECTRUM_RPC_ADDR` instead of `--electrum-rpc-addr`). Similarly, for each argument an option in config file exists with underscores instead o hypens (e.g. `electrum_rpc_addr`). In addition, config files support `cookie` option to specify cookie - this is not available using command line or environment variables for security reasonns (other applications could read it otherwise).
## Electrum client
Finally, you need to use a number in config file if you want to increase verbosity (e.g. `verbose = 3` is equivalent to `-vvv`) and `true` value in case of flags (e.g. `timestamp = true`)
If you happen to use the Electrum client from [the **experimental** Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), it's pre-configured out-of-the-box already. Read below otherwie.
There's a prepared script for launching `electrum` in such way to connect only to the local `electrs` instance to protect your privacy.
## Electrum client
```bash
# Connect only to the local server, for better privacy
$ ./scripts/local-electrum.bash
+ ADDR=127.0.0.1
+ PORT=50001
@ -104,6 +156,7 @@ $ electrum setconfig server 127.0.0.1:50001:t
$ electrum # will connect only to the local server
If you use [the **experimental** Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), it is cleaner to install `tor-hs-patch-config` using `apt` and then placing the configuration into a file inside `/etc/tor/hidden-services.d`.
Restart the service:
```
$ sudo systemctl restart tor
@ -165,7 +220,7 @@ $ sudo systemctl restart tor
Note: your server's onion address is stored under:
@ -178,6 +233,8 @@ For more details, see http://docs.electrum.org/en/latest/tor.html.
### Sample Systemd Unit File
If you use [the **experimental** Debian repository](https://github.com/romanz/electrs/blob/master/doc/usage.md#cnative-os-packages), you should skip this section, as the appropriate systemd unit file is installed automatically.
You may wish to have systemd manage electrs so that it's "always on." Here is a sample unit file (which assumes that the bitcoind unit file is `bitcoind.service`):
Martin Habovštiak
5 years ago
GitHub