rootzoll
6 years ago
3 changed files with 117 additions and 198 deletions
@ -1,155 +0,0 @@ |
|||
# Background: Blockchain Download |
|||
|
|||
## Why do we need to download the blockchain and not syncing it? |
|||
|
|||
The RaspiBlitz is powered by the RaspberryPi. The processing power of this SingleBoardComputer is too low to make a fast sync of the blockchain from the bitcoin peer to peer network during setup process (validation). To sync and index the complete blockchain could take weeks or even longer. Thats why the RaspiBlitz needs to download a prepared blockchain from another source. |
|||
|
|||
## Is downloading the blockchain secure? |
|||
|
|||
The downloaded Blockchain is pre-indexed and pre-validated. That is secure enough because if the user gets a "manipulated" blockchain it would not work after setup. The beginning of the downloaded blockchain needs to fit the genesis block (in bitcoind software) and the end of the downloaded blockchain needs not match with the rest of the bitcoin network state - hashes of new block distrubuted within the peer-2-peer network need to match the downloaded blockchain head. So if you downloaded a manipulated blockchain it would simply just dont work in practice. |
|||
|
|||
There might be theoretical scenarios why it would be bad for the whole network if anybody is downloading a prepared blockchain and not syncing and self-validating every block, but with a lot of self-validating full nodes already out there, just putting some RaspiBlitz with a downloaded blockchain into the mix and runnig as a self-valifating full node from that point on, is practically just strengthening the the overall network. And dont forget, the most important part of running a full node is to validate the new blocks summited by the miners - to balance out there power and to keep the Bitcoin network running by our rules. |
|||
|
|||
If you have any link to a more detailed discussion of this topic, please add it here for people to do their own research. |
|||
|
|||
## Blockchain Data |
|||
|
|||
The RaspiBlitz needs the following files from a bitcoind (>=0.17.0) that is fully sync and has indexing switched on (has line txindex=1 in bitcoin.conf file) - all files from the directories: |
|||
``` |
|||
/blocks |
|||
/chainstate |
|||
/indexes |
|||
``` |
|||
Notice: If you had your node fully sync but not indexed, once you switch it on you need to start your client and wait for it to verify the chain and index before copying the folders above to your external HDD. This can take a couple of hours. |
|||
|
|||
Make sure not to add other files like wallet data or lock files to a prepared download. |
|||
|
|||
You might want to include the testnet data also - then add the testnet3 folder that just contains the same three folders from the testnet blockchain. |
|||
|
|||
On download all those files need to be placed finally in the /mnt/hdd/bitcoin folder of the RaspiBlitz. |
|||
|
|||
## Download Process |
|||
|
|||
At the moment the RaspiBlitz offers two technical ways to download the blockchain: |
|||
|
|||
### FTP Download (fallback) |
|||
|
|||
The easiest way is to put the blockchain data on a FTP server and let people download it. FTP is able to download complete directories - HTTP can just handle single file. FTP clients and server hosting is widly available. |
|||
|
|||
The downside that this is a central point solution and is hard to scale up (without spending a lot of money). But it is available as a backup, if other solutions do not work. |
|||
|
|||
### Torrent Download (default) |
|||
|
|||
The preferred way is to to download the blockchain data thru the peer2peer torrent network. This way the community can help to seed the torrents (at least while downloading). Supporters of the project can setup constant seeding. There is no single point of failure within the download structure. |
|||
|
|||
In the beginning we used just on torrent file - containing all the directories and data like mentioned above. But this had the downside, that everytime when we update the torrent the seeding is bad in the beginning and downloads are slow. Good seeding needs time to build up. |
|||
|
|||
Thats why there are two torrent files now: |
|||
|
|||
#### Base Torrent File |
|||
|
|||
Inspired by the website getbitcoinblockchain.com we use one of their base torrent files to have a basic set of blocks - that will not change for the future. This torrent contains most of the data (the big file) and we dont need to change the torrent for a long time. This way the torrent can get establish a wide spread seeding and the torrent network can take the heavy load. |
|||
|
|||
At the moment this is just the bitcoin blk and rev files up to the number: |
|||
- /blocks : 01390 |
|||
- /testnet3/blocks: 00152 |
|||
|
|||
For litecoin its blk and rev files up to the number: |
|||
- /blocks : 00124 |
|||
|
|||
#### Update Torrent File (Description) |
|||
|
|||
All the rest of the files get packaged into a second torrent file. This file will be updated much more often. The seeding is expected to be not that good and download may be slower, but thats OK because its a much smaller file. |
|||
|
|||
This way a good balance between good seeding and up-to-date blockchain can be reached. |
|||
|
|||
#### Update Torrent File (Creation) |
|||
|
|||
To create the Update Torrent file, follow the following step ... |
|||
|
|||
Have a almost 100% synced bitcoind MAINNET with txindex=1 on a RaspiBlitz |
|||
(remove all funds from this node - because blockchain get messed with) |
|||
|
|||
Stop bitcoind with: |
|||
``` |
|||
sudo systemctl stop bitcoind |
|||
``` |
|||
|
|||
Delete base torrent blk-files with: |
|||
``` |
|||
sudo rm /mnt/hdd/bitcoin/blocks/blk00*.dat |
|||
sudo rm /mnt/hdd/bitcoin/blocks/blk0{1000..1390}.dat |
|||
``` |
|||
|
|||
Delete base torrent rev-files with: |
|||
``` |
|||
sudo rm /mnt/hdd/bitcoin/blocks/rev00*.dat |
|||
sudo rm /mnt/hdd/bitcoin/blocks/rev0{1000..1390}.dat |
|||
``` |
|||
|
|||
Now change to your computer where you package the torrent files and transfere the three directories into your torrent base directory (should be your current working directory): |
|||
``` |
|||
scp -r bitcoin@[RaspiBlitzIP]:/mnt/hdd/bitcoin/blocks ./blocks |
|||
scp -r bitcoin@[RaspiBlitzIP]:/mnt/hdd/bitcoin/chainstate ./chainstate |
|||
scp -r bitcoin@[RaspiBlitzIP]:/mnt/hdd/bitcoin/indexes ./indexes |
|||
``` |
|||
|
|||
Also have a almost 100% synced bitcoind TESTNET with txindex=1 on a RaspiBlitz |
|||
|
|||
Stop bitcoind with: |
|||
``` |
|||
sudo systemctl stop bitcoind |
|||
``` |
|||
|
|||
Delete base torrent blk-files with: |
|||
``` |
|||
sudo rm /mnt/hdd/bitcoin/testnet3/blocks/blk000*.dat |
|||
sudo rm /mnt/hdd/bitcoin/testnet3/blocks/blk00{100..152}.dat |
|||
``` |
|||
|
|||
Delete base torrent rev-files with: |
|||
``` |
|||
sudo rm /mnt/hdd/bitcoin/testnet3/blocks/rev000*.dat |
|||
sudo rm /mnt/hdd/bitcoin/testnet3/blocks/rev00{100..152}.dat |
|||
``` |
|||
|
|||
Now change again to your computer where you package the torrent files and transfere the three directories into your torrent base directory (should be your current working directory): |
|||
``` |
|||
mkdir testnet3 |
|||
scp -r bitcoin@[RaspiBlitzIP]:/mnt/hdd/bitcoin/testnet3/blocks ./testnet3/blocks |
|||
scp -r bitcoin@[RaspiBlitzIP]:/mnt/hdd/bitcoin/testnet3/chainstate ./testnet3/chainstate |
|||
scp -r bitcoin@[RaspiBlitzIP]:/mnt/hdd/bitcoin/testnet3/indexes ./testnet3/indexes |
|||
``` |
|||
|
|||
(Re-)name the "torrent base directory" to the same name as the torrent UPDATE file itself later (without the .torrent ending). For the correct naming see the chapter "Torrent Files Naming Schema" below. |
|||
|
|||
Now open your torrent client (e.g. qTorrent for OSX) and create a new torrent-file with the freshly renamed "torrent base directory" as source directory. |
|||
|
|||
Add this list of trackers to your torrent and start seeding (keep a free/empty line between the three single trackers): |
|||
``` |
|||
udp://tracker.justseed.it:1337 |
|||
|
|||
udp://tracker.coppersurfer.tk:6969/announce |
|||
|
|||
udp://open.demonii.si:1337/announce |
|||
|
|||
udp://denis.stalker.upeer.me:6969/announce |
|||
``` |
|||
|
|||
Name the new torrent file |
|||
|
|||
To create the torrent file can take some time. Finally add the generated torrent file to the /home.admin/assets/ of the github and change the name of the updateTorrent varibale file in the script 50torrentHDD.bitcoin.sh |
|||
|
|||
#### Torrent Files Naming Schema |
|||
|
|||
The base torrent file should always have the following naming scheme: |
|||
|
|||
`raspiblitz-[CHAINNETWORK][BASEITERATIONNUMBER]-[YEAR]-[MONTH]-[DAY]-base.torrent` |
|||
|
|||
So for example the second version of the base torrent for litecoin created on 2018-10-31 would have this name: raspiblitz-litecoin2-2018-10-31-base.torrent |
|||
|
|||
The update torrentfile should always have the following naming schema: |
|||
|
|||
`raspiblitz-[CHAINNETWORK][BASEITERATIONNUMBER]-[YEAR]-[MONTH]-[DAY]-update.torrent` |
|||
|
|||
So for exmaple an update torrent created on 2018-12-24 for litecoin that is an update to the second base torrent version would have this name: raspiblitz-litecoin2-2018-12-24-update.torrent |
|||
@ -1,43 +0,0 @@ |
|||
# Background: RaspiBlitz Settings |
|||
|
|||
## Before Version 1.0 |
|||
|
|||
The RaspiBlitz started as small collection of shell scripts to setup a bitcoin+lightning node. At this time it was not needed to have settings file. The idea was that the scripts analyse the system state and make the changes required - automatically trying to catch up. That was OK as long RaspiBlitz was just a helper to setup your Lightning node and the rest was up to you. |
|||
|
|||
Over time users that are running a RaspiBlitz expected that it can handle more complex setup and customization. Also it should be easy to update the system (exchange sd card with a newer image) and should be able to have the same configuration afterwards - keeping its state. Thats why starting from version 1.0 there will be a raspiblitz config file stored on the HDD that stores stores the config state. |
|||
|
|||
## The Config File |
|||
|
|||
The RaspiBlitz config file is stored on the HDD root: |
|||
|
|||
`/mnt/hdd/raspiblitz.conf` |
|||
|
|||
Its simple structure is: one key-value pair per line. In the end its bash-script syntax to define varibales. The RaspiBlitz shell scripts can import this file with: |
|||
|
|||
`source /mnt/hdd/raspiblitz.conf` |
|||
|
|||
After this line all the config values are available and can be worked with. I prefer to call this line in scripts explicitly and not setting this values as environment variables, because when you read as a newbie such a script, you get an idea where the config file is stored. |
|||
|
|||
## The Config Values |
|||
|
|||
So see what config parameters are available check the comments in the following script: |
|||
|
|||
`/home/admin/_bootstrap.sh` |
|||
|
|||
## Adding new Config Values |
|||
|
|||
If you extend the RaspiBlitz scripts and you have the need to add a new config key-value add it to the `/home/admin/00enforceConfig.sh` script. There is a section for default values and setting them in the config file, if they dont exist there yet. Because this script runs on every startup, you can be sure that the default value is then available to your extended script - especially if people update their system. |
|||
|
|||
## Bootstrap Service: Enforcing the Config |
|||
|
|||
On every start of the RaspiBlitz take the config file and check if the system is running as stated in the config file and when needed make changes to the system. This is done by calling this script on startup with systemd: |
|||
|
|||
`/home/admin/_bootstrap.sh` |
|||
|
|||
So if you change the config by hand or you write a script that changes the config, then simply trigger a restart the RaspiBliz. |
|||
|
|||
Having this script checking the system on every startup, the user can easily update the SD card with a fresh image and the system will automatically establish the old state. |
|||
|
|||
## What to put into the config file and what not |
|||
|
|||
All values users put into setup or setting dialogs and that is not stored on the HDD (for example in the config files of lnd or bitcoin) is a hot cadidate to put into the raspi config file. Some values make sense to get stored as a duplicate (for performance or easy of access) - but dont get to wild. |
|||
Loading…
Reference in new issue