Browse Source

Merge pull request #631 from rootzoll/master

rebase master
v1.3
Christian Rotzoll 6 years ago
committed by GitHub
parent
commit
ab087f3c02
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
  1. 4
      README.md
  2. 16
      WORKSHOP.md
  3. 17
      home.admin/00infoLCD.sh
  4. 8
      home.admin/00raspiblitz.sh
  5. 2
      home.admin/30initHDD.sh
  6. 3
      home.admin/AAunlockLND.sh
  7. 323
      home.admin/XXcopyStation.sh
  8. 2
      home.admin/config.scripts/blitz.datadrive.sh

4
README.md

@ -455,6 +455,8 @@ The RTL Webinterface is a LND Control Dashboard you can run in your browser with
![RTL](pictures/RTL-dashboard.png)
Read an Intro-Tutorial to RTL: https://medium.com/@suheb.khan/how-to-ride-the-lightning-447af999dcd2
Feedback is welcome by the RTL programmer: https://github.com/ShahanaFarooqui/RTL
##### LND Auto-Unlock
@ -486,6 +488,8 @@ and [Zeus (iOS/Android)](https://github.com/ZeusLN/zeus) are available.
Please keep in mind that if you also want to connect to your smartphone also from the outside (when you are outside of your local network) with your RaspiBlitz you might need to open/forward ports on your router and should look into the DynamicDNS features to handle changeing IP of our Home-DSL.
This youtube video explains the "port forwarding" on your router in more detail: https://www.youtube.com/watch?v=KESo7hHXQtg
* [How do I shrink the QR code for connecting my Shango/Zap/Zeus mobile phone?](FAQ.md#how-do-i-shrink-the-qr-code-for-connecting-my-shangozap-mobile-phone)
#### EXPORT: Macaroons and TLS.cert

16
WORKSHOP.md

@ -135,7 +135,7 @@ Basically you follow the assemble instructions on the RaspiBlitz GuitHUb README.
## Prepare HDDs with Blockchain Data
This is the most time consuming part of the preparation. Try it once to get a feel for how much time you need to prepare one HDD.
This is the most time consuming part of the preparation. Try it once to get a feel for how much time you need to prepare one HDD. If you prepare more then one HDD check out the "Copystation" script below.
A prepared HDD is formatted in EXT4 and named "BLOCKCHAIN". In folder called `bitcoin` it contains a copy of the following data folders from a running Bitcoin core client (same version on RaspiBlitz).
@ -161,6 +161,20 @@ Once you have that "template" you can make a image from that and write that imag
## Prepare Blockchain Copy Station
In the RaspiBlitz Github repo and also on every RaspiBlitz (since v1.3) you can find the script:
`/home/admin/CCcopyStation.sh`
This can be used to prepare and keep multiple HDDs in snyc with blockchain data in preparation of a workshop. You can start it directly on a RaspiBlitz and turn it into "Copy Station Mode" with executing on the command line:
`sudo /home/admin/CCcopyStation.sh`
*Beware that it will not run as a Lightning Node during that time (LND is stopped). And to reset it back into normal mode you need to stop the script with `CTLR+c` and the reboot with `sudo shutdown -r now`.*
In "COpy Station Mode" the RaspiBlitz will just run the bitcoind (so it needs network connection), copy fresh blockchain data over to a template folder on the HDD called `/mnt/hdd/templateHDD` and from there syncs it to further HDDs that get connected to it.
If you run it in a setup lke on this photo with an extra powered USB hub, you can connect up to 10 HDDs at once to be synced with an almost up-to-date blockchain.
At the moment the "Blockchain Copy Station" is just a computer (laptop - not a RaspberryPi) having a image of a "template" HDD (see above) and you can attach (with a USB3.0 Hub) multiple fresh HHDs to it and start writing in the template image to that.
To update the "template" HDD for the next workshop use it for a fresh clean RaspiBllitz setup just days before, sync the blockchain to 100% and repeat the process above.

17
home.admin/00infoLCD.sh

@ -53,16 +53,12 @@ while :
fi
# get config info if already available
source ${infoFile}
configExists=$(ls ${configFile} 2>/dev/null | grep -c '.conf')
if [ ${configExists} -eq 1 ]; then
source ${configFile}
fi
# if setup not marked as done (=100) load boostrap info file
if [ "${setupStep}" != "100" ]; then
source ${infoFile}
fi
# if no information available from files - set default
if [ ${#setupStep} -eq 0 ]; then
setupStep=0
@ -87,8 +83,7 @@ while :
# when in presync - get more info on progress
elif [ "${state}" = "presync" ]; then
# get blockchain sync progress
blockchaininfo="$(sudo -u root ${network}-cli -datadir=/home/bitcoin/.${network} getblockchaininfo 2>/dev/null)"
blockchaininfo="$(sudo -u root bitcoin-cli --conf=/home/admin/assets/bitcoin.conf getblockchaininfo 2>/dev/null)"
message="starting"
if [ ${#blockchaininfo} -gt 0 ]; then
message="$(echo "${blockchaininfo}" | jq -r '.verificationprogress')"
@ -198,6 +193,14 @@ while :
continue
fi
if [ "${state}" = "copystation" ]; then
l1="COPY STATION MODE\n"
l2="${message}"
dialog --backtitle "RaspiBlitz ${codeVersion} ${localip}" --infobox "$l1$l2" 6 56
sleep 2
continue
fi
# if LND is syncing or scanning
lndSynced=$(sudo -u bitcoin /usr/local/bin/lncli --chain=${network} --network=${chain}net getinfo 2>/dev/null | jq -r '.synced_to_chain' | grep -c true)
if [ ${lndSynced} -eq 0 ]; then

8
home.admin/00raspiblitz.sh

@ -75,6 +75,14 @@ if [ "${state}" = "retorrent" ]; then
exit
fi
# singal that copstation is running
if [ "${state}" = "copystation" ]; then
echo "Copy Station is Runnning ..."
echo "reboot to return to normal"
sudo /home/admin/XXcopyStation.sh
exit
fi
# if pre-sync is running - stop it - before continue
if [ "${state}" = "presync" ]; then
# stopping the pre-sync

2
home.admin/30initHDD.sh

@ -56,6 +56,8 @@ if [ ${existsHDD} -eq 1 ]; then
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
echo "You HDD was detected with the size of ${isSize} bytes"
echo "For ${network} at least ${minSize} bytes is recommended"
echo "If you know the HDD is bigger then detected, please"
echo "change HDD to 1 partition on another computer first,"
echo "If you want to change to a bigger HDD:"
echo "* Unplug power of RaspiBlitz"
echo "* Make a fresh SD card again"

3
home.admin/AAunlockLND.sh

@ -30,5 +30,6 @@ while :
echo ""
echo "network(${network}) chain(${chain})"
sudo tail -n 1 /mnt/hdd/lnd/logs/${network}/${chain}net/lnd.log
echo "Wallet still locked - please try again or Cancel with CTRL+C"
echo "Wallet still locked - please try again or"
echo "Cancel with CTRL+C - back to setup with command: raspiblitz"
done

323
home.admin/XXcopyStation.sh

@ -0,0 +1,323 @@
#!/bin/bash
# command info
if [ "$1" = "-h" ] || [ "$1" = "-help" ]; then
echo "# Turns the RaspiBlitz into HDD CopyStation Mode"
echo "# lightning is deactivated during CopyStationMode"
echo "# reboot RaspiBlitz to set back to normal mode"
exit 1
fi
####### CONFIG #############
# where to find the BITCOIN data directory (no trailing /)
pathBitcoinBlockchain="/mnt/hdd/bitcoin"
# where to find the LITECOIN data directory (no trailing /)
pathLitecoinBlockchain="/mnt/hdd/litecoin"
# where to find the RaspiBlitz HDD template directory (no trailing /)
pathTemplateHDD="/mnt/hdd/templateHDD"
# 0 = ask before formatting/init new HDD
# 1 = auto-formatting every new HDD that needs init
nointeraction=1
# override values if XXcopyStation.conf files exists
# use when you run this outside RaspiBlitz
# - clean Ubuntu install
# - install bitcoind as systemd service
# - disable automount: https://askubuntu.com/questions/89244/how-to-disable-automount-in-nautiluss-preferences#102601
# - clone the github to get script (or download)
# - set your pathes bitcoin/template in conf file
source ./XXcopyStation.conf 2>/dev/null
# -- start script with parameter "-foreground"
####### SCRIPT #############
# check sudo
if [ "$EUID" -ne 0 ]; then
echo "Please run as root (with sudo)"
exit 1
fi
# make sure that its running in screen
# call with '-foreground' to prevent running in screen
if [ "$1" != "-foreground" ]; then
screenPID=$(screen -ls | grep "copystation" | cut -d "." -f1 | xargs)
if [ ${#screenPID} -eq 0 ]; then
# start copystation in sreen
echo "starting copystation screen session"
screen -S copystation -dm /home/admin/XXcopyStation.sh -foreground
screen -d -r
exit 0
else
echo "changing into running copystation screen session"
screen -d -r
exit 0
fi
fi
clear
echo "******************************"
echo "RASPIBLITZ COPYSTATION SCRIPT"
echo "******************************"
echo
echo "*** CHECKING CONFIG"
# check that path information is valid
if [ -d "$pathBitcoinBlockchain" ]; then
echo "OK found $pathBitcoinBlockchain"
else
echo "FAIL path of 'pathBitcoinBlockchain' does not exists: ${pathBitcoinBlockchain}"
exit 1
fi
# check that path information is valid
if [ -d "$pathTemplateHDD" ]; then
echo "OK found $pathTemplateHDD"
else
echo "Creating: ${pathTemplateHDD}"
mkdir ${pathTemplateHDD}
chmod 777 ${pathTemplateHDD}
fi
# make sure that lnd is stopped (if runnning)
systemctl stop lnd 2>/dev/null
systemctl stop background 2>/dev/null
if [ "${nointeraction}" == "1" ]; then
echo "setting RaspiBlitz LCD info"
sudo sed -i "s/^state=.*/state=copystation/g" /home/admin/raspiblitz.info 2>/dev/null
sudo sed -i "s/^message=.*/message='Disconnect target HDDs!'/g" /home/admin/raspiblitz.info 2>/dev/null
echo "Disconnect target HDDs! .. 30ses until continue."
sleep 30
else
echo
echo "*** INIT HDD SCAN"
echo "Please make sure that no HDDs that you want to sync later to are not connected now."
echo "PRESS ENTER when ready."
read key
fi
# finding system drives (the drives that should not be synced to)
systemDrives=$(lsblk -o NAME | grep "^sd")
echo "OK - the following drives detected as system drives:"
echo "$systemDrives"
echo
if [ "${nointeraction}" == "1" ]; then
sudo sed -i "s/^message=.*/message='Connect now target HDDs ..'/g" /home/admin/raspiblitz.info 2>/dev/null
sleep 5
fi
# BASIC IDEA:
# 1. get fresh data from bitcoind --> template data
# 2. detect HDDs
# 3. sync HDDs with template data
# repeat
echo
echo "*** RUNNING ***"
lastBlockchainUpdateTimestamp=1
while :
do
################################################
# 1. get fresh data from bitcoind for template data
# only execute every 30min
nowTimestamp=$(date +%s)
secondsDiff=$(echo "${nowTimestamp}-${lastBlockchainUpdateTimestamp}" | bc)
echo "seconds since last update from bitcoind: ${secondsDiff}"
echo
if [ ${secondsDiff} -gt 3000 ]; then
echo "******************************"
echo "Bitcoin Blockchain Update"
echo "******************************"
# stop blockchains
echo "Stopping Blockchain ..."
systemctl stop bitcoind 2>/dev/null
systemctl stop litecoind 2>/dev/null
sleep 10
# sync bitcoin
echo "Syncing Bitcoin ..."
sudo sed -i "s/^message=.*/message='Updating Template: Bitcoin'/g" /home/admin/raspiblitz.info 2>/dev/null
# make sure the bitcoin directory in template folder exists
if [ ! -d "$pathTemplateHDD/bitcoin" ]; then
echo "creating the bitcoin subfolder in the template folder"
mkdir ${pathTemplateHDD}/bitcoin
chmod 777 ${pathTemplateHDD}/bitcoin
fi
rsync -a --info=progress2 ${pathBitcoinBlockchain}/chainstate ${pathBitcoinBlockchain}/indexes ${pathBitcoinBlockchain}/blocks ${pathBitcoinBlockchain}/testnet3 ${pathTemplateHDD}/bitcoin
if [ -d "${pathLitecoinBlockchain}" ]; then
# sync litecoin
echo "Syncing Litecoin ..."
echo "creating the litecoin subfolder in the template folder"
mkdir ${pathTemplateHDD}/litecoin 2>/dev/null
chmod 777 ${pathTemplateHDD}/litecoin 2>/dev/null
sudo sed -i "s/^message=.*/message='Updating Template: Litecoin'/g" /home/admin/raspiblitz.info 2>/dev/null
rsync -a --info=progress2 ${pathLitecoinBlockchain}/chainstate ${pathLitecoinBlockchain}/indexes ${pathLitecoinBlockchain}/blocks ${pathTemplateHDD}/litecoin
fi
# restart bitcoind (to let further setup while syncing HDDs)
echo "Restarting Blockchain ..."
systemctl start bitcoind 2>/dev/null
systemctl start litecoind 2>/dev/null
# update timer
lastBlockchainUpdateTimestamp=$(date +%s)
fi
################################################
# 2. detect connected HDDs and loop thru them
sleep 4
echo "" > ./.syncinfo.tmp
lsblk -o NAME | grep "^sd" | while read -r detectedDrive ; do
isSystemDrive=$(echo "${systemDrives}" | grep -c "${detectedDrive}")
if [ ${isSystemDrive} -eq 0 ]; then
# check if drives 1st partition is named BLOCKCHAIN & in EXT4 format
isNamedBlockchain=$(lsblk -o NAME,FSTYPE,LABEL | grep "${detectedDrive}" | grep -c "BLOCKCHAIN")
isFormatExt4=$(lsblk -o NAME,FSTYPE,LABEL | grep "${detectedDrive}" | grep -c "ext4")
# init a fresh device
if [ ${isNamedBlockchain} -eq 0 ] || [ ${isFormatExt4} -eq 0 ]; then
echo "*** NEW EMPTY HDD FOUND ***"
echo "Device: ${detectedDrive}"
echo "isNamedBlockchain: ${isNamedBlockchain}"
echo "isFormatExt4:" ${isFormatExt4}
# check if size is OK
size=$(lsblk -o NAME,SIZE -b | grep "^${detectedDrive}" | awk '$1=$1' | cut -d " " -f 2)
echo "size: ${size}"
if [ ${size} -lt 250000000000 ]; then
read key
whiptail --title "FAIL" --msgbox "
THE DEVICE IS TOO SMALL <250GB
Please remove device and PRESS ENTER
" 9 46
else
# find biggest partition
biggestSize=0
lsblk -o NAME,SIZE -b | grep "${detectedDrive}" | while read -r partitionLine ; do
partition=$(echo "${partitionLine}" | cut -d ' ' -f 1 | tr -cd "[:alnum:]")
size=$(echo "${partitionLine}" | tr -cd "[0-9]")
if [ ${size} -gt ${biggestSize} ]; then
formatPartition="${partition}"
biggestSize=$size
fi
echo "${formatPartition}" > .formatPartition.tmp
done
formatPartition=$(cat .formatPartition.tmp)
rm .formatPartition.tmp
if [ ${#formatPartition} -eq 0 ]; then
whiptail --title "FAIL" --msgbox "
NO PARTITIONS FOUND ON THAT DEVICE
Format on external computer with FAT32 first.
Please remove device now.
" 10 46
else
# if config value "nointeraction=1" default to format
if [ "${nointeraction}" != "1" ]; then
whiptail --title "Format HDD" --yes-button "Format" --no-button "Cancel" --yesno "
Found new HDD. Do you want to FORMAT now?
Please temp lable device with: ${formatPartition}
" 10 54
choice=$?
else
choice=0
sudo sed -i "s/^message=.*/message='Formatting new HDD: ${formatPartition}'/g" /home/admin/raspiblitz.info 2>/dev/null
fi
# on cancel
if [ "${choice}" != "0" ]; then
whiptail --title "Format HDD" --msgbox "
OK NO FORMAT - Please remove decive now.
" 8 46
exit 1
fi
# format the HDD
echo "Starting Formatting of device ..."
sudo mkfs.ext4 /dev/${formatPartition} -F -L BLOCKCHAIN
fi
fi
fi # end init new HDD
################################################
# 3. sync HDD with template data
partition=$(lsblk -o NAME,FSTYPE,LABEL | grep "${detectedDrive}" | grep "BLOCKCHAIN" | cut -d ' ' -f 1 | tr -cd "[:alnum:]")
if [ ${#partition} -gt 0 ]; then
# temp mount device
echo "mounting: ${partition}"
mkdir /mnt/hdd2 2>/dev/null
sudo mount -t ext4 /dev/${partition} /mnt/hdd2
# rsync device
mountOK=$(lsblk -o NAME,MOUNTPOINT | grep "${detectedDrive}" | grep -c "/mnt/hdd2")
if [ ${mountOK} -eq 1 ]; then
if [ "${nointeraction}" == "1" ]; then
sudo sed -i "s/^message=.*/message='Syncing from Template: ${partition}'/g" /home/admin/raspiblitz.info 2>/dev/null
fi
rsync -a --info=progress2 ${pathTemplateHDD}/* /mnt/hdd2
chmod -r 777 /mnt/hdd2
rm -r /mnt/hdd2/lost+found 2>/dev/null
echo "${partition} " >> ./.syncinfo.tmp
else
echo "FAIL: was not able to mount --> ${partition}"
fi
# unmount device
sudo umount -l /mnt/hdd2
fi
fi
done
clear
echo "**** SYNC LOOP DONE ****"
synced=$(cat ./.syncinfo.tmp | tr '\r\n' ' ')
echo "HDDs ready synced: ${synced}"
echo "*************************"
echo "Its safe to disconnect/remove HDDs now."
echo "To stop copystation script: CTRL+c"
echo ""
sudo sed -i "s/^message=.*/message='Ready HDDs: ${synced}'/g" /home/admin/raspiblitz.info 2>/dev/null
sleep 25
clear
echo "starting new sync loop"
sleep 5
done

2
home.admin/config.scripts/blitz.datadrive.sh

@ -105,7 +105,7 @@ if [ "$1" = "1" ] || [ "$1" = "on" ]; then
# mount the BTRFS drive
echo "Mounting under /mnt/data ..."
sudo mkdir -p /mnt/data
sudo mount /dev/${dev1} /mnt/data
sudo mount barrier=1 /dev/${dev1} /mnt/data
echo "OK"
echo ""

Loading…
Cancel
Save