@hiveos
Have you or anybody ever tried moving to using pixz instead of using pxz for the parallel compression and parallel decompression of the boot archive (i.e. moving from hiveramfs.tar.xz to hiveramfs.tpxz)?
I tried dissecting through the scripts and I can't seem to find the part where the system knows to use tar to extract the hiveramfs.tar.xz file into tmpfs.
I've tried looking in /path/to/pxeserver/tftp and also in /path/to/pxeserver/hiveramfs and I wasn't able to find where it codifies the instruction and/or the command to unpack the hiveramfs.tar.xz.
If you can provide some guidance as to where I would find that in the startup script, where it would instruct the client to decompress and unpack the hiveramfs.tar.xz, that would be greatly appreciated.
Thank you.
*edit*
I've implemented pixz now for both parallel compression and the creation of the boot archive hiveramfs.tpxz and the decompression of the same.
It replaces the boot archive hiveramfs.tar.xz.
The PXE server host, if you are running an Ubuntu PXE boot server, will need to have pixz installed (which you can get by running sudo apt install -y pixz, so it's pretty easy to get and install.)
The primary motivation for this is on your mining rig, depending on the CPU that you have in it, but usually, at boot time, you will have excess CPU capacity, and therefore; if you can use a parallel decompression for the hiveramfs archive, then you can get your mining rig up and running that much quicker.
The side benefit that this has also produced is that in the management of the hiveramfs image on the PXE server, pixz worked out to be faster in the creation of the FS archive compared to pxz.
Tested on my PXE server which has a Celeron J3455 (4-core, 1.5 GHz base clock), it compressed the FS archive using pxz in 11 minutes 2 seconds whilst pixz was able to complete the same task (on a fresh install of the HiveOS PXE server) in 8 minutes 57 seconds. (Sidebar: For reference, previously, when using only xz (without the parallelisation), on my system, it would take somewhere between 40-41 minutes to create the FS archive.)
On my mining rig, which has a Core i5-6500T, it takes about 8.70 seconds to decompress hiveramfs.tpxz to hiveramfs.tar and then it takes about another 1.01 seconds to unpack the tarball file.
Unfortunately, I don't have the benchmarking data for how long it took my mining rig to decompress and unpack hiveramfs.tar.xz file.
Here are the steps to deploying pixz, and using that to replace pxz.
On the PXE server, install pixz:
sudo apt install -y pixz
Run the pxe-config.sh to specify your farm hash, server IPv4 address, etc. and also the change the name of the FS archive from hiveramfs.tar.xz to hiveramfs.tpxz.
DO NOT RUN HiveOS update/upgrade yet still!!!
When it asks if you want to upgrade HiveOS, type n for no.
For safety/security, make a backup copy of the initial hiveramfs.tar.xz file that can be found in /path/to/pxeserver/hiveramfs.
(For me, I just ran sudo cp hiveramfs.tar.xz hiveramfs.tar.xz.backup.)
You will need to manually create the initial hiveramfs.tpxz file that the system will act upon next when you run the hive-upgrade.sh script.
To do that, run the following:
/path/to/pxeserver$ sudo mkdir -p tmp/root
/path/to/pxeserver$ cd tmp/root
/path/to/pxeserver/tmp/root$ cp ../../hiveramfs/hiveramfs.tar.xz .
/path/to/pxeserver/tmp/root$ tar --lzma -xf hiveramfs.tar.xz
/path/to/pxeserver/tmp/root$ tar -I pixz -cf ../hiveramfs.tpxz .
/path/to/pxeserver/tmp/root$ cd ..
/path/to/pxeserver/tmp/root$ cp hiveramfs.tpxz ../../hiveramfs
/path/to/pxeserver/tmp/root$ cd ../../hiveramfs
/path/to/pxeserver/hiveramfs$ cp hiveramfs.tpxz hiveramfs.tpxz.backup
Now, edit the pxe-config.sh:
at about line 51, it should say something like:
#adde pxz (typo included)
copy lines 51-53 and paste it after line 53
(basically, add an i so that where it says pxz now says pixz instead)
edit the lines to read:
#adde pixz
dpkg -s pixz > /dev/null 2>&1
[[ $? -ne 0 ]] && need_install="$need_install pixz"
save, quit
Run pxe-config.sh again.
DO NOT RUN HiveOS update/upgrade yet still!!!
Now, your farm hash, IP address, etc. should all have been set previously. Again, when it asks you if you want to upgrade HiveOS, type n for no.
Now, we are going to make a bunch of updates to hive-upgrade.sh.
(For me, I still use vi, but you can use whatever text editor you want.)
/path/to/pxeserver$ sudo vi hive-upgrade.sh
at line 71, add pixz to the end of the line so that the new line 71 would read:
apt install -y pv pixz
I haven't been able to figure out how to decompress the hiveramfs.tpxz archive and unpack it in the same line.
(I also was unable to get pv working properly so that it would show the progress indicator, so if someone else who is smarter than I am can help figure that out, that would be greatly appreciated, but you can also remote into your PXE server again in another terminal window and run top to monitor your PXE server to make sure that it is working in the absence of said progress indicator.)
So the section starting at line 79 echo -e "> Extract Hive FS to tmp dir" now reads:
line80: #pv $FS | tar --lzma -xf -
line81: cp $FS .
line82: pixz -d $ARCH_NAME
line83: tar -xf hiveramfs.tar .
line84: rm hiveramfs.tar
Line84 is needed because otherwise, without it, when you go to create the archive, it will try to compress the old hiveramfs.tar in as well, and you don't need that.
Now fast forward to the section where it creates the archive (around line 121) where it says:
line121: echo -e "> Create FS archive"
line122: #tar -C root -I pxz -cpf - . | pv -s $arch_size | cat > $ARCH_NAME
line123: tar -C root -I pixz -cpf - . | pv -s $arch_size | cat > $ARCH_NAME
(in other words, copy that line, paste it, comment out the old line, and add an i to the new line.)
line125 is still the old line where it used the single threaded xz compression algorithm/tool, which should be already commented out for you.
The rest of the hive-upgrade.sh should be fine. You shouldn't have to touch/update the rest of it.
Now you can run hive-upgrade.sh:
/path/to/pxeserver$ sudo ./hive-upgrade.sh
and you can run it to check and make sure that it is copying the hiveramfs.tpxz from /path/to/pxeserve/hiveramfs to /path/to/pxeserver/tmp/root, decompressing the archive, and unpacking the files properly.
If it does that properly, then the updating portion of it should be running fine, without any issues (or none that I observed).
Then the next section that you want to check is to make sure that when it repacks and compresses the archive back up, that that should be working properly for you.
Again, it is useful/helpful to have a second terminal window open where you've ssh'd into the PXE server again, with top running so that you can make sure that the pixz process is working/running.
After that is done, you can reboot your mining rig to make sure that your mining rig is picking up the new hiveramfs.tpxz file is ok and that it is also successful in decompressing and unpacking the archive.
I have NO idea how it is doing that because normally, I would have to issue that as two separate commands, but again, it appears to be working with my mining rig.
*shrug*
It's working.
I don't know/understand why/how.
But I'm not going to mess with it too much to try and figure out why/how it works, because it IS working.
(Again, if there are other people who are smarter than I am that might be able to explain how it is able to decompress and unpack a .tpxz file, I would be interested in learning, but on the other hand, like I said, my mining rig is up with the new setup, so I'm going to leave it here.)
Feel free to ask questions if you would want to implement pixz so that you would have faster compression and decompression times.
If your PXE server is fast enough for you such that pxz is fast enough for you and this isn't going to make enough of a difference for you, then that's fine. That's up to you.
For me, my PXE server, running on a Celeron J3455 is quite slow, so anything that I can do to speed things up a little bit is still a speed up.
Thanks.
