TL;DR:
Don't use HiveOS diskless PXE because there's a massive, critical security flaw with it.
The issue:
For some inexplicable reason, the moderators and the forum administrators over at the HiveOS forum has decided to take it upon themselves to delete the topic (https://forum.hiveos.farm/t/massive-significant-substantial-pxe-security-risk/43187/1) that I had posted where I talk about the fact that the HiveOS diskless PXE boot system currently has a massive security flaw in it right now. (*edit/update 2021-07-14 18:08 UTC*: I've also written extensively about it in the HiveOS diskless PXE forum thread as well whereby the moderators and administrators of the HiveOS forums are also fully and completely in denial about this MASSIVE security flaw in the HiveOS diskless PXE deployment environment.)
As reported by RedPandaMining in his video, he talks about the current security flaw with HiveOS where if your system/worker/rig is open/exposed to the public internet and the port for ssh (port 22) is open as well, that hackers can then hack into your mining system, change the configuration, change the password and hijack your system to mine for said hacker.
So, my thought was that if you run a diskless PXE setup/configuration/deployment, that shouldn't be a problem right?
Well, yes, and no.
(Note: For the purposes of this blog post, I am going to be focusing in on the security of the HiveOS diskless PXE deployment rather than the actual network security side of things. This isn't to say that one area is more important than any other. As such, you can google how to secure your home network as there are plenty of resources that talks about this. From my research, there is little, if anything, written about the security of HiveOS diskless PXE setup/deployment/system so this is the area that I am going to focus on specifically, in this blog post. In fact, there is so little that's written about this topic and that may have something to do with the fact that the HiveOS forum moderators and administrators will delete any thread or posts talking about this topic, which is why I am writing it on my own personal blog instead since it is quite obvious, this is an area that they don't want to talk about.)
See, the thing with a diskless PXE setup is that whilst yes, you will be able to "fix" the hack simply just by either issuing a reboot command to your mining rig/worker/system via HiveOS' online management interface, or, if you can't get to it because the hackers have hijacked your rig, you will have to be able to physically power cycle the system, then it will just go back to booting like it normally would off of your PXE server, and that would undo all of those changes that the hacker was able to implement.
But that's not the end of it.
But that's not the end of it.
The way the hacked worked, according to Simon Bell, it installs basically a loader script that will connect to the hacker/hijacker/attacker's web server address that will then reconfigure your system (mostly to lock you out and prevent itself from being removed) and then it will download the rest of the payload and supporting files that will then change your actual configuration files specific to mining/the miner itself.
So, with that, again, the thought process could be that if you have a diskless PXE system/deployment, all you would need to do is power cycle your rig/worker and that will restore everything back to its default and that would be the end of the hack/hijack.
But that's not the case.
The truth of the matter is that some of the recommendations that Simon Bell makes are to change your password, both through the HiveOS online management interface, and if that fails/doesn't work (as it happened to be in my case when I tried it), you can always ssh into your rig/worker/system and run the command:
hive-passwd
to change the password locally on the system and that way you can set a stronger, more secure password.
Another mitigating action that you can take is also to disable password authentication for ssh altogether and authenticate only using keys instead which would make it more secure.
And you can google the steps on how to do that or follow the link from Simon Bell's post in regards to the specific instructions on how to do that.
But here is the problem with both of those actions: the moment you power cycle and/or reboot your rig, and it goes through the diskless PXE boot process all over again, it will (re-)download the diskless PXE image (hiveramfs.tar.xz) from your PXE server and then start to unpack it and ALL of those changes that you've just made to secure your rig is now, once again, NOT secure again, as the defaults are restored.
I know because I accidentally tested this.
I tried to change the password for my rig/worker/system via the HiveOS online management interface and that didn't seem to work so that ssh into my rig/worker/system, and ran:
hive-passwd
and changed my password to something more secure. I log out, and when I tried to log back in, it had failed. Maybe my password was too secure to transmit over ssh? Who knows.
Either way, I couldn't log back into my own system again.
Either way, I couldn't log back into my own system again.
So, what do I do to "fix" this?
I power cycle the system and as I mentioned above, it resumed the normal PXE booting process, downloaded the hiveramfs.tar.xz file from my PXE server and proceeded to boot up the rig/worker/system like normal.
But when went to log back into the system, instead of it being the password that I had changed it to, it reverted back to the default password.
That's a HUGE problem.
The default password is built into the hiveramfs.tar.xz image that it creates when you run the pxe-setup.sh initially PXE configuration script as well as the hive-upgrade.sh script.
This means that by default, the diskless PXE boot image is NOT secure because if you change the password and/or if you disable password authentication for ssh, all you would have to do to undo those changes is just to simply reboot the rig/worker/system.
And if your rig/worker/system is still wide open to the public internet and port 22 for ssh is still open on your modem/router, then the attacker/hijacker can attack your rig/worker/system repeatedly even if you were using a diskless PXE setup/server/deployment.
What's worse - if the attacker was more sophisticated, they can actually hack and hijack your rig/worker/system to a point where it will try and see if you are using/running a diskless PXE system/server, find out the IP address of the PXE server, and then try and implement the changes in the hiveramfs.tar.xz file directly so that even if you tried to reboot your diskless PXE rig/worker/system to try and "restore to default", it would be downloading a hacked/hijacked boot image where the hack/hijack would then be built into the hiveramfs.tar.xz image and then this would mean that your HiveOS diskless PXE server is toast (along with your rig).
So....where do we go from here?
Well, for starters, the HiveOS development team should change/modify the pxe-setup.sh script such that it asks you what you want the default rig/worker password to be rather than just cascading or setting up the default password for everything, and making/leaving it all to be the same (and that it doesn't change unless you either change it via the HiveOS online management interface and/or by ssh'ing into your system individually and changing them one at a time).
Secondly, I think that the HiveOS development team should also ask you whether you want to disable password authentication for ssh by default when run pxe-setup.sh script in order to make it more secure and therefore; you DEFINITELY won't have this default user/default password problem that hackers/attackers can take advantage of.
Again, at the per rig level, if you are running HiveOS on a disk of some sort, then these actions can go a long way to help mitigating this type of an attack. However, if you're running a diskless PXE setup, then these mitigations will be erased/cleared out/reset the moment you reboot the system because the inherent hiveramfs.tar.xz file will NOT have these changes implemented and for that to happen, the pxe-setup.sh deployment script will need to be modified by the HiveOS team so that these mitigations will be implemented by default, so that even if you reboot the system, it won't wipe out these security mitigations and worse, put your system back into a state where it can be attacked again.
