27 July 2021

Here Is The Security Problem With HiveOS' Diskless PXE Boot System

Here is the problem with the HiveOS diskless PXE boot system: 

If you have a HiveOS installation (this security flaw exists regardless of whether you are running the HiveOS' diskless PXE system/setup or whether you are using a disk of some sort for the HiveOS image), that is connected to the public internet (which is preferred so that your cryptocurrency shares that you are mining/farming will have the lowest latency back to the presumably, pool servers as much as possible), and that you have the SSH port open (port 22), then hackers and attackers can remotely take over your system, as laid out in Simon Bell's blog post Sneaky Malware Reconfigures Hive OS Wallet for Profit.
 
After you install HiveOS' diskless PXE system, if you look at what's inside /etc/shadow, you will notice that the salt and the key for user is the same, even if you update the image. (The more interesting question is whether the salt is the same across different users or not, but I am unable to personally test and verify this.)

This is important is because this is how the HiveOS diskless PXE is able to keep it so that the default password for the default user ("user") as "1".

(It uses the SHA-512 based password algorithm.)

The fact that the salt and the key is consistent through different HiveOS diskless PXE images, even if you update the image is one of the reasons why it is fundamentally and principally insecure.

(Ironically, if you do end up getting hit with this by hackers/attackers, one of the first things that you can do if you have a PXE installation is to power cycle your system/rig whereby you can take advantage of this fact that the HiveOS image that is served up by your PXE server will just reset and restore the default setup/configuration of your entire rig and since it will pull the flight sheet from your HiveOS account, you won't have a problem there.

In other words, there are pros and cons to using the HiveOS' diskless PXE setup/system because if you first notice that your crypto in your wallet or pool address isn't increasing as your system is mining and it is supposed to be, then you will be able to very quickly resolve and rectify this problem by just power cycling your rig given the fact that the HiveOS diskless PXE image lives in the local RAM of your rig, which means that turning your rig off -- any of the changes that the hacker/attacker has made will be gone/lost as far as the hacker/attacker is concerned since the image only lived in RAM and not on a non-volatile storage device like a HDD or SSD.

By the same token, the fact that the default password is the same for the default user ("user") also is a security flaw unto itself for HiveOS' diskless PXE system because this is what allows the hacker/attackers shell script to be able to run in the first place.

The HiveOS team, in their Telegram post, writes:


But as my testing and research has shown, you can't actually abide by their recommendation of using more secure passwords because any changes that you make to your machine/system/rig's HiveOS password will ONLY be temporarily changed until the next reboot or power cycle.

To test and prove this, you can either actually perform a reboot, or you can just extract any of the hiveramfs.tar.xz images (or any of the backup images in <<path_to_your_pxeserver_root>>/backup_fs) and then cat ./etc/shadow | grep user and you will see that in your current image and any of your backup images, the result is the same.

The other really big security flaw with this setup is also the fact that the HiveOS team DOES give you a command (hive-passwd) so that you can change the password locally on your system.

The problem with this command is that you have to pass in the new password as an input argument to that command, which means that the password will be typed in, in plain text.

This also means that by default, this system, which is based on the Ubuntu 18.04.5 LTS distribution, also uses bash as the default shell. As a virtue of the bash shell, if you type in the command: history, you will be able to read, in plain text, the password that the user has changed their system to.

HiveOS' diskless PXE setup does NOT clear your bash shell history with each login or with each log off.

And then, on top of all this, if a hacker/attacker is even mildly or remotely sophisticated, and your mining rig is directly connected to the public internet with port 22 open, it won't take them much to be able to find out whether you are running a diskless PXE setup or not. They can literally just read the config files, in plain text, to be able to determine and ascertain that. If you are running a PXE server, then you run the risk where they can start hacking into the rest of your network via your PXE server, or if they are only interested in hacking your mining rig and stealing your crypto, then all they would need to do is to run the hive-upgrade.sh script, and after your system is done updating, they would just need to unpack the PXE image into a temporary location that you aren't aware of on your PXE server, change the salt and the key for their new password and encode that into /etc/shadow, and then re-pack the hiveramfs.tar.xz file back up, and then pass the hacked version of hiveramfs.tar.xz as the "good" image (which as far as I can tell, HiveOS doesn't check), send the reboot signal to your rig so that it will pull the new, hacked image, and now you're hosed to the point where even a power cycle won't help you fix/restore your PXE image.

(And this also assumes that they haven't locked you out of the hive-upgrade.sh so that you can just run the update again, and undo those changes, which I can only imagine that a hacker/attacker that's slightly more sophisticated may probably be able to do that with what I can only assume, would amount to minimal effort (to lock you out of the hive-upgrade.sh command).

In fact, if they were smart, they would delete the hive-upgrade.sh command after they're done modifying your image, and also delete the pxe-setup.sh command as well so that you can't re-setup/re-download the image from the HiveOS server/repository which will undo said hacker/attacker's changes.)

And now your entire PXE setup is screwed until you rebuild the PXE server again, so that you can re-deploy the diskless PXE image again.

These are a bunch of inherent security flaws with the HiveOS' diskless PXE system.

I tried to warn the HiveOS team on the HiveOS forums, but thanks you @HaloGenius there, he decided to take it upon himself to just label this as fear, uncertainty, and doubt (FUD) when the statements that I have presented here are VERY easy to prove (because I've been able to prove them and I'm an idiot as far as computers and sysadmin is concerned). It doesn't take much.

In fact, I am such an idiot that I used the hive-passwd command to change the password to a more secure password and then I forgot what it was that I changed said password to, so I power-cycled my rig so that it would pull the HiveOS image from my PXE server again just to "reset" the password. (Because if you use the hive-passwd command and you mess it up, you don't really have a good way of recovering from that (or at least it's not an easy way to recover from that).) Whoopsies! (When I said that I'm an idiot when it comes to this stuff, I'm not kidding.)

So....what does this all mean? And what can you do?

The best thing that the HiveOS team can do at this point is to take a good, long, hard look at the security issues that I have presented here, building on top of the work that Simon Bell had presented almost three weeks ago (8 July 2021).

What I would LIKE the HiveOS team to do would be that when you run the pxe-setup.sh command for the first time, it will ask you what you want the default rig password to be (you WILL need to make sure that you don't screw this one up because if you toast it, then you will have to rerun the pxe-setup.sh command again to correct it). 

That way, it will download the original image with the default password for the default user, and then right away, you will get to change it and it will replace the line that's in /etc/shadow with the new password, at minimum, and preferably, with a new salt and password.

Right now, it will ask you what your farm hash is, what's the IP address of your PXE server, etc. The HiveOS team should be able to add another, new question, "what do you want the default password to be?" and then you can type it in (preferably echo off), and IF and/or anywhere where HiveOS might need to store that in plain text anywhere, that it will only do so temporarily and then it will clean up after itself so that it won't leave any reminents behind.

When I read Simon Bell's blog and I started thinking about what this means for a diskless PXE setup, I never realised how many failure modes there are that existed with the diskless PXE setup until I started really taking a look into this.

That's would be the minimum of what I would recommend the HiveOS team to take a look at, and implement those changes or something along those lines.

In the meantime, here is what you can do to protect yourself:

1) Secure your local area network. There are no shortage of videos on YouTube that you can look up and watch (or google it and read) where there are people who are dedicated to this topic, so follow their best practices.

2) Use a VPN. It won't really help secure what's inherently insecure, but at least it adds one more layer of protection. It will take your rig off the public internet, and therefore; will increase your latency times for shares, but I suppose that's better than getting hacked, right?

3) Close off port 22 on your router if you don't absolutely need SSH to remote into your network. I know that this isn't always possible, but if and where it is possible, you should probably close that off. (If you DO need port 22 to remain open, there are ways to do so in a more secure fashion as well. You can google and/or YouTube that as well.)

4) Change the password to a more secure password. Yes, I know that you have to pass in your new password in the hive-passwd command, but if you do it right after your system boots up (each time, preferably), then it will help safe guard your system. This is especially true if you've upgraded the HiveOS image on your PXE server.

Keep in mind that any password changes on your rigs are local and therefore; only temporary.

(I haven't checked to see if rig.conf stores the rig password in plain text, but I wouldn't put it past the HiveOS team to send the rig password, via rig.conf, in plain text as well. But I can't say for certain being that I don't use rig.conf files.)

5) After you have changed your rig's password, make sure you clear the bash history on your rig by running history -c. This is important because, again, since you have to pass in your new password, in plain text, as an input argument for the hive-passwd command, this will mean that the bash history will retain a record of it. Running history -c will clear said bash history.

I wished that the HiveOS diskless PXE system would do this automatically whenever you have ran the hive-passwd command by default, but right now, as it stands, it doesn't.

Stay safe.

Happy mining!
 
 
Credits and acknowledgements:

Credit goes to @HaloGenius from the HiveOS forums for deleting my thread and comments about these security issues/flaws with the HiveOS diskless PXE system/setup where instead of, or rather than addressing these security concerns, you took it upon yourself to delete the thread and comments under the guise of this being fear, uncertainty, and doubt (FUD).

As a direct result of your actions there, you inspired me to revive my blog that I haven't touched in years, and turn this blog into me blogging about engineering and IT stuff such as this so I want to acknowledge you, thank you, and give you credit for inspiring me to write this so that people will know what kind of security issues/flaws exists with the HiveOS diskless PXE system, which is sold as a "feature" on the HiveOS features page.

Apparently, security flaws with HiveOS diskless PXE system/setup is a "feature" of HiveOS.