ALERT Nightmare at HostMediaUK

[wardmundy]

We've had a VPS in England for a number of years where we run our main Fusion PBX platform. Until this last week, the beauty of the platform was that HostMediaUK generates an automatic backup of your VPS every week and keeps them for a month... luckily. Then this past Sunday rolled around and not only did our VPS disappear but also the server console. Nearly a dozen of their servers suddenly went off line as shown here. The host for our VPS still is not fully functional after FOUR DAYS. After watching the server and our VPS reappear and disappear a half dozen times over the past couple days, here was the cordial greeting (to the wrong person) that I received earlier today:

Hello Grant,

To clarify, the instability was caused by a root-level compromise on the parent host node (Aqua), where a malicious systemd service (ustar.service) was installed. This service repeatedly executed malicious binaries and modified /etc/ld.so.preload, causing massive CPU spikes on the hypervisor that resulted in the platform outages you experienced.

We are currently reviewing the latest available backup for your VPS to ensure a clean and successful restore. Please note that we cannot provide an exact ETA at this moment, as we need to coordinate the process with our relevant team to ensure everything is handled correctly.

We are moving forward with this as a priority and will keep you updated on our progress right here. Thank you very much for your patience and understanding.


Best regards
Host Media Team

Long story short, it turns out the malicious software wasn't on our VPS. It was on their host after they had told us just the opposite and recommended loading a backup on our VPS (which they did). I think this marks the first time I have ever seen an entire hosting platform compromised which they now acknowledge was caused by malware named ustar.service.
MORAL OF THE STORY: If you ever want to fully compromise a hosting provider, do it on Sunday. The third string techies had no idea what was happening.

 

[wardmundy]

Update: Back off line after less than 2 hours. This is about the fifth or sixth crash we have observed, and the 2 hour timetable is about what we have come to expect.

 

[TaylorDrue]

Sorry to hear this Grant.

Just kidding Ward, don't mean to make light of the situation. Thank you for informing the community.