the bad sysadmin trilogy part 1 (explained)

I just got another reminder of why I am a bad sysadmin. One of my servers, the one that had Perfect Dark run on it in Wine (with XFCE as desktop manager) crashed 8 days ago due to unknown origins. I say unknown origins because I don’t know what happened. Maybe the entire node had a problem, and I didn’t realize my server was on it, and when the node was forcefully rebooted my server crashed. The log files in the server control panel are inconclusive. According to them I started the server on January 1st, this year. So far so good. I can believe that. Most of my servers have ridiculous uptime. But it then only puts a reboot in the log today, at 10pm, which is when I tried to reboot it since it was caught in a boot loop. So it crashed 8 days ago, possibly because of a node error, and didn’t get back up, and I only noticed it today (which is last Friday when I’m writing this (last Friday being the 3rd of May (this year))).

Now I’ve got it booted up in a rescue system (I love GRML) and the disk doesn’t show partitions, nor is my encrypted LVM found. This is great. Not that there was any data on there that was of particular importance (there wasn’t) but losing any data is still terrible. Plus I lost the connectivity to the Perfect Dark network. That’s actually worth something, to me, in internet bonus points (in a point system made up by me, for me).

I need to figure out if an encrypted volume can be repaired. The problem is I can’t decrypt it because luks can’t find it, and grml doesn’t show me any partitions on it. So it looks like the disk is broken. Great. Just great.

This day just got ruined (speaking about Friday May the 3rd) and technically it got ruined 8 days before. Or the ruining was set up for 8 days.

I’ll report back.