What I dream of at night (Part 1) A new storage solution

My first NAS (Network Attached Storage) was/is an HP ProLiant MicroServer with 4x 3 TB HDDs and a 16 GB USB stick with Debian Wheezy on it. Configured in a RAID-5 that gave me 8 TB of storage, and it didn’t take me long to fill it up with garbage. Lots of anime, I mean. Totally legally obtained, not at all pirated, anime. And other live action tv shows, cartoon shows, movies and stuff. Oh, and porn. I don’t remember the correct number but it was at least half a TB of porn. Which doesn’t sound much but at the time I got it I wasn’t really into collecting porn.

Nowadays my porn stache is 1.2 TB large. (On non-redundant disks. (More on that later.))

My NAS worked fine for a couple of years. Then I made a grave error in deciding to not give a fuck about the health of my RAID, thus not checking it, and not noticing when one of my drives died. In a RAID-5 setting with four disks, if one drive dies it’s no biggie, the other three drives can carry on (wayward son), but the redundancy that you use RAID for, that’s gone. One more failure, and you’re out.

And those three drives did keep working. Without issues for almost a year. Without issues. The first drive failed in 2016. I shut my NAS down in 2017 before I left for the US of A, and I turned it back on nearly three months later. That didn’t go so well. Predictably (if one had known about the first drive failure (which I didn’t)(at the time)) the RAID crashed when it tried to reassemble. (Is that what it does?) Or rebuild. It doesn’t matter. It crashed. At the time it wasn’t full, I had maybe 4.5 ~ 5 TB of data stored on there. Most of it pirated stuff, so relatively easy to get back from online sources (with the exception of German fansub that were licensed now, because most German fansub groups then remove their download links (but even those can be obtained if one knows where to look)). Only about 200 GB (estimated) were personal files now lost.

In the weeks following the disaster, I spent hours trying to figure out if maybe, maybe there was a way to fix this. Some people on stackoverflow’s unix board/community suggested one might be able to rebuild the RAID if one would first replace most or all of the drives with newer ones, while copying the data with ‘dd’. This “solution” would have required at least 2 (possibly 4) new drives, and I was unemployed at the time and didn’t have the money.

So I dropped it and decided: “Well, fuck it, I guess I’m not getting that back, but I learned a valuable lesson!”

It’s always good if you can learn from your mistakes.

So I’ve been going the past 2 years without redundant storage. Just 3 drives sitting in my desktop, slowly filling up. Their mostly full now. Here’s a picture:

/dev/sdb1 2.8T 2.8T 28K 100% /media/storage0
/dev/sdc1 2.8T 2.8T 2.1G 100% /media/storage1
/dev/sdd1 2.8T 2.8T 8.7G 100% /media/storage2

(Okay, it’s just the output of ‘df -h’, don’t kill me.)

So as you can see this solution is coming to an end. Luckily, since I’m gainfully employed again, I can now rent servers from respectable companies and outsource my storage!

XXX@YYY:/go/to/directory 7.3T 2.6T 4.8T 35% /media/storage3
XXX@YYY:/go/to/directory 2.8T 2.8T 5.6G 100% /media/storage4

The second one is a server hosted by Hetzner, which is already full, as you can see. The first is a Whatbox seedbox, and while the server isn’t full my slot is already at 50% capacity.

Not a long-term solution.

I could get a storage box from Hetzner. Those go up to 10 TB, and would fit all of my data. But I don’t really want to do that. I’d prefer if I could keep my data where I touch my balls. I haven’t come up with an ideal or concrete plan yet, but here are some of my ideas.

From a friend I got a case that can fit 6 3.5″ drives. In my dreams I would get 6x 10 TB WD Reds, and those would either be configured in a RAID-10 (making it ~30 TB) or RAID-6 (making it ~45 TB). Currently those drives retail at ~400€, so about 2400€ total. Not feasable at the moment.

I could get 6x 1 TB. Those retail for ~50€ on Amazon, so 300€ total. But since RAID-0 is not an option, one would have to go RAID-5 (also not an option (I have severe PTSD)) or RAID-6, which would give me roughly 4 TB of storage. Not remotely good enough.

If I go with 6x 3 TB that would cost me approximately 500€, and result in RAID-6 12 TB. This is better and, most likely, the best solution given my financial situation.

6x 4 TB would bring us closer to 20 TB of redundant storage (in a RAID-6 about 18 TB). The pricetag is 700€. My wallet starts to scream again, so I’m going to stop making examples that don’t mean anything, now.

But what about saving money?

If I had been doing that for the past two years, that’d be great. But I didn’t. And I have 10+ figure orders just this year (re: my buyfaggotry article), so saving money would probably take too long. While I don’t have it guaranteed that any of my current drives will fail in the near future, it becomes inevitable the longer they keep running. Some of those drives are 8+ years old.

Every time I check the S.M.A.R.T values of the drives I shiver. I need a solution, and I need it quick.