RAID != BACKUP
This is a story worth sharing with anyone concerned in the tech community.
It’s a sad tail describing the death of a popular Internet service, Journalspace.
A Goodbye from Journalspace
Journalspace is no more.
DriveSavers called today to inform me that the data was unrecoverable.
Here is what happened: the server which held the journalspace data had two large drives in a RAID configuration. As data is written (such as saving an item to the database), it’s automatically copied to both drives, as a backup mechanism.
The value of such a setup is that if one drive fails, the server keeps running, using the remaining drive. Since the remaining drive has a copy of the data on the other drive, the data is intact. The administrator simply replaces the drive that’s gone bad, and the server is back to operating with two redundant drives.
But that’s not what happened here. There was no hardware failure. Both drives are operating fine; DriveSavers had no problem in making images of the drives. The data was simply gone. Overwritten.
The data server had only one purpose: maintaining the journalspace database. There were no other web sites or processes running on the server, and it would be impossible for a software bug in journalspace to overwrite the drives, sector by sector.
The list of potential causes for this disaster is a short one. It includes a catastrophic failure by the operating system (OS X Server, in case you’re interested), or a deliberate effort. A disgruntled member of the Lagomorphics team sabotaged some key servers several months ago after he was caught stealing from the company; as awful as the thought is, we can’t rule out the possibility of additional sabotage.
But, clearly, we failed to take the steps to prevent this from happening. And for that we are very sorry.
So, after nearly six years, journalspace is no more.
If you haven’t yet, visit Dorrie’s Fun Forum; it’s operated by a long-time journalspace member. If you’re continuing your blog elsewhere, you can post the URL there so people can keep up with you.
We’re considering releasing the journalspace source code to the open source community. We may also sell the journalspace domain and trademarks. Follow us on twitter at twitter.com/jsupgrades for news.
Yea, ’cause OS X is really known for randomly overwriting entire disks.
It’s worth saying again: RAID configurations are not the same as backups performed routinely, separately, and on media that will last.
If you liked this post, then please be sure to subscribe to my feed.