Data Storage: From the Floppy Disk to the Cloud

As part of my annual New Year's resolutions, I've been cleaning my home office. And not just the usual surface job, but a real deep cleaning that entails going through every box, drawer, shelf, and other container, ruthlessly analyzing whether I actually need and/or use the object in question and then, more often than not, tossing out or otherwise removing the offender. The goal is for my office to look less like the set of "Sanford and Son" and more like Jerry's apartment on "Seinfeld": clean and, above all, efficient.

This effort has taken the better part of three weeks, believe it or not. But that's because I'm doing it methodically, and because my regular work doesn't just come to a halt because I feel like doing other things too. The task has involved physically removing everything except for the giant L-shaped desk that forms my workspace -- although even that was repositioned, opening up the room and providing more space. But the big thing that's differentiated this year's cleaning is that I'm not setting aside piles of stuff to deal with later. Judgment is immediate, and final.

Except for one category of detritus, that is. Stuffed in drawers and in the back of my closet, I've come across some tantalizing reminders of the past: dozens of hard drives, IDE and SATA, internal and external, desktop and laptop, all full of data that I've put off erasing. And not just hard drives, but other, older forms of data storage: 3.5-inch floppies, 100MB Zip disks and 200MB QIC-WIDE cartridges, hundreds and hundreds of backup CDs and DVDs, and at least three forms of digital video tape.

These must all be carefully dealt with in some manner.

Examining and (occasionally) extracting the data from the hard drives is simple enough, although securely erasing them so that I can safely dispose of them will be time consuming. (I'm using Sysinternals' SDelete and DiskWipe, both free, if you're curious.)

But those other data storage devices, well, those are going to require some work. And although it's possible that I've already converted their contents to more long-lasting storage solutions -- redundant, on-site hard drives on my home server and an offsite, cloud-based copy (in my case using the online service Crashplan) -- I'm going to want to make sure that's so before they're destroyed.

The floppies, Zip disks, and QIC cartridges date back to the mid-1990s. After sifting through the floppies, I realized I didn't need to save any. There were a few smile-inducing oddities -- including two original sets of the MS-DOS 6.22 floppies, but in looking through the collection I recalled that I had already backed up the important stuff long ago. (When I first started writing, my original co-author and I would save multiple sets of backups on floppies and label them according to our respective PCs -- a Dell 486 and a Gateway-based Pentium -- and I can see these on my home server, safe and sound.) So the floppies were tossed.

All the Zip disks are of the original, 100MB variety, so I hunted down an inexpensive, used, USB-based Zip drive online (remember, the original units were actually parallel port-based). That's on the way, and although I'm not completely sure there will be any true treasures in this set of disks, I barely remember even owning Zip disks let alone a drive, so I'm curious.

The QIC cartridges are the most mysterious. Intriguingly, one of them is labeled "95 to NT," meaning it contains my important data files from sometime in 1996, when I switched from Windows 95 to Windows NT 4.0 Workstation. But I'm not even sure what of type drive I can get now to read these things, and it wouldn't surprise me if they never worked at all. I can recall my mid-1990s Pentium Pro, NT-based server having particular difficulty ever successfully restoring data even when this stuff was new. As a result, I still harbor some doubts about cloud storage, because I don't believe a backup is ever valid until you've actually restored it locally.

The optical discs are more recent and are more clearly labeled. They can be safely, but not easily, destroyed, because of the sheer number of them. I'm not sure yet how I'm going to handle this particular task, but perhaps an optical disc shredder is in order. I know the town I live in occasionally offers this service for free, but it's sporadic and impossible to schedule.

Anyway, all of this data, locked into various storage schemes, some easily accessible and some not, has obviously gotten me thinking about the perils of data retention. And it's worse than I've described. If anything, in fact, I've over-simplified the variety of data storage hardware that's come through my home office. For example, just looking at external hard drive storage, I've used USB, USB 2.0, USB 3.0, Firewire (400), Firewire 800, and eSATA-based solutions. Not surprisingly, I don't even have the connectors/adapters necessary to access some of these devices. Optical discs are similar, although at least most of these discs will work on today's combo drives. I've used CD-R, CD-RW, DVD+R, DVD-R, DVD+RW, and DVD-R DL formats, and possibly more, although I've fortunately never fallen for more recent formats such as recordable HD-DVD or Blu-Ray.

What a mess.

And it's not just the storage formats that concern me. The actual file formats in which the data is stored can also be problematic. Unrelated to my recent office cleaning, in late December I came across a PC buying guide I had written in January 1995, over 17 years ago. (I published this as "Blast From The Past: Buying A Computer In 1995" because I found it amusing, given how much has changed since then.) That document was created in Microsoft Word 6.0, and as I discovered when I opened it with Microsoft Word 2010 (which is, by my math, at least seven versions newer), Word documents of that age are now considered unsafe and are blocked for editing by the application default.

Thank the heavens I used a popular format, at least. Data stuck in other, less successful formats would be harder to access, if not impossible, and this situation will only grow worse as technology moves ahead faster and faster. The need for portable and/or open document formats is suddenly all the more obvious. (Recent versions of the Word document format are, of course, open standards, and thus even safer from a retention standpoint.)

This all ties into a theory I have about the evolution of data backup. In the old days (i.e. just a few years ago), much of our crucial data would be tied to a single hard drive on a single PC (for individuals) or on a single server (for companies). And we would back up that data in various ways, using optical media, external hard drives, and/or tape and other methods. Over time, the concept of offsite backup evolved, so we'd physically move some backups, perhaps in sets, to a different physical location, to be preserved in the event of a disaster. (Theft, fire, natural disasters, and so on.)

As connectivity improved, these methods could often be undertaken digitally. A corporate office could back up to a branch office, for example, or vice versa. And an individual could back up to the cloud, perhaps, or, less technically, simply move a USB hard drive to a friend's or relative's house. (I've done both.) Sync solutions evolved so that we could sync our most important data between multiple PCs, providing a modicum of redundancy, and/or to cloud storage, although of course the available storage was, in the beginning, limiting.

Today, we're entering a new era and my now-old fashioned vision of the future -- a storage closet in every home -- is revealed to be short-sighted. We're quickly moving instead into a time period where our master copies of data will be stored in the cloud instead of on a single hard drive on a single PC or server, and our PCs will act like other devices and simply be a sync endpoint for that data.

As with other transformations, this is happening first with consumers and individuals. We see services such as  iCloud and iTunes Match taking users' music collections and putting them in the cloud. We see cloud storage schemes from Amazon, Google, and others putting music, documents, TV shows, and movies in the cloud. And we see a multitude of online document storage solutions, from Dropbox to SkyDrive to everything in between, democratizing our data and putting it in reach from any device at any time. Each has limitations and questions. But each also points to the future, and to change.

Sitting here in my office surrounded by boxes and boxes full of various storage devices and media, many now obsolete, I'm struck by how a decade and a half of faithfully backing up has protected my data, yes, but also left me struggling to deal with its aftermath. Were I starting out today, it's likely that my digital data legacy would be in the cloud, and not tied to physical media in single location. It's likely that this will be the case regardless. In fact, I'm pretty sure it's inevitable, and that any concerns about privacy and security will be quickly overcome.

And no, I'm not going to miss all this junk at all.