Zero Data: Thoughts About Sync, Storage, and Backup

While your personal requirements for data storage will differ from mine, it’s instructive to periodically review not just how much storage you’re using, but what you’re using it for. And as with the physical accumulation of junk in your home, you’ll find that much of what you’re storing electronically is unnecessary and unwanted.

Of course, when it comes to technology generally, and data storage specifically, I’m pretty much on the fringe. In my cellar, I have an HP micro server running Windows Server 2012 Essentials. It has what can only be described as an unreasonable amount of storage, poorly configured. There are five 3-TB hard drives attached to this system, two external and three internal, and it’s sagging under the weight of data duplication and local and online backups.

Seeking a fix to this disaster, I experimented with creating a Windows 8-based “home server” PC, as documented in Windows 8 Tip: Repurpose an Old PC as a Home Server. As I alluded to in that article, a big part of the experiment was seeing how little storage I could get away with, a refutation of sorts to the configuration monster I had created on the micro server.

In the case of the Windows 8 machine, I stripped the configuration down to a single 2-TB data disk and a single 2-TB local backup disk. Today, the data disk has 1.5 TB free. The backup disk has 535 MB free. That is significantly less storage than the server, which has over 15 TB of storage attached to it.

So how did I drop all that storage? And what’s missing on the Windows 8 box?

Obviously, I was aggressive about not copying everything from the server to the PC. And to be fair, as this experiment continues, I’ll be putting more server-based content on the PC. Furthermore, much of what I’ve decided to do is specific to my own needs. So please evaluate this with your own needs in mind.

Documents. I have work-related documents that date back to the early 1990s. On the server, this information takes up about 450 GB. I copied none of this information to the PC, for reasons described below.

Photos/home movies. All of my personal photos and home movies were copied over to the PC. This is about 237 GB of information.

Music. I intend to keep the PC home server free of music, since my master music collection is available via both iTunes Match and Amazon Cloud Player Premium. However, because I’ve been writing about Xbox Music lately, I wanted to match as much of my collection to that service as possible, and I wanted to do so on a stable machine that would be around for a while. So I downloaded the collection from iTunes Match and then manually matched it all to Xbox Music on this PC, over remote desktop. This takes up about 43 GB of space. I’ll be deleting it eventually.

Video. I have terabytes of ripped DVDs and other digital video on the server. One big reason for this is that my son is deaf, needs captioning, and I can reliably ensure that the digital files have this captioning, which he can access at home (Apple TV/Xbox), on his personal devices, and when we’re traveling. I copied none of this information to the PC, though as noted below, I will be copying a small portion of it over in the future.

Software. As time goes on, the need to install software manually from disc has dwindled. On the server, I maintain a Software share that is an amazingly large 187 GB. The version on the PC, which was cut down to just the few local applications I always install and the drivers for PCs and devices I actually still use, is just 15 GB. And even that can and will be culled down a bit further.

OK, so how does one get from here (the server, 15 TB of storage) to there (the PC, with 4 TB of storage, with only half of it used)?

The key is to be aggressive and honest about the storage, and to develop a strategy for determining which content can be stored where. That is, the point behind “zero data” is to reduce my reliance on local storage that I manage and to put as much of this content in the cloud. But people have misinterpreted this to mean that I will be ceding control of my content to cloud providers that may or may not have my best interests at heart. More to the point, it’s important to have multiple copies of your data, for redundancy purposes. This is not about throwing it all away.

What this is about is differentiating between sync, storage, and backup.

With another reminder that everyone’s needs are different, the goal here is for the content to be available where it makes the most sense and as redundant as makes sense for each content type. So let’s go through that list of content types again, so I explain where I’m heading.

Documents

My mostly work-related document archive dates back about 20 years now, but I don’t access most of it almost ever. So I’ve changed how I work, and how I store this information.

I don’t want to lose my archives. My Windows Server-based home server duplicates this content locally, and backs it up locally, both of which preserve the information on multiple physical hard drives in my home. Because it’s backed up to Crashplan, there is also an offsite backup of the data. But my long term goal for this “archive” of documents is to get it all out of my home. And that means two cloud providers, because I feel that this information, while rarely accessed, is still important. The Crashplan backup can sit there, but I’m looking into a secondary service, perhaps Amazon Glacier, as well. Once I get that data into two cloud services, I can decommission it locally.

Of course, not all of my documents are “deep archives” like the stuff dating back to 1993. Because of the nature of my work, I have a number of articles and article series ongoing all the time. I use to store these things on the server. But in 2012, with the advances in SkyDrive and the SkyDrive desktop application, I’ve moved my current work set to SkyDrive. This is replicated across all of my current PCs and, not coincidentally, to that Windows 8-based “home server” PC. I’m organizing things quite a bit differently now, but at the moment, I have my last two to three months’ worth of content available there, and I’m thinking that over time, I’ll begin pushing the oldest SkyDrive-based content—perhaps stuff older than three or six months—to a deep archive in Crashplan, Glacier, and/or whatever.

Speaking of which: The Windows 8 “home server” PC is also backed up to the cloud, in this case to Carbonite. So everything on there, including my SkyDrive content, is backed up offsite as well.

Photos and home movies

I assume it’s understandable that our personal photos and home movies are pretty much the most important content we’re storing, so special care must be taken here. Currently, our entire photo collection is backed up to Crashplan through the server and to Carbonite through the Windows 8 “home server” PC, and much of it was previously backed up to Amazon S3 (when I was using a Windows Home Server plug-in for this purpose) and to Google Picasaweb (now Google+ Photos). I share many photos on Facebook as well, but I don’t consider that a permanent solution. In fact, I’m pretty suspicious of it.

So there are two main issues here: Sharing and safe, permanent backup.

The goal for sharing is obvious: You want your photos to be viewable in ways that make sense and are convenient. Facebook works well for this purpose, and it lets me share photos with people I don’t get to see in person frequently. We’ve used a variety of methods for viewing photos in home, usually through the TV, as well.

I’d like to figure out a secondary permanent backup for photos. I’ve not kept up with the Google stuff because it’s a manual process and is time consuming. And I’ll likely be killing the S3 storage, which costs about $8 a month. (Down from $15 previously, as these costs always come down over time.) But it doesn’t matter: Photos is one area where I will not be shedding my need for local storage. It’s just too important.

Music

I noted before that I had begrudgingly copied my music collection to the Windows 8 “home server” PC for Xbox Music testing. Truth is, I’ll be killing that soon too. I don’t need music files on a server, as the entire collection is available from both Amazon and Apple. It’s also backed up to Crashplan and Carbonite too. Since I use Xbox Music Pass and listen to most music from that, my own collection is increasingly irrelevant. My wife listens to Pandora for the most part.

Music is an easy one. I’m letting go.

Videos

After compulsively ripping all of my DVDs to digital H.264 files—something I did as recently as Christmas Day 2012, by the way—and storing them redundantly on the server and then backing them up to the cloud (Crashplan), I’m going to toss most of it.

As an aside, I had rented a movie from Xbox Video recently and my son asked about watching it the next day. When I told him he could, he sort of reflexively asked If it was captioned, since that’s a requirement for him. And the thing is, so much digital content is captioned now, we’re almost to the point where a video not being captioned is the exception. All new movies on Xbox Video, virtually all video content on iTunes, Netflix, and most other services is captioned. So when we go on vacation in a few weeks, I can let my kids rent a few movies for the trip instead of packing along a hard drive with a bunch of content. Over time, this is actually less expensive—and a lot less work—than keeping the stuff on hand.

Now, I will need to go through the video collection and save a few things because some of it is not available elsewhere (I have a lot of technical videos for example) or is not captioned for whatever reason. But I bet I will save only a tiny percentage of the total video collection. And while I will likely just make a last backup to an external USB 3.0 drive just in case, I’m going to blow the whole thing away.

I’ll write about the specifics of all this as it happens. But in a nutshell, I see it working like this.

Sync: I’m using SkyDrive to sync recent documents, a subset of favorite photos, and a few other things between all my PCs and devices (and, of course, the Microsoft consumer cloud). Usage will grow over time and I’ll be more selective about what gets synced to the PCs/devices.

Local storage. The Windows 8 “home server” PC has my entire photo collection, my synced documents, my software, and will eventually have some videos. My goal is to never expand the storage footprint beyond the two 2-TB disks, one of which is for local backups.

Backup. I sync and backup locally between PCs, and to the cloud (Crashplan/Carbonite/SkyDrive). I will be moving document archives to two online services (Crashplan/Glacier perhaps) and removing my reliance on the home server. This is the local storage I will take offline for good.

It’s hard to write about this stuff at such a high level. I’ll try to tackle some of the content types more specifically in the weeks ahead. Remember: Zero data is a long term goal and like any attempt at decluttering, it’s something that requires vigilance and ongoing commitment.

More soon.