Could Unstructured Data Management Technology Replace the Relational Database Someday?

I’m going to say something that will make many of you mad and wonder if I’m a complete idiot—maybe someday we won’t need relational databases. Let me be clear about a few things before going further. First, I don’t think we’re anywhere close to saying “Thanks Mr. Relational Database, but we don’t need you anymore.” Second, I’ve known the basic rules of normalization longer than some of you have been alive. Trust me, I get databases. But maybe, just maybe, technology is reaching the point in which some data management paradigms that used to be shoehorned into a relational database no longer need to be. Here’s what got me thinking these crazy and heretical thoughts.

I joined the iPhone nation this past November. I don’t want to say it’s changed my life, but honestly, having a useful browser in your pocket and all of the apps is pretty darn cool. Evernote is now one of my favorite apps. It’s sort of like Microsoft OneNote except 1,000 percent better. Evernote is one of those newish classes of applications that basically sticks everything in a bucket and lets you tag your entries. Calling my data in Evernote “unstructured” is actually being generous. It’s antistructure and more akin to having a big filing cabinet and simply throwing random pieces of paper in it. But the indexing and tagging actually work. It works well and fast. I keep all of my receipts in it. I keep business cards in it from conferences. I keep meeting notes in it. More and more, I’m keeping an awful lot of stuff in it. And frankly, the simple tagging model works amazingly well. Evernote can do text recognition from pictures and PDF files. It’s amazing how quickly and easily I can find the stuff I’m looking for. And did I mention that my Evernote data is in the cloud, so it’s magically always there regardless of which device I happen to be using?

I don’t expect that a big telephone company is going to put their call records in a tag-based bit bucket in the cloud any time soon. But I can certainly see certain classes of corporate data management needs being well addressed by these approaches.

And have you ever heard of a company called Google? Sometimes it scares me how smart it is at guessing the full phrase I’m going to type in a search box. Indexing? Fast access to data? Yeah, Google (and Bing) are pretty good there. What if you could Google “Show me the 3^rd quarter sales numbers for the magenta widgets” and the search engine knew exactly what you were talking about through the clever contextual knowledge it has about you?

I don’t think bit buckets will ever replace “real” databases. Locking, concurrency, high availability, one version of the truth, and a few semesters worth of other database topics make me believe we will always need databases. Will databases always be relational? Does it matter? I suspect many problems in the database world will always be best described using relational math and set theory. However, many database pros are at risk of having their heads in the sand when it comes to some of the newer trends involving unstructured data, including, but not limited to, SharePoint. It’s time to pay attention. Your users and customers are.