New breed of storage providers hopes garbage trucks need big data, too

If a few years ago it seemed like cavernous hard disks might finally outpace the demand for places to put all our data, the tables have since turned — and a new breed of storage startups hopes to help your business buy some room to grow.

According to IDC, the global demand for storage was 4.4 zettabytes in 2013 — but that demand is projected to skyrocket to 44 zettabytes by 2020.

Much of that data is going to live in the cloud, meaning it's Someone Else's Problem. But a lot of it can't.

Sometimes, the cloud is just too expensive. Even Apple is finding that out the hard way, as it moves around its iCloud storage service providers in a bid to keep costs down while it builds its own data centers.

Other times, it's just too much of a risk or violates compliance rules, whether because the data is too personally sensitive or too critical to the business.

So a new breed of storage providers are hoping to offer more space that's more accessible, with a heavy focus on trying to scale up flash storage. the New York Times just profiled one of those companies, Pure Storage, and gave a hint at just why so many companies — including non-tech companies — are suddenly getting Big Data religion.

Pure's chief executive, John Hayes, has found customers in some unexpected markets, like championship auto racing, where the company won over Mercedes AMG Petronas with its 16 petabyte appliance.

“Health care, manufacturing and natural resources companies can all justify owning this much storage,one analyst told the paper.In 10 years, a big sanitation company with sensors on its Dumpsters to manage pickups could have tens of petabytes.”

That could be a big opportunity for IT professionals — if almost every company needs Big Data, then that's a lot of storage capacity to be managed, and even more data analysis that will need to be done. The question is how much of it will be managed by the businesses themselves, and how much will be farmed out to companies that specialize in data management and analysis.

Will every sanitation company really want petabytes of storage to manage, or will they find it makes more sense to lean on a company that just provides sanitation-data-as-a-service?