ClearSky's Cloud Storage Passes Myers-Briggs' Data Management Test

When most people think of Myers-Briggs, the first thing that comes to mind is its famous personality test. But the company does much more. In fact, these days Myers-Briggs thinks of itself first and foremost as a technology company—one that helps organizations develop leaders and improve teams and hiring processes.

Providing these services to companies requires fast, responsive service on the front end, as well as efficiency on the back end. It also requires secure access to large amounts of storage, along with solid backup and recovery.

Until a few years ago, the company relied on a traditional on-premises storage infrastructure instead of cloud storage. As data stores grew, it became more and more complex, labor-intensive and expensive. The result was less than optimal response time.

Clearly, it was time for change, and it made sense to attack the problem by addressing the storage issue specifically. According to Michael Johnson, the company's director of global infrastructure, "We all know whatever storage you put behind a system is really the bottleneck of the system."

Johnson took his time, looking for the best capacity, performance and price. He also put a premium on reducing operational capacity. Along the way, he learned about a different way of accessing and managing storage: the cloud caching model. Instead of the traditional method—back up data onsite, sending that backup offsite and replicating the data to a secondary site—this model worked differently. It would decouple the storage from the compute but cache it locally, retaining the performance of an all-flash array. Different tiers of storage would be sent to the cloud, with snapshots taken along the way. The data would then be sent to a third cloud-based location of the company's choosing, such as Amazon S3, Google or Microsoft Azure.

With this method, the cloud caching company would own the relationships with the cloud storage vendors. This would solve Johnson's problems, he thought, reducing complexity, providing better responsiveness and uptime, and reducing cost. So Johnson jumped at the opportunity.

While the idea was good, the solution Johnson chose didn't provide the resiliency he expected. The last straw was the day that the system crashed, ruining Johnson's vacation and causing the team to spend a full week trying to recover everything.

Despite the false start, Johnson still believed the cloud caching model had merit. So when he came across another opportunity in early 2019, he decided to give it another try. This time, the vendor was ClearSky, which delivers on-demand primary storage with offsite backup and disaster recovery, all as one service. ClearSky owns everything: the bandwidth, the primary storage, and provisioning to both a second and third tier.

Here's how it works: A 2U flash appliance fully managed by ClearSky sits at Myers-Briggs' headquarters. That appliance handles "hot" data—data likely to be used within the next few weeks. According to Johnson, this data represents about 10% of the total data set.

Somewhere nearby, an Equinix IBX data center stores and caches both warm and hot data from Myers-Briggs. If needed, Myers-Briggs can access the data within just a few milliseconds. Cold data, as well as a master copy of all data, is stored in the cloud. ClearSky uses an intelligent algorithm to move data from on-premises to Equinix to the cloud.

With this model, ClearSky manages everything. It owns and provisions the bandwidth, using multiple carriers to create redundancy and ensure uptime. ClearSky serves up the storage to Myers-Briggs as needed and constantly monitors the system to intercept and fix issues that may occur.

The ClearSky solution also will allow Myers-Briggs to keep all data in its originating country, inside an Equinix data center. That will help immensely with General Data Protection Regulation (GDPR) compliance requirements, he said.

Everything Has Changed

With this setup, virtually everything about backup and recovery has changed, Johnson said. Most importantly, it has simplified data management at every level. No longer is Johnson's group responsible for sizing, managing or provisioning storage. "We set a recipe and forget it. That's it," he said.

Both recovery point and recovery time objectives also have improved significantly.

"With the traditional way of doing things, it was taking hours for us to come back up again, and there would be some data loss. Our RPO was two to three hours, and RTO was four to six hours," he said. This way, he said everything is up and running within minutes, with zero data loss.

The company also no longer has much need for racks of storage devices in the data center. After the transition, Johnson said the data center's rack space was reduced by 70%.

In addition, Myers-Briggs has been able to cut bandwidth costs by two-thirds. Today, Johnson said he has three times as much bandwidth at one-third the cost.

"We had approximately 500 megabytes of bandwidth for a point-to-point line on one link, and they were able to increase that to 1 gigabyte while reducing our cost by two-thirds to that one link alone—and that doesn't even take into account a 200MB circuit upgraded to 1G and another 300MB circuit also upgraded to 1G," he explained.

In addition, Johnson is saving the company money, both up front and over time. He estimates saving up to $300,000 over the next five years, but says that he actually expects to save much more after that point. That's because he won't have to spend time negotiating contracts, and his team can spend time on other tasks.

Johnson is also considering establishing access to ClearSky from VMware Cloud on AWS as a datastore option, alongside the cloud storage option that VMware Cloud delivers. Right now, however, VMware Cloud on AWS offers only 10TB of NVMe storage on each ESX host, which would require Myers-Briggs to have eight ESX hosts with 10TB each, served up as one pool of storage. That would work for today, but that is currently the cap, which wouldn't allow the company's storage to grow. Johnson hopes that will change over time.