The Edge Computing Model for Storage Is Looking Better All the Time

Not all that long ago, IT managers were investing tremendous mindshare and resources into cloud migrations and cloud storage. The cloud is still key, of course, but IT is increasingly turning its focus to the edge computing model--especially when it comes to the internet of things and figuring out where to store all of the data IoT devices generate.

In the early days, the edge computing model was all about placing computing resources in geographically close proximity to the users who depended on them. For example, organizations would sometimes place domain controllers and file servers in branch offices in an effort to reduce bandwidth consumption and provide a better overall end user experience.

Today, though, much of the edge computing discussion has transitioned from edge servers to edge devices, including IoT devices.

Although IoT devices differ widely in scope, they are probably best known for producing vast amounts of data. Some devices create more data than others, but it is becoming increasingly common for an organization’s IoT devices to create a greater volume of data than its line of business applications. If left unchecked, this trend will continue as organizations adopt an ever-increasing number of devices.

For IT pros dealing with IoT, the question becomes a matter of what to do with all of the data that is being created at the edge. Conventional wisdom would seem to suggest that the data generated by these devices should be stored in the cloud. However, organizations have to consider the cost of cloud storage and the impact on their available internet bandwidth. Never mind the fact that a loss of connectivity (or even excessive latency) could potentially lead to data loss.

There are currently two main strategies that seem to be effective for dealing with the data that is created by edge devices.

The first of these strategies is to incorporate edge intelligence. Edge intelligence comes in many different forms, but in some cases it can be used to minimize the data footprint. The idea behind this technique is to allow the edge device (or a connected appliance) to preprocess its data. Rather than sending every bit of the data created by the device to the cloud, it may make more sense (depending on the device type) to send only the data that is of interest.

To put this concept into perspective, imagine that you have a security camera watching your front door. It probably doesn’t make sense to store all of the camera’s raw footage in the cloud for an extended period of time. A better option would be to transmit data to the cloud only when the camera detects motion. Similarly, the data stream coming from an edge device can be analyzed as the data is created so that meaningful data can be stored, with the rest of the data being discarded.

Another way that organizations are starting to decrease their edge costs is through edge storage. At its simplest, this means streaming data to an on-site storage appliance rather than to the cloud. This approach frees the organization from having to pay a per-gigabyte per-month storage fee to a cloud provider, while also freeing up a significant amount of internet bandwidth.

Some IoT device vendors are building storage into their devices. This internal storage may exist as an SD card, or it could be based on another technology such as NVMe. Regardless, this approach is used primarily in situations in which the IoT vendor recognizes that device data will be largely transient. Again, consider a security camera as an example. For most organizations, it is perfectly acceptable to overwrite security camera footage once a certain amount of time has passed. If the camera happens to record something interesting, that data can always be offloaded.

I recently saw one example of an edge device that used both preprocessing and integrated storage. While I can’t give you a lot specific detail because of a nondisclosure agreement, I can tell you a little bit about the device’s architecture. The device was designed to stream its data to a backend application that would filter the data and decide what to save. The device’s internal storage acted as a buffer, protecting the data in the event that connectivity was lost.

The rate at which IoT and other devices are being placed onto our networks seems to show no signs of slowing down. As such, organizations will need to come up with creative ways for dealing with the massive volumes of data that these devices can produce. Depending on the nature of the data, the use of the edge computing model may be a viable, low-cost option.