Entering the new year, I find myself in dire need of a storage upgrade.
It’s not just because I’m running low on space, although that’s a significant factor driving the upgrade. Another key reason is the need for reliable storage. Don’t get me wrong, I’m not currently experiencing any problems, but the disks in my storage array have been constantly in use for about five years. During this period, I’ve only replaced three or four disks, and it’s only a matter of time before the remaining disks reach the end of their lifespan. As such, I would rather proactively replace my existing storage than deal with the hassle of replacing aging hard disks as they fail.
Given the age of my storage hardware and its dwindling free storage space, a refresh seemed all but inevitable. However, it’s important to recognize that a storage refresh involves more than just purchasing new hardware. The first step in any IT hardware refresh is to precisely define your objectives. Having a list of very specific goals makes the refresh easier to plan. For this particular project, my goals were as follows:
- Increase storage capacity to meet my needs for the next five years.
- Complete the storage upgrade without any downtime.
- Perform the storage upgrade without experiencing data loss.
- Ensure that the new storage maintains or improves upon the current level of resilience.
- Match the performance of the new storage with my current setup.
Even though all these goals were important, meeting the capacity requirement ultimately proved the most challenging. Initially, I had assumed that acquiring sufficient storage capacity would be the easiest part of the process.
Predicting Future Storage Capacity
It’s tough to predict future storage needs. Even so, I came up with what I thought was a pretty good estimate. I took the amount of storage space consumed in the past year, multiplied it by five (i.e., for the next five years), added a 20% buffer just in case, and then added my current data volume. You can see what this looks like below, although I have used made-up numbers rather than my real numbers, just to make the math easy:
- 20 TB of data created in the last year
- 100 TB projected over the next five years
- 120 TB is required when the 20% safety margin is added in
- 250 TB of total estimated storage will be required when the existing 130 TB of stored data is factored in
My existing storage arrays each include 16 drive bays. However, due to parity-related overhead, the capacity of one drive within each storage array is lost, and an additional drive functions as a hot spare. Hence, my usable storage capacity is equal to the combined total of 14 drives. Even with this, achieving more than the required 250 TB of storage is possible by using 20 TB drives, but there are a couple of other factors to take into account.
Potential Issues
One of the my main concerns revolved around the capacity of my current storage arrays. While I vaguely remember that the specs mentioned the maximum drive capacity they could support, I couldn’t recall the exact limit. Additionally, I couldn’t find the original specs for my arrays. Needless to say, I was worried that my arrays might not work with 20 TB drives. I wasn’t even sure if 20 TB drives existed five years ago.
Another thing that bothered me about installing higher-capacity drives into my existing storage arrays was needing the arrays to maintain the same level of performance they currently deliver. In theory, installing larger drives should not diminish the arrays' performance. However, if a drive were to fail, the disk size is what ultimately determines the time needed to provision the replacement disk. For example, provisioning a 20 TB disk would take roughly twice as long as provisioning a 10 TB disk.
Benefits of Replacing the Storage Arrays
Given these and a few other considerations, I ultimately decided that replacing my existing storage arrays would be more prudent than merely replacing the disks within the arrays. In doing so, I opted for arrays that have a couple of extra drive bays. While I would have preferred to continue using my existing storage arrays, upgrading to larger and more up-to-date storage arrays offered at least four notable benefits.
1. Native support for large disks: The new storage arrays natively support large disks, so there is no wondering about the compatibility of high-capacity disks.
2. Optimized disk size and cost: Using 18-bay arrays allows me to use smaller-capacity disks. This means shorter provisioning times whenever a disk must be replaced. Additionally, the lower-capacity disks tend to have a lower per-disk cost, potentially reducing maintenance costs in the long run.
3. Scalability with extra drives: The inclusion of a couple of extra drive bays makes it easier to increase my total storage capacity in the future.
4. Enhanced storage performance: Having extra drive bays should give me slightly better storage performance than what I am getting now. I’m oversimplifying things a bit, but while every hard disk can handle a certain number of random IOPS, the overall performance of a storage volume is influenced by the total number of drives within the volume. For example, a volume (with no parity or other overhead) consisting of five disks can handle five times the number of IOPS that a single disk can. As you add more and more disks to a volume, you increase the total IOPS that the volume can handle. Again, this is an oversimplification, but the underlying principle is sound.
Choosing 18 Drive Bays for Cost Efficiency
Considering the benefits of switching to a larger storage array, you may be wondering why I decided to go with arrays containing 18 drive bays instead of purchasing arrays with an even greater number of drive bays. The decision came down to costs. For whatever reason, going beyond 18 drive bays resulted in much higher costs, based on my experience. I also had to account for the potential impact on power consumption and cooling, especially since the arrays would be installed in my home rather than a data center.
In the coming weeks, as the new hardware arrives, I plan to write a series of articles about the migration process and any lessons learned along the way.
About the author
Brien Posey is a bestselling technology author, speaker, and 21x Microsoft MVP. In addition to his ongoing work in IT, Posey has trained as a commercial astronaut candidate in preparation to fly on a mission to study polar mesospheric clouds from space.
