Remote Storage Service

Until recently, I didn't pay much attention to Microsoft Remote Storage Service (RSS). However, as more companies consider deploying their Windows servers and applications on high-capacity Storage Area Networks (SANs) and other "storage utility" environments, and as application data continues to bloat, RSS is becoming a tool that Windows administrators can't afford to ignore. Let's examine RSS and consider what it can do for your organization.

RSS: Microsoft's HSM
If you've worked in IT for any reasonable length of time, you know that the promise of Hierarchical Storage Management (HSM) preceded the Windows platform. HSM lets you prune data from total storage to accomplish key goals such as reducing total storage consumption, shrinking backup windows, and migrating unneeded data from local storage.

I've always thought of HSM as a three-tiered approach to data management. Tier 1 is online storage, at which data remains immediately accessible to applications and user requests. This storage includes straightforward nonremovable storage mechanisms such as local hard disks, SAN storage, and application stores. Tier 2 is near-line storage, which is data you've migrated to an intermediate secondary storage container (either removable or nonremovable) in the form of a secondary magnetic disk device, an optical disk device, or even an application-specific storage mechanism. Tier 3 is far-line storage, which is data that you've migrated to a removable storage device such as tape or other media for later use and recovery.

What drives HSM products is the need to segment data stores into context- and time-sensitive containers of manageability. For example, as a Microsoft Exchange Server administrator, I can try to work with a monolithic mass of data that continually frustrates me and leaves me feeling helpless, or I can implement a data-management strategy that includes HSM to gain control of that data. If I'm a Windows administrator managing a file server with terabytes of data, I can manage this data as one set or approach it as several subsets—for example, frequently used, rarely used, and never used data. I can then decide whether to include policies for each subset of data (e.g., back up frequently used data every night, migrate less frequently used data to tape). These features are the essence of HSM technology that many enterprises are beginning to think about in the Windows space.

Microsoft introduced RSS, its implementation of HSM, with Windows 2000. However, RSS is simpler than HSM in that it differentiates between just two levels of storage—local and remote. As a result, RSS migrates data transparently from local disk storage to remote devices such as tape libraries.

The Win2K Server OS includes RSS, and computers running Win2K Professional are also HSM-aware. Many elements of the storage, application, and GUI are HSM-aware and work together to ensure that HSM integrates fully into the user experience.

What Remote Storage Buys You
Some of RSS's benefits are immediately obvious. The concept of managing data files from primary storage paves the way to cost savings by limiting storage growth (or at least flattening the growth curve), which sounds good to administrators—especially when it requires little intervention on their part. You can encourage users to store data on file servers so RSS can manage that data according to its use. Local-server storage becomes the primary place for frequently used files, and infrequently used files can go elsewhere. The idea is to have data reside on the most effective media according to the data's usage profile (i.e., frequently or infrequently used). You can base policy-driven file migration on file size, last-access date, and custom inclusion and exclusion rules.

To users, files in local storage appear much as they did before RSS. However, users will notice some differences in files that you've migrated to remote storage. From a command line, users will notice that files residing in remote storage have parentheses around the file size in the directory listing. From Windows Explorer, files appear with a high-latency icon, as the three designated files in Figure 1 show, to denote that they reside in remote storage and will therefore take a bit longer to read or open. These placeholders, known as Reparse Points, tell the Windows I/O subsystem about a migrated file's status.

Recalling files from remote storage is almost transparent to users. Files that you migrate to remote storage can exist in two states in local storage: premigrated and truncated. A premigrated file exists in both local storage and in remote storage, but RSS has flagged it. A truncated file is simply a placeholder for a file whose data stream resides in remote storage. In either case, when you open a file, you trigger the reparse mechanism in the file placeholder, which causes the RSS system (a file-system filter driver and file-system agent) to recall the complete file to local storage. This kernel-mode process ensures that file recall operations occur rapidly and efficiently.

RSS can trim storage costs by reducing the amount of storage infrastructure required. By keeping frequently used files in local storage and infrequently used files on tape, you reduce your total storage requirements. You might also realize benefits in areas such as backup, because important data will take up less space, resulting in a smaller data-set size and smaller backup and restore windows for mission-critical data.

The Costs
For some organizations, RSS has no inherent additional costs because it can leverage existing Windows removable storage devices that are in place for disaster recovery. However, on a larger scale, HSM solutions often require additional infrastructure that complements or is in addition to existing backup devices.

I'm not aware of any best-practice recommendations about which option is preferable. However, I would be inclined to manage an RSS deployment as separate from my disaster-recovery facility, which means that my RSS deployment would incur additional costs. You must weigh the trade-offs carefully if you choose to look more closely at RSS. In other HSM products and future implementations of RSS, support for multiple tiers and devices other than tape tend to contribute to higher costs of implementation. In addition, HSM initiatives do incur additional management overhead as administrators need to plan, deploy, administer policies, and manage (including providing disaster recovery for RSS) their HSM implementations.

What's Next for RSS?
Most planned enhancements to RSS address key shortcomings that accompanied the Win2K version. First on customer-feedback wish lists is optical support, which is also an important competitive feature for Microsoft because other HSM products already support optical media. Windows Server 2003 will support optical media (i.e., magneto-optic—MO—not DVD) for the remote storage tier, providing an additional choice and, more important, adding a higher performance option when recalling files from remote storage (because optical is a bit faster than tape media). Also on the wish list is command-line support. These new features will add functionality and bring RSS more in line with other HSM options, which are inherently more expensive (because Microsoft includes RSS with Windows Server products, it's essentially free). If you're looking for ways to reduce the size of your primary storage pool, consider RSS.