Exploring Cairo

No doubt, the massed choirs of Windows 95 users aresinging from the rooftops, because Office 95 under Windows 95 allows longfilenames. They think this is going to be of so much more use to them; 8+3 was adreadful limitation, even 15 years ago.

But what narrow thinking! Going from an 8+3-character filename to a256-character version of the same thing is progress only in as much as it's amove from disastrous to meagre on the usefulness scale.

Let's look at it another way. If you were to take all the documents onyour desktopcomputer-or, better still, your corporate LAN-and put them into one directory,would 256-character filenames really be enough? No, of course not! You'd stillwant the filtering, sorting, and collating facilities offered by subdirectories.But as soon as you put documents into subdirectories, you're scattering yourdata into the unknown far reaches of your hard disks, never to be seen again ina hurry, because the retrieval tools are so weak and rigid.

Indeed, to look at it from still another perspective, think about a databasedesign. You would definitely want a unique key-or a unique combination ofkeys-on a database entry. Allowing multiple entries to have the same master keyis a recipe for disaster. Suggesting that a database manager scatter datarandomly across multiple tables so you can sort and query them more easily wouldundoubtedly end in a major argument. The database manager would argue that allyou need is to add another field or two to each record to uniquely define it.

Now spin that mindset into a disk-filing system, and you can see how even a256-character filename strategy is pathetically weak. You need unique documentidentifiers and multiple "database-alike identity fields" tocategorize your data. Hence, even 256-character filenames are a largely uselesssolution for the late '90s, especially since gigabytes of disk space storinghundreds of thousands of files cost mere hundreds of dollars today.

Not surprisingly, Microsoft has a strategy in place to sort this mess out. It'scalled Cairo. For what seems like an eternity, Microsoft has been proclaiming, "Weare all on the road to Cairo." Indeed, this future, still unseen release ofWindows NT has been coming for so long that it has gained almost mythicalstatus. Private discussions with Microsoft have clearly indicated that Cairohasn't vanished, that work has been steadily progressing over the last couple ofyears, and that 1996 is going to be the year of the grand roll-out of Cairo-atleast initially-in beta form.

So, I thought it would be useful to explore the current state of play to seewhere this mythical Cairo technology is coming from. Hopefully, by the end ofthis journey from Office to Cairo, you'll agree that large components of Cairohave been sitting on your desktop in full view for nearly two years.

Of course, there are many facets to a large project like Cairo. Although I can'tspeculate now about how all of them will work, one clear thread is worthpursuing: how the object-storage system works today-how it might well bedeveloped for Cairo and how it will help solve our data storage and retrievalproblems.

Office and Storage
The questions must have crossed your mind: When you embed an Excel worksheet ina Word document and then save the document, how does Word "know" howto store the Excel file? Where is the Excel file stored? And if it's stored inthe Word file, is it converted to a Word table?

Since you can reload the Word file and then double-click on the Excel file tobring up an in-place editing instance of Excel with all the editing andformatting capabilities of Excel supported, then the Excel worksheet must bestored natively within the Word document. These functions simply wouldn't bepossible if Word converted the Excel table to a Word table.

If you embed a different kind of document, such as a PowerPoint slide, the samelogic applies: The native PowerPoint presentation is stored within the Wordfile. Indeed, Word has no native concept of a presentation so it's hard toconceive of how such a Word-centric conversion could work.

Normally, this is the stuff of private data formats. It would immediatelysuggest that Microsoft had written some private Office data format that couldstore any data file from any Office application. That would make complete sense,except for one thing: You can embed any data file from any Object Linking andEmbedding (OLE) 2-supporting application in a Word file. The implication isclear. This "embed and store" mechanism can't be relying on privateformats, because you can embed objects from applications that have no connectionwith Microsoft. Thus, adding a Visio drawing is no more difficult than bringingin an Excel worksheet.

There must be a standardized, open-storage, container-and-content model at workhere, something that any vendor can tap into. There is! It's called StructuredStorage, and it's been around since the release of OLE 2.0.

Structured Storage is fascinating for many reasons, not the least of which isthat so few people have actually looked carefully at how it works. Fortunately,Microsoft has tools that let you look inside a Structured Storage file to seewhat's going on. The best tool to use is DFVIEW.EXE, which is a StructuredStorage DocFile viewer supplied with Microsoft Developers Network (MSDN) Level2. In addition, there's a good book on OLE 2 by Kraig Brockschmidt entitledInside OLE, Second Edition from Microsoft Press. The chapter "StructuredStorage and Compound Files" is detailed and informative, although as awhole the book is a deeply technical reference that warrants several readings.

To illustrate how Structured Storage works, let's look at a basic Word 7 file. Icreated a new Word document, put two words into it, and saved it to C:tempword1.doc. I then loaded this file into DFView (see Screen 1).

You can see that it shows a hierarchical layout with the top level being thestorage file itself. This is the file on the disk. At the next level, there arefour streams of data: The first is called Comp-Obj for Compound Object; thesecond is called WordDocument and contains the actual Word document data. Thethird and fourth are a SummaryInformation stream and aDocumentSummaryInformation stream. These last two contain summary informationtags like Author, Date Created, Size, Last Printed, and so on. Each of thesefour streams is a separate data stream in the Structured Storage file.

Having closed the DFView display of the file and reopened it in Word 7, Iinserted a simple two-cell, two-row Excel 7 worksheet into the Word file, usingthe Excel button provided on the Word button bars. Then, I resaved word1.doc andreopened it in DFView (see Screen 2).

Things are definitely becoming interesting. There are still the same four Wordobject streams in place, but there is also a new stream, called ObjectPool. Ifyou open the ObjectPool stream, you'll see another storage object with anobscure numbered label. If you then open this object, you'll see what looks likeanother complete document stream, just like the one at the root of the tree.There are the same CompObj, SummaryInformation, and DocumentSummaryInformationstreams. But the WordDocument stream has been replaced by a Book stream. Thiscontains the Excel Book worksheet information-the Excel worksheet object thatwas inserted into the Word document. In addition, there are a couple of extrastreams here to render the object when its container is running. In other words,when you are editing the Word document, you want to see a visible representationof the contents of the Excel object, even though Excel isn't running at thatpoint.

Let's take this one step further and put a WordArt object inside the Excelobject, which is inside the Word object. Phew, my head is spinning. How aboutyours?

Screen 3 shows what's happening-I have in-place activated the Excel object, andthen inserted the WordArt 2 object into the Excel object. You can't currentlyin-place edit more than one level deep-OLE 2 doesn't support multiple nesting ofall the negotiation that goes on between the various applications. So, althoughWord will negotiate with Excel to bring up the Excel "look and feel"in the Word framework, it can't negotiate with WordArt to go two levels deep.

Let's look at this triple-layer sandwich using DFView to see the result of thismultiple embedding (see Screen 4). Not surprisingly, there is a WordArt object stored inside the Excel object, which is stored inside the Word object.

What Does It Mean?
OLE 2 containers can contain objects of completely foreign formats, and as faras the container is concerned, the contents of the embedded object are opaque.Word doesn't know-or need to know-how Excel stores its internal data. Nor doesExcel need to know how WordArt stores its data. They simply need to know how tocontain them, and this is defined in the Structured Storage standard.

Although I only inserted one object into another, there's no limit on the numberof objects you can insert in parallel. For example, I could have put two Excelworksheets, each containing different objects, into the Word document. Theresulting tree structure is easy to imagine. Just think of Structured Storage asan object filing system. The similarities between what DFView has revealed andwhat's in a standard directory/subdirectory/file structure are plain.

Into a Binder
Office 95 includes a new application called Binder. It allows you to take Officedocuments and put them into a composite file together. Binder embodies theconcept of putting the contents of a folder (or a directory) into a singleobject. You can then switch among the various objects in a binder, and Binderdoes all the in-place editing activation necessary using OLE 2.

How does it work? Binder uses Structured Storage for its file store. Forexample, I created a new Binder object and dragged and dropped the compositeWord-document object into the binder. Then I created a new Excel object in thebinder (see Screen 5). Look carefully at this structure. It has two main Binderobjects, called 1 and 2. The first is the Word document with the Excel andWordArt embedded objects. The second is the Excel sheet I dropped into thebinder. Finally, look at the standard Binder type at the top level, togetherwith its SummaryInformation and DocumentSummaryInformation streams.

Are you starting to see a pattern? The reality is: With Structured Storage,Microsoft has been running file-sized full-blown object filing systems. Indeed,a Structured Storage file could be regarded as a full-blown storage engine initself-a sort of prototypical Cairo Object Filing System! This is why, when somecorporate IT managers call Microsoft's ability to create an Object Filing Systeminto question, I point out that there are already hundreds of millions ofdocuments out there that use Structured Storage and they work just fine, thankyou. Microsoft has been using this storage engine in Office ever since version4.x.

Binder is important, because it's the first application from Microsoft thatworks "one layer out" from the main applications. Binder is acontainer of Word, Excel, PowerPoint, etc. objects. As such, it is effectivelyan active container surface into which you can pour data objects to work onthem. Imagine the Windows 95 shell allowing direct Binder-type operations, andyou can see how a Cairo active desktop might well be a container system itself.

Near Future
Now, let's make a leap sideways to a forthcoming piece of software fromMicrosoft-Exchange Server (see "Groupware: The PC Team Sport" on page33). As you may know, Exchange Server replaces Microsoft Mail and brings, atlast, a full Messaging Applications Programming Interface (MAPI) to the desktopin a proper client/server email system.

However, Exchange Server has more important tricks up its sleeve. It has PublicFolders, which automatically replicate data among Exchange Server engines aroundyour enterprise. If you wish, you can use Public Folders to store email. Butit's far more potent to drop Structured Storage files in them. Just fire up FileManager (or Explorer, if you're running the Windows 95 shell), and drag and dropa pile of Word, Excel, and PowerPoint documents into a Public Folder. ExchangeServer swallows up the files and stores them within its active storage engine(see Screen 6).

Now the magic can really start. Look carefully at the columns in the Folderview-no great surprise. There are columns detailing From, Subject, Received, andSize. However, after a little fiddling with the Folder columns settings, I'vemade dramatic changes to the information shown (see Screen 7). Exchange Serveris "surfacing" the properties applied to each object-in this case,Manager, Client, Value, and Authorised. Manager is a standard property held inthe Summary tab of the file Properties window in each Office application. Theother three are custom properties that I added to each object (see Screen 8).

Since Exchange Server is storing these documents in an active repository, theycan be replicated and distributed around the organization. I can attach fullsecurity rights to the Folder and its contents. But most important, I havecompletely removed the importance of the filename from the object. No longer doI need to refer to an object only by its filename.

Although Windows 95 and NT have full 256-character filenames for files, that'sfar from enough. Length is not the issue; the granularity and the number ofattributes that can be attached to an object is the real key. For example, if Iwant to mark all the files belonging to Project X, I don't want to have to putProject X into each filename. It would be far better to have a Project Nameattribute attached to each object. And Structured Storage, Office, and ExchangeServer allow me to do this. These tags are stored in the SummaryInformation andDocumentSummaryInformation streams. Each Structured Storage Office-compatibleapplication has these streams, even for subobjects in a compound embeddedobject.

Power Exchange Operation
For anyone interested in a really powerful document management system, Screen 9should raise the hair on the back of your neck. Exchange Server has, at myrequest, sorted and collated the objects in this Public Folder by the Managerproperty and presented the results in a tree diagram. You can collate up to fourlevels deep, so there's no problem displaying a tree of customers, where eachcustomer's information is sorted by "Invoice Paid? Yes/No" and then bydate, or something like that.

If you attended the December 1993 Microsoft Windows "Chicago"Professional Developers Conference in Anaheim, California, you'll probablyremember the awe-inspiring demonstration of Cairo given on the first day. Thesorting and collating I've described was demonstrated as a Cairo feature.

On to Cairo
At this point, I must make it clear that I'm dropping out of the realm of theknown and into the realm of informed speculation. However, what I'm describingis logical and consistent with the above.

Exchange Server uses a modified Jet database engine for its storage. It's knownthat Cairo has a native Object Filing System (OFS) based on the NT File System(NTFS) disk format. NTFS is a very flexible storage engine and has many featuressimilar to those of a transactional database engine. For our purposes, the mostimportant feature is that each file object can have an unlimited number ofproperties attached to it. (I don't want to go into too much detail on NTFS.Suffice it to say, Helen Custer's Inside the Windows NT File System fromMicrosoft Press is required reading.)

Reading Custer's book and matching it up to the Exchange Server and StructuredStorage information, coupled with discussions with senior Microsofties, leads meto believe that NTFS will be extended under Cairo. It will appear to bebackwardly compatible with the current NTFS, but it will have a whole new set ofactive processes running on top of it. In other words, the Jet database-styleengine of Exchange Server will be replaced by the Cairo OFS, based on NTFS.

NTFS-based OFS has everything you need: It can hierarchically store objects; itcan index on properties; it can store Custom properties; and it is robust and aproper filing system. For example, when you save a Word file, it talks to thelocal OLE storage engine running on your machine. This engine creates theStructured Storage disk file, and Word pours its contents into the file. The OLElibraries manage all the file handling.

When you store documents in Exchange Server, Word talks to the OLE storageengine which talks to Exchange. Word basically pours its contents into theExchange Server engine, again via the OLE libraries. No major changes werenecessary to allow Word to do that. In the Cairo future, the OLE libraries willtalk to the Cairo OFS store, and Word will pour its data into that instead.

What more will Cairo uncover? Exchange Server only sees the outer layer of theonion. It doesn't look inside the Structured Storage object. It pulls out theSummaryInformation, but embedded objects within the document are still hidden.

Cairo will actually break a Structured Storage document into its componentparts. This will allow it to display all the components in the document and toshow all the standard and custom Summary tags for each object too. Remember,each object in a compound document has its own set of tags.

This revealed structure will enable multi-user document creation and editing.Since Cairo, by virtue of its NTFS host, will be able to do database-stylerecord locking on each individual component of the Structured Storage document,it will be possible, for example, for several users to work on different partsof a compound report, enabling much finer document indexing and retrievalprocesses. After all, a list of "hot words" on each object is justanother set of custom property tags that can be individually indexed, sorted,and collated in the Cairo OFS storage engine.

The Path Is Clear
By focusing on how Structured Storage works and the benefits it brings, you cansee the direction that Microsoft is taking and how Exchange Server is a big stepalong that road. Ignore all the email parts of Exchange Server, and lookclosely-it's really Cairo 0.75.

It's also important to realize that Cairo won't be just a server-side solution.To operate at its best, it will need client-side applications that understandboth OLE and Structured Storage. It's not surprising that Microsoft's own Officeproduct has been doing precisely this for the past two years.

I'm told that there will be a clean and simple upgrade path from Exchange Serverto Cairo itself, when Cairo is ready to ship. Indeed, Exchange Server 2 willprobably be fully Cairo-hosted. This is a sweet victory for Cairo, because allthe indications are that Exchange Server 1 was never really supposed to happen.According to all the time scales and published plans, we should be running onfull Cairo by now.

The delays in Cairo-and in getting NT 3.5 out the door-mean that Microsoft hadno choice but to put together an interim solution. Exchange Server 1 is thatsolution. And although it may be an interim solution, it offers some extremelyinteresting technologies for the future that you can put to work today. Theexodus from the filename starts now.