Inside NT's Object Manager

Uncovering this important but little-knownExecutive subsystem

The Object Manager is the Windows NT Executive subsystem thatreceives the least amount of attention or recognition. Ironically, the ObjectManager provides a resource management support infrastructure that all other NTExecutive subsystems (including the Memory Manager, I/O Manager, and ProcessManager) rely on. The Object Manager is a support subsystem that performs itswork behind the scenes. As an NT systems administrator or a Win32 programmer,you probably will never interact directly with the Object Manager; however,almost everything you do, from opening files to starting a program to viewingthe Registry, requires its assistance.

In this tour of the Object Manager, I'll first describe where the ObjectManager fits into NT's architecture, the role it plays in common operations, andthe services it provides. Next, I'll explain how NT's subsystems define objecttypes and what kind of information the Object Manager tracks. Finally, I'll lookat the Object Manager namespace, which is the doorway to file system namespacesand the Registry namespace.

Resource Management
A major part of NT's role is managing a computer's physical and logicalresources, such as physical and virtual memory, disks, files, processes,threads, synchronization primitives (semaphores, events, etc.), printers, andvideo displays. NT must provide a mechanism whereby programs can look upresources, share them, protect them, read and modify their attributes, andinteract with them. Thus, resource management encompasses tracking the state ofresources, allowing only actions consistent with their state, and an API so thatprograms can manipulate them.

Files are a visible example of a common resource. NT provides an API forcreating and opening files; modifying their attributes (hidden, read-only,etc.); and on NTFS, ensuring that programs honor file security settings.Programs expect these capabilities, and NT implements its resource management(i.e., the way NT efficiently keeps track of a file's state) hidden fromapplications. The operating system does the work of tracking the state andprotecting resources.

In NT, the typical way a program accesses a resource such as a file is toopen or create the resource and then manipulate it. Usually, a resource isassigned a name when it's created so that programs can share it. To look up oropen an existing shared resource or a global resource (e.g., a file system,disk, or other physical device attached to the system), a program specifies theresource's name. However, programs often create unnamed resources, whichtypically are logical resources (e.g., synchronization primitives), that anapplication will privately use.

Regardless of whether resources are physical resources (such as disk drivesand keyboards) or logical resources (such as files and shared virtual memory),NT represents them as object data structures, which the Object Managerdefines. Objects are shells that other NT Executive subsystems can fill in sothat they can build custom object types to represent the resources they manage.The Object Manager tracks information that is independent of the type ofresource an object represents; the subsystem-specific core of an object containsdata relevant to a particular resource.

The reason NT builds objects using the Object Manager's infrastructure issimple: Each subsystem does not need to reinvent the wheel to track thesystem-related state of an object, look up its resources, charge applicationsfor the memory required to allocate an object, and protect resources withsecurity. By concentrating these functions in the Object Manager, Microsoft canshare code across subsystems, write and validate NT's security code once, andapply the same naming conventions to all resources.

Object Types
The Object Manager's duties involve creating object types, creating anddeleting objects, querying and setting an object's attributes, and locatingobjects. As I mentioned previously, Executive subsystems (including the ObjectManager) create objects that represent the resources they manage. Before asubsystem (e.g., the I/O Manager) can tell the Object Manager to make an object(e.g., a file), the Object Manager must define the underlying object typethat the subsystem will instantiate the object from. Object types, like classesin C++ parlance, store information common to all objects representing the sametype of resource. This information includes statistical information and pointersto the method procedures that the subsystems will invoke when they performactions on an object of the corresponding type. Thus, when the Executivesubsystems initialize, they call a function in the Object Manager to definetheir object types.

For example, open files require resource management, and the I/O Managersubsystem builds file objects (with help from the Object Manager) to track openfiles. Inside the file object's body, the I/O Manager keeps track of the file'sname, its logical volume, and whether the file is marked for deletion; the fileobject also provides storage for private data associated with the file systemdriver that the file belongs to.

Other examples of subsystem object types include process objects, whichrepresent active processes; section objects, which represent memory-mapped filesor shared memory; and device objects, which represent physical or logicaldevices. Table 1 summarizes the 23 object types NT 4.0 defines.

As with objects, NT implements object types as data structures. Figure 1shows typical information an object type data structure stores. The objecttype's statistical data is useful for system monitoring. This data includesinformation such as the name of the object type, how many objects of that typecurrently exist, the maximum number of objects that have existed at any time,and the default amount of memory that NT charges a process each time thesubsystem creates an object of that type.

Object type procedures or methods are what really differentiate objecttypes. When a subsystem creates an object type, the subsystem passes to theObject Manager a data structure that contains pointers to all the object typeprocedures. The Object Manager calls these procedures when the subsystemrequires actions performed on an object. For example, if a subsystem wants toclose an object, the Object Manager first calls the Okay-To-Close Procedure (ifthe subsystem has specified one). If that procedure returns a FALSE value, anerror returns to the closer to signal that the object cannot be closed. If theprocedure returns a TRUE value, the Object Manager then calls the CloseProcedure so that the subsystem responsible for the object can clean up theobject's state. Each object type procedure receives a predefined list ofparameters that includes information pertinent to the requested operation. Thecall-out mechanism the Object Manager uses for object types lets subsystems seeand control the actions taken on the objects (and therefore, the resources) thatthey manage.

I'll describe the purpose of most of the object type procedures throughoutthe rest of the article, but I'll comment on two unrelated procedures thatappear in Figure 1 now. The Dump Procedure is not defined for any object type,nor does NT ever reference it. Microsoft developers probably relied on dumpprocedures in the early days of NT, but they removed the code after they had thebasic object building blocks in place and working. The Security Procedure letssubsystems implement object security schemes that differ from NT's defaultsecurity policies. NT falls back on its default security policies if a subsystemdoes not define a Security Procedure.

Objects
When a subsystem directs the Object Manager to create an object, thesubsystem passes the Object Manager a pointer to an object type, which serves asthe connection to the object type's global data and procedures. Other parametersinclude an optional name and security information to be applied to the newobject, the size of the subsystem-specific body of the object, and pool chargesthat can override the default charges in the object type.

To the subsystem creating the new object, the Object Manager returns apointer to the body of the new object. In this body area, subsystems can storedata that tracks the state of the resource the object represents. Preceding thisbody (effectively private to the Object Manager) is an object header, whichFigure 2 illustrates. The object header stores the name of the object, theparameters passed to the creation function, the object's security attributes,and a pointer to the object type.

Two fields in the object header, Handle Count and Reference Count, trackdifferent kinds of references to the object. Handle Count refers to the numberof times applications have opened the object. When a program opens or creates aresource, NT's APIs return a special value, or handle, that the programcan use to refer to the open resource. APIs that manipulate a resource's stateuse the resource's handle in lieu of its name; thus, handles provide aconvenient way to refer to open resources.

Although handles are opaque values to applications, handles referenceentries in a process' handle table (also shown in Figure 2). A handletable is a dynamically managed array (i.e., no hard upper ceiling exists for howlarge the array can become) that the Object Manager indexes via handles tolocate the objects the handles refer to. Handles are process specific, so twoprocesses can have different handle values for the same open resource.

When a program closes an object, the Object Manager calls the object type'sClose Procedure and passes it the object's handle count. One example of asubsystem that monitors object handle counts is the I/O Manager, which notifiesfile systems when it closes all handles for a file object so that the filesystem can perform necessary cleanup operations. At that time, a file systemwill delete a file marked for deletion, because no application is using it.

The second field tracked in object headers, Reference Count, is the totalnumber of references to an object. Operating system components can reference orcreate objects without going through the NT API, and consequently, do notrequire handles. Reference Count records the number of handles for an objectplus the number of active references that operating system components make tothe object. The Object Manager uses this count to determine when the system nolonger needs an object. When Reference Count drops to zero, nothing in thesystem is using the object, so the system can remove the object's state andstorage. The Object Manager will call an object type's Delete Procedure (whicheliminates the object, not the resource the object represents) with the objectas a parameter.

Locating Objects
Up to this point, I've avoided the details about how applications open anobject by specifying the object's name. Every NT object that has a name liveswithin the Object Manager namespace. This namespace, which is very much like thefamiliar file system namespace, consists of directories that containsubdirectories or objects. In fact, you enter the file system namespace (withnames like C:tempfile.txt) and the Registry namespace (with names likeRegistryMachine) via the Object Manager namespace. First, let's look at theObject Manager namespace, and then we'll look at how NT embeds alternativenamespaces within it.

If you study Table 1, you'll see the Object Manager's Directoryand SymbolicLink object types. NT uses these object types to define the ObjectManager namespace. The optional name given to an object when it's createdlocates the object within the namespace. When NT is initializing, varioussubsystems create directories in the Object Manager namespace. The I/O Managercreates a device subdirectory that it will use to store named device objects,and the Object Manager creates a subdirectory called ??. The ?? subdirectoryholds objects accessible via the Win32 API. Thus, any Object Manager resourcereferenced from Win32 must have a corresponding named object in thissubdirectory.

For example, serial ports named COM1, COM2, and so forth in Win32 haveobjects with those names in the ?? subdirectory. You'll also find C: and otherdrive letters in this directory. The objects with those names are symbolic linkobjects, which point (with alternative names) at objects elsewhere in thenamespace. Drive letters point to the device subdirectory at device objectsthat have names associated with the hard disk partitions they reside on. Forexample, C: might point at deviceharddisk0partition1.

An object type's Parse Procedure is what lets NT connect alternativenamespaces to the Object Manager namespace. When the Object Manager performsa name lookup and encounters an object, the Object Manager checks to seewhether the object's type has a Parse Procedure, and calls it. The subsystemmanaging the object type can then take the remaining portion of the name andperform a lookup within the subsystem's namespace.

The same sequence of events happens when you open C:tempfile.txt from aWin32 program. First, Win32 translates the name to ??C:tempfile.txt. Next,NT calls the Object Manager's name-parsing routine, which locates the C:symbolic link object in the ?? directory. The Object Manager then looks updeviceharddisk0partition1, which the symbolic link points to, and finds adevice object. The Object Manager passes the rest of the name, tempfile.txt,to the I/O Manager's device object type Parse Procedure, which locates the filesystem responsible for C: and hands it the name.

You enter the Registry namespace similarly via the Registry key objecttype Parse Procedure. Figure 3 presents a simplified depiction of these threenamespaces and how they are connected. (In Figure 3, HKLM stands for HKEY_CU stands for HKEY_CURRENT_USER.)

Most Win32 programmers and systems administrators don't know about theObject Manager namespace because they don't need to know about it to open filesand Registry keys. However, you can use native NT system services to obtaininformation about what's in the namespace. With the Win32 software developmentkit (SDK), Microsoft provides the WinObj tool, which will display the namespaceas if you were browsing with Explorer. Unfortunately, the WinObj tool hasseveral significant bugs that cause it to display incorrect information (e.g.,inaccurate handle and reference counts).

Another version of the WinObj tool, which you can get at http://www.ntinternals.com/winobj.htm,doesn't suffer from the same problems as Microsoft'sWinObj tool, and it displays additional information about certain object types.Screen 1 shows the view of the Object Manager's ?? directory that thisalternative WinObj tool displays. One subdirectory worth noting is theObjectTypes subdirectory, which contains all the defined object types.

A Little Knowledge Goes a Long Way
Although you can get along just fine managing or programming NT withoutknowing about the Object Manager, some familiarity with it is useful. Forexample, using the Control Panel's Ports applet, you unfortunately can direct NTto create invalid serial ports. The WinObj tool lets you look in the ??subdirectory of the Object Manager namespace for COM objects and determine whichserial ports really exist. Even if you don't run into such problems, knowledgeof NT object management can give you a better understanding of NT's architecture and the Win32 API.