Directory Integration and the Metadirectory

Tools to help you streamline directory administration

Directories are fundamental to network services. Directories store and publish user and network resource information. They contain user accounts for network OSs, messaging systems, and applications, and network configuration information for computers, printers, routers, and corporate networks' security policies. Forrester Research and GartnerGroup concluded in separate studies that the typical Fortune 1000 company maintains an average of 181 directories. Managing so many directories is difficult. Network administrators must duplicate their efforts to create, modify, and remove directory information in multiple directories to maintain the directories' accuracy when changes occur. This work produces significant costs for network administration. According to the Burton Group, a 25,000-user company can spend $364,583 annually on directory changes if the company has only seven user directories.

How do you reduce the cost of directory administration when you can't eliminate existing directories or avoid creating new directories? The only answer is directory integration. Directory integration lets network administrators manage directory information from one directory and automate the process of changing information in multiple directories. In the short run, directory integration lowers the cost of directory management because it reduces human involvement in directory management. A comprehensive directory-integration system often requires an enterprise directory to store and unify directory information in a central repository, or metadirectory. In the long run, you can incorporate into a metadirectory new network services—for example, single sign-on (SSO), to simplify the user logon process; public key infrastructure (PKI), to manage digital certificates for e-commerce; and Directory Enabled Networks (DEN), to deploy Quality of Service (QoS) and manage network resources. Also, new directory-enabled applications can take advantage of existing metadirectories and help you avoid adding new user directories to your network. (For more information about metadirectories and directory integration, see "Related Articles in Windows NT Magazine," page 101.)

Directory integration has become a high priority in today's network management. In fact, many companies are deploying or planning enterprise directories not only for directory integration but also for e-commerce projects. The four techniques of directory integration—virtual directory, synchronization, join, and information broker—can meet these enterprise needs. In this article, I discuss the four directory-integration techniques and their roles in the metadirectory. I also describe metadirectory architecture and two metadirectory types. With a clear understanding of directory-integration techniques and the metadirectory, you can easily characterize third-party directory-integration solutions by their implementation methods and choose the product that meets your company's present and future needs. The sidebar "Password Synchronization and Security" discusses password synchronization in directory integration.

Virtual Directory
A virtual directory is an administrative tool that lets you manage multiple directories from one console. The console's interface is linked to managed directories in the network. You can manage directory information from the console.

A virtual directory that many NT administrators know is Entevo's DirectAdmin. DirectAdmin lets you centrally manage NT domains. You install DirectAdmin on an NT computer from which you can map NT 4.0's flat domain structure to an Active Directory (AD)-like hierarchical tree structure, delegate individual administrative tasks, and move users from one domain to another. With the DirectAdmin NDS Plus Pack, you can manage NT and Novell Directory Services (NDS) from one console. Other cross-platform virtual directories include Computer Associates' Unicenter TNG Directory Management Option (DMO) and IBM's Tivoli User Administration. DMO lets you manage messaging and database directories in addition to various network operating system (NOS) directories.

Microsoft provides a directory development service, Active Directory Services Interface (ADSI), that helps independent software vendors (ISVs) develop virtual directory tools to manage different vendors' directories. For example, Entevo used ADSI to develop DirectAdmin products. ADSI is an open set of COM programming interfaces for directory services. (A similar product to ADSI is OLE DB for relational databases.) Applications that developers write to ADSI can work with any directory service that offers an ADSI service provider. Applications (e.g., a virtual directory tool) can access any directories (e.g., NT 4.0 domain directory, NDS, Lotus Notes, and Lightweight Directory Access Protocol—LDAP—enabled directories) through a corresponding directory's ADSI interface and service provider, as Figure 1 shows.

The virtual directory, however, is purely a client solution for administrative use. (Virtual directories are sometimes called virtual directory administration clients). A virtual directory is not a serverside directory service that users access for network logon and name lookup. Virtual directories generally don't include complicated management services such as directory synchronization. Although some vendors offer both virtual directory and synchronization functions in their products (e.g., DirectAdmin NDS Plus Pack), synchronization functions often run on servers.

Synchronization
Directory synchronization plays an important role in directory integration by ensuring, through an automatic update process, that directory information is consistent across directories. Directory synchronization populates information from one directory to another. Directory synchronization differs from directory replication, which uses an identical format and schema to replicate data across similar directories or data stores. Synchronization can provide simple data translation. A synchronization event occurs either when the synchronization service finds a change in a directory or at a predefined time. An early implementation of directory synchronization was the directory synchronization between different email systems that products such as Lotus' Soft-Switch provided. During the past 3 years, many vendors delivered synchronization tools to synchronize directories between NOSs, email systems, and applications.

Two synchronization methods currently exist: one-to-one and one-to-many. In one-to-one synchronization, the synchronization service synchronizes only two directories to provide one-way or two-way synchronization. One-way synchronization always populates information from one directory to the other. For example, Microsoft offers Directory Service Manager for NetWare (DSMN) in NT. In a mixed NT and Novell NetWare environment, you can use your NT domain as a single point of control to manage user accounts. DSMN feeds changed user account information from the NT domain to a NetWare Bindery directory. Microsoft delivers a similar directory synchronization service for NDS in Windows 2000 (Win2K). Novell's retired Workstation Manager implemented a similar function but used NDS as a single point of control. (Novell replaced Workstation Manager with NDS for NT. NDS for NT uses NDS to substitute for the NT domain SAM, which doesn't require synchronization.)

In contrast to one-way synchronization, two-way synchronization lets two directories synchronize each other. For example, Netscape's Directory Server includes a two-way directory synchronization service for NT. The utility replicates changed user information in NT to Directory Server and feeds changed user information in Directory Server to NT. Two-way synchronization allows flexible directory management and can head off political conflicts if directory administration must be distributed throughout an organization.

The second synchronization method, one-to-many, is more comprehensive than one-to-one synchronization and scales beyond two directories. In one-to-many synchronization, you need to designate one directory as an enterprise directory and inject information from other directories into this centralized repository, or distribute information from the enterprise directory to other directories. Figure 2, page 102, depicts three possible directory relationships in one-to-many synchronization. NetVision's Synchronicity uses NDS as an enterprise directory. Synchronicity can synchronize NDS with NT, Microsoft Exchange Server, Lotus Notes, and NetWare Bindery.

Both the one-to-one and one-to-many synchronization methods use a straightforward duplication process to populate directory information. Both methods also often require unique usernames in all connected directories. However, in a large network with many directories, users often have multiple usernames if the organization doesn't enforce a unique-username convention. If user John's NT username is John but his NDS username is Johnny, synchronization will simply create a new Johnny account in NT when replicating accounts from NDS to NT. Some well-designed synchronization tools, such as Synchronicity, can give you options in this situation: for example, replacing an existing object, creating a new object, or generating an error. Alternatively, some synchronization systems offer a limited function to match attributes other than username. For example, if you define a unified employee ID in a user object across the directories you want to synchronize, you can choose this attribute as an alternative match to usernames. This method leads us to the join concept.

Join
The join technique of directory integration works with the synchronization technique to create one metadirectory and link user objects from different directories to the metadirectory. Join makes directory integration possible in a complex environment. Suppose employee John's NT username is Johnny, his NDS username is John, and his human resources (HR) ID is JSmith. Join can associate John's three sets of user attributes from the three directories in which those attributes reside with John's user object in the metadirectory. The metadirectory can hold all of its underlying directories' user attributes or selected user attributes from those directories. Figure 3 shows how the join technique links multiple directories to the metadirectory.

In addition to linking multiple user objects to one user object, join lets you associate and synchronize specific object attributes in a connected directory with specific attributes of the corresponding object in the metadirectory. Also, you can manage a directory population's information flow with join. For example, you can synchronize John's HR attributes—such as his full name, employee number, department, phone number, and office address—with John's corresponding attributes in the metadirectory, feeding the attributes to the metadirectory from the HR database. Also, you can replicate some of John's metadirectory attributes—such as his full name and description—from the metadirectory to his NT directory, and other attributes—such as his full name and department—to his NDS directory. A successful join requires the metadirectory and all underlying directories to remain active during the join process.

When you join two objects, you need to identify a common attribute in the two objects to serve as the join point. Network administrators often use attributes such as full name, employee number, and social security number as common attributes. Metadirectory products such as ISOCOR's MetaConnect and ZOOMIT's VIA provide an easy-to-use GUI and scripting tools to set up rules for automatic joins under a common attribute in objects in a metadirectory and an underlying directory. Joining two objects with no common attribute is a tedious manual process.

The metadirectory that the join technique creates can serve as a companywide central point of directory access. Network administrators can manage user information in the metadirectory and then feed any part of the user information to underlying directories. Conversely, administrators managing local directories can replicate selected user information (e.g., all user attributes without salary information) from the local directories to the metadirectory. Network users can use the metadirectory to search for information, such as an employee's phone number, instead of using the phone book.

Information Broker
Collecting all network information in one place can make metadirectories overweight. To avoid this problem, you can keep some information locally in underlying directories and make the information available to the metadirectory only when the metadirectory needs the information. The information broker technique lets metadirectories locate information that underlying directories store, so the metadirectory isn't forced to store information it doesn't need. This technique can save some replication and synchronization between directories.

Information broker is a relatively new term in directory integration. The information broker concept, however, is similar to chaining in the X.500 directory standard. X.500's chaining function lets one X.500 directory forward a directory search request to another X.500 directory if the first directory can't find the requested information locally. The second directory returns the result if it has the information. Otherwise, the second directory sends, or chains, the original request to a third directory. In a metadirectory, chaining is termed static brokering. Figure 4, page 104, shows how static brokering works. A client sends a request for a specific user's work-hour information to the metadirectory. Because the metadirectory doesn't keep user work hours, it chains, or brokers, the client's request to an HR database. The HR database returns the search result to the metadirectory, which returns the result to the client.

Another kind of information broker is dynamic broker. A dynamic broker-capable directory can support realtime functions. For example, a change in such a directory can trigger specific operations, such as network policy changes. Figure 5 shows how dynamic broker works in Cisco Networking Services for Active Directory (CNS/AD), which implements CiscoAssure Policy Networking using Win2K's AD and DEN for Cisco equipment. When the HR department adds a new employee to its database, the database propagates the new employee information to CNS/AD. This change in CNS/AD triggers the policy server to create a policy profile for the new user (e.g., how much bandwidth the user can use). The policy server can update AD with the new user profile and send the profile policies to network equipment such as routers and switches.

Information broker can cause problems when you use it in the metadirectory. If enough information doesn't physically reside in the metadirectory, information broker will degrade search performance by requiring the metadirectory to find the information that information broker requires in underlying directories. A more practical use of information broker is to keep frequently changed information in local underlying directories (e.g., keep contract employees' work hours in an HR database).

Metadirectory Architecture
A metadirectory integrates multiple directories using the techniques of synchronization, join, and information broker, and provides a unified directory from which administrators can name and access user and network resources throughout an enterprise. A metadirectory consists of three parts: the metadirectory namespace, a join engine, and connectors. Figure 6 illustrates typical metadirectory architecture.

The metadirectory namespace is a unified view of all managed underlying directories. The namespace contains physical directory information that synchronizes and joins with corresponding information in underlying directories, and logical links that broker information in the underlying directories. Metadirectories generally use an LDAP-compliant directory to store their namespace.

The join engine implements the directory join function, which synchronizes the metadirectory namespace and underlying directories at the object attribute level. The join engine is often a service running on the metadirectory server and provides a GUI tool for join configuration.

A metadirectory's connectors import information from specific underlying directories and export metadirectory information to the underlying directories for the join engine. A connector is often a service running on the metadirectory server or the underlying directory server. To communicate with underlying directories, a connector uses a protocol that the underlying directories can understand. For example, a connector might use LDAP to talk to an LDAP-compliant directory, SQL to talk to a relational database, a messaging protocol to talk to an email directory, a proprietary protocol to talk to a proprietary directory such as an NT domain directory, or a programmable script to talk to a specific directory.

A connector can replicate a complete copy of a managed underlying directory into its namespace. Because the connector can replicate underlying directories, the join engine can communicate with the connector's namespace via a standard protocol (usually LDAP) instead of with a proprietary protocol to join attributes in the metadirectory and connector namespaces. This communication eases join engine implementation by moving the customization task to the connector for a specific underlying directory. For example, by using a connector specifically developed for SAP, a metadirectory can easily manage an SAP user directory that doesn't support LDAP. After the connector retrieves SAP user information into its namespace by using SAP's proprietary APIs, the join engine can use LDAP to communicate with the connector namespace without having to understand SAP's language. If an underlying directory supports LDAP, however, the join engine can directly communicate with the underlying directory, so the connector doesn't always need to duplicate the LDAP information into its namespace.

Two Metadirectory Types
Major directory vendors, including Control Data, IBM, ICL, ISOCOR, Microsoft, Netscape, Novell, Siemens, and ZOOMIT, have embraced the metadirectory concept. Among this group, Control Data, ISOCOR, and ZOOMIT have delivered full-featured metadirectory products. Although these products follow the typical metadirectory architecture that Figure 6 illustrates, two distinct metadirectory types exist. One type is the dedicated metadirectory; the other type is the add-on metadirectory.

The dedicated metadirectory contains a directory store to host the metadirectory and connector namespaces. When you deploy a dedicated metadirectory, you must therefore add another directory to your network. Control Data's Rialto Global Directory/Meta Edition and ZOOMIT's VIA are typical dedicated metadirectory products. Control Data uses its Global Directory Server as the metadirectory store. ZOOMIT developed a proprietary metadirectory store as part of VIA.

The add-on metadirectory also includes a directory store but uses a third-party directory to fulfill this function. Add-on metadirectories give you the flexibility to choose an enterprise directory based on your company strategy. For example, if you choose AD as your strategic directory, you can use an add-on metadirectory product to position AD to be your future metadirectory platform. If you rely on NDS for directory services across NOSs, you can apply an add-on product to NDS and use NDS as your metadirectory. ISOCOR's MetaConnect is an add-on solution that can use ISOCOR's Global Directory Server, Microsoft's AD, and Netscape's Directory Server as a directory store. MetaConnect will support Novell's NDS in the future, according to ISOCOR.

The add-on metadirectory appears more attractive than the dedicated metadirectory because add-on metadirectories let you choose your enterprise directory as your metadirectory and don't require you to install an additional directory. However, the add-on metadirectory has shortcomings. Add-on metadirectory vendors are limited in developing products, because the vendors must base their add-on metadirectory products on third-party directories. A third-party directory might not support specific functions that the add-on metadirectory vendor wants its product to include. In addition, third-party directories might not be able to provide desired performance when handling large amounts of directory-update data for the add-on product. In contrast, a dedicated metadirectory vendor can develop without restrictions a directory store based on its own and its customers' requirements.

Microsoft Support for the Metadirectory
Microsoft will deliver AD, the company's scalable directory service, in Win2K. Can AD be a metadirectory? Out of the box, AD is not a metadirectory, although Win2K will include a directory synchronization service for NDS and an Exchange Server connector to synchronize AD and Exchange Server directories. However, you can use AD as a metadirectory store and turn AD into a metadirectory by applying an add-on metadirectory product such as MetaConnect.

As directory integration becomes increasingly important in the enterprise, Microsoft has positioned AD as a metadirectory platform. Using AD as a metadirectory platform involves three major efforts: developing AD connectors for other directories; working with other directory vendors, such as ISOCOR and ZOOMIT, to develop directory-integration products on an AD platform; and working with software and application vendors such as Baan and SAP to develop AD-enabled applications.

In addition to developing ADSI for directory integration, Microsoft submitted an Internet-draft to the Internet Engineering Task Force (IETF) in March (http://www.ietf.org/internet-drafts/draftarmijo-ldap-dirsync-00.txt). This specification defined how to use an LDAP-based control to synchronize information between heterogeneous directories. Doing so lets a client request changes to a directory and can capture directory changes at the attribute level. Microsoft calls this control DirSync and has used DirSync to implement the company's directory synchronization service for NDS in AD. Several directory vendors, including ISOCOR, NetVision, and ZOOMIT, plan to use DirSync in their future directory-integration and metadirectory products. For a listing of current products, see "Directory Products."

Where Are Directories Headed?
Two years ago, many vendors accepted LDAP as a standard directory access protocol, began to add LDAP support in their products, and delivered LDAP-compliant directories. Many people once thought that LDAP would quickly become the only tool required for directory integration and would thus remove the need for metadirectories. However, LDAP is not a panacea to integrate legacy directories that don't support LDAP. LDAP also lacks key standards for directory replication, access control, and schema. Because of proprietary replication functions that vendors' directories implement, one vendor's LDAP directory can't natively replicate to and synchronize with another vendor's LDAP directory. The IETF's lightweight directory update protocol (LDUP) workgroup is working hard to standardize these important features in LDAP as extensions. Several years of work are required, however, before all directories can support LDAP and its new extensions, as well as LDUP. The metadirectory will continue to play a crucial role in integrating legacy and LDAP directories with synchronization, join, and information broker techniques.

If you spend time updating user information in multiple directories, consider incorporating a metadirectory into your network service infrastructure. When you start to plan and deploy a metadirectory, consider the following six-step strategy that many companies are using. First, choose an LDAP-compliant metadirectory that fits your long-term IS strategy. Second, use your chosen metadirectory as much as possible as a central point of directory management and access, and as a central repository for new network resources such as PKI, IP address management, QoS, and DEN. Third, consolidate information in the metadirectory (e.g., eliminate paper phone books). Fourth, add only LDAP-compliant directories to your network when you have to add new directories. Fifth, to reduce the number of new directories you must add, ask for LDAP-enabled applications from vendors and internal developers. Sixth, ensure that your metadirectory solution matches your security policy. If you don't have a security policy, create one to match your business and legal requirements.