MSMQ to the Rescue

Increase the reliability of distributed applications

Imagine you are a Windows NT systems administrator who is responsible for ensuring the continued functionality of a Web application running on a Web server. Suppose the application lets customers browse and purchase products. The application must gather product information from a products database and save customer orders in an orders database on a separate database server. As customers select products, the application writes orders to a back-end database so the order-fulfillment department can process the orders.

Consider what would happen if the disk array on the orders database server malfunctioned, as Figure 1, page 140, shows. You would have to take the orders database server offline to replace the disk array. Thus, the application would be temporarily unusable. Disabling an application even temporarily is costly. You might lose dozens of orders and jeopardize your position in the company.

Microsoft Message Queue Server (MSMQ) and IBM's MQSeries for Windows NT provide a solution and let you keep applications online while you perform system maintenance. These technologies use Message-Oriented Middleware (MOM) to help distributed application components work together. MOM is a class of application that provides custom applications with a set of APIs. These APIs hide the underlying complexities involved in making asynchronous connections in interprocess communications between systems over multiple protocols. A benefit of MOM-type applications is guaranteed message delivery.

MSMQ competes against MQSeries in the MOM class of applications. Both products provide reliable message-queuing functionality. Differences exist in price, proprietary system commands, interface, and supported platforms. MSMQ runs only on NT, Windows 98, and Win95. MQSeries runs on platforms other than Windows, including MVS, OS/2, DOS, and several versions of UNIX. (For a complete list of supported platforms, go to http://www.software.ibm.com/ts/mqseries/platforms.) MSMQ strengthens NT's position as an enterprise OS and helps it effectively challenge UNIX (which has had MOM functionality for several years in supporting mission-critical applications). In this article, I discuss MSMQ's advantages. (For more information about MSMQ, see Tao Zhou, "Queuing Data with Microsoft Message Queue Server," October 1998.)

How MSMQ Works
When processing occurs on a typical network, the network and the applications involved in the processing must be online and functioning properly. If the application sending data or receiving data is offline, processing might stop. MSMQ is a fast store-and-forward service that lets applications running at different times communicate across heterogeneous networks and systems that might be temporarily offline. An application can use MSMQ's store-and-forward functionality to continue accepting orders even if the back-end database server goes down. Applications send messages to MSMQ, and MSMQ stores the messages in queues for later delivery. A message is a unit of data that one computer sends to another. Messages can be text or binary, and they have a maximum size of 4MB. When the order-storing portion of the system comes back online, MSMQ can forward messages from the message queues. Applications and application components communicate through message queues rather than directly.

MSMQ's functionality is helpful for users who connect to the network via laptops. Network connections are temporary, and you typically cannot use network applications unless you are connected. MSMQ lets you use network applications even when you are not connected to the network. MSMQ stores information in local queues until you return to the office. Then, MSMQ forwards the data to applications as necessary. With MSMQ, processing is time-independent. You do not have to be connected to the network to use a network application.

MSMQ's Role in RPC, MTS, and MS DTC
MSMQ's functionality is similar to remote procedure call (RPC) functionality: MSMQ and RPC help enable distributed applications. A difference is that MSMQ applications communicate indirectly through message queues, whereas RPC establishes a process-to-process connection to send messages. Using MSMQ, applications send messages to queues, then continue with other tasks without having to wait for a reply from the receiving application. MSMQ functions asynchronously, storing messages until the receiving application retrieves them. MSMQ lets you build distributed applications without having to deal with multiple protocols. Using RPC, an application can make calls to another application. In this scenario, the calling application typically waits until the called application performs a task and returns a result before the calling application continues running. You typically use MSMQ to build distributed, n-tier applications that work well together, and you use RPC in two-tier client-server applications. You typically do not use MSMQ with RPC.

MSMQ and Microsoft Transaction Server (MTS) work together to provide a reliable system. Applications get MSMQ's send-and-receive functionality inside MTS's transaction support. You maintain data integrity, because MTS commits or aborts MSMQ operations, as well as other transaction resources (e.g., database updates).

MTS and the Microsoft Distributed Transaction Coordinator (MS DTC) work together to provide transaction support to MSMQ through a standards-compliant XA interface. A transaction is a unit of work that you can commit atomically (i.e., on an all-or-nothing basis). MS DTC can coordinate transactions across multiple resource managers on multiple computers. The XA interface is a standard that lets a transaction manager (e.g., MTS) establish a two-phase commitment with a resource manager. Transaction managers create and control transactions. Resource managers are databases, file systems, message-queuing systems, and other objects that provide access to shared resources.

MSMQ System Requirements
MSMQ can perform several roles. Usually, you must install SQL Server and MS DTC on a machine before MSMQ can run. (Some MSMQ client installations do not require SQL Server; I discuss these setups later in the article.) MSMQ typically uses the TCP/IP protocol but also supports IPX/SPX.

SQL Server stores MSMQ configuration information such as computer and queue information, as well as enterprise topology and settings, in a replicated database known as the MSMQ Information Store (MQIS). You must have appropriate SQL Server licenses for MSMQ roles that require SQL Server. These roles include the Primary Enterprise Controller (PEC), Primary Site Controller (PSC), and Backup Site Controller (BSC) machines.

Several versions of SQL Server 6.5 are available, and they all work with MSMQ. The NT Option Pack (which includes MSMQ) has a folder named MSSQL that contains a 120-day evaluation version of SQL Server. Do not use this version on a production machine, because it stops working after 120 days. Developer Studio 97, Enterprise Edition ships with a developer version of SQL Server. It is not time-sensitive, but it limits concurrent users to 15. NT Server, Enterprise Edition (NTS/E) ships with a limited version of SQL Server 6.5. You can use this version with an MSMQ enterprise controller if the MQIS is smaller than 87MB. To install this version of SQL Server, you must be running NTS/E. Visual Studio 6.0, Enterprise Edition will ship with a workstation version of SQL Server 7.0. To get maximum MSMQ functionality, you will need to install the full retail version of SQL Server 6.5.

Windows 2000 (Win2K--­formerly NT 5.0) might end MSMQ's dependence on SQL Server. The Win2K version of MSMQ will probably use the Active Directory (AD) service instead of SQL Server.

Testing Your MSMQ Installation
When you install MSMQ, the installation program installs a command-line sample application that you can use with NT or Win95 systems to test the connectivity between MSMQ machines on a connected network. Also, you can run two instances of an application on one machine.

To use the application, run two instances of the command prompt on one computer or on different computers. In both instances, go to the MSMQ directory (located, by default, at C: program filesmsmq). In one instance, enter

msmqtest -r

to start the sample application in receiving mode. In the other instance, enter

msmqtest -s

to start the MSMQ test (MSMQTest) in send mode.

In the sending window, run MSMQ Explorer to see whether MSMQ created a queue named msmqtest. (You might have to refresh the MSMQ Explorer view to see the queue.) Enter any text at the prompt that appears, and press Enter. The text you enter becomes the message that the second MSMQ instance receives. This message appears in the receiving window. Under normal conditions, the msmqtest queue in MSMQ Explorer has no messages, because the receiving application receives the messages, rather than MSMQ holding the messages in the queue.

In the receiving window, press Ctrl+C to disable the receiving application. In the sending window, continue to send messages. You'll notice that the msmqtest queue in MSMQ Explorer begins acquiring messages. But MSMQ cannot deliver the messages, because you disabled the receiving application. (You might have to refresh the MSMQ Explorer view to see the messages accumulating.) To restart MSMQTest, enter

msmqtest -r

MSMQ then forwards the stored messages from the message queue to the receiving application. These messages disappear from MSMQ Explorer and appear in the receiving window. When you are finished with the receiving application, enter

quit

to close both instances. Then, delete the test queue from MSMQ.

MSMQ Performance
MSMQ is one of Microsoft's fastest, most scalable technologies. Whether you are sending or receiving transactional, express, or recoverable messages, MSMQ's asynchronous functionality gives applications incredible performance. External factors, such as the size of your database, amount of network traffic, and problems with MS DTC, can cause performance to degrade.

I conducted my MSMQ tests on a 233MHz Pentium Pro system with an IDE disk. I used the full version of SQL Server 6.5 with Service Pack 4 (SP4). My network is a 10Mbps Ethernet network running TCP/IP over Category 5 cable.

To evaluate MSMQ's reliability, I ran some simulated failure tests in which I wrote several messages to a queue and then shut down the server. In addition, I used NT's Task Manager to shut down MSMQ-related services. Then, I shut down and restarted the server. When MSMQ restarted, it restored to its previous state and delivered the messages to their intended recipients.

To test MSMQ's performance, I sent 100 transactional messages. Transactional messages are messages that an application running in MTS generates or receives. MSMQ sent each message as a separate transaction, with an average of six transactions per second from MSMQ and MS DTC. This performance level is acceptable for a single-user GUI application but might not be adequate for a controller application. You can add multiple threads to applications to increase performance. Multiple threads increase MSMQ and MS DTC's throughput to more than 100 transactional messages per second.

Using internal transactions to send messages greatly improves performance. MSMQ can use one thread to send 170 messages per second. However, additional threads cannot handle 170 messages per second for each thread. With five threads, you can achieve a total of 450 messages per second (90 per thread). Performance does not increase symmetrically, because of the overhead associated with creating and maintaining threads.

MSMQ can send messages as express or recoverable. Express messages use fewer resources than recoverable messages use and are therefore faster. The trade-off is that you cannot recover express messages if the computer storing the message files fails. Recoverable messages use more resources and are slower than express messages, but you can recover them if the sending, receiving, or storing computer fails.

When I tested MSMQ recoverable messages, I could send 580 messages per second on one thread and 1320 messages on five threads (264 per thread). When I tested express messages, I could send 9462 messages per second on five threads.

If your tests do not produce comparable results, check your MS DTC installation, or the MS DTC and MSMQ integration. Message size can also affect performance. Sending large messages requires more resources than sending small messages does. Thus, large messages decrease MSMQ's performance.

Uninstalling MSMQ: Troubleshooting
Typically, uninstalling MSMQ from an NT system is easy. However, you might experience difficulty. For example, you might receive the following message: An MSMQ application is running. You must terminate the application before uninstalling. You might receive this message even after you close all running applications.

If you receive this message, you need to determine whether you are running services that might be holding references to MSMQ components. For example, you might be running SNA Server on the system from which you are trying to uninstall MSMQ. If so, you must use the Services applet in Control Panel to terminate all SNA services. Before you disable a service, ensure that the OS does not require the service to function properly. If you are unsure whether the OS requires a service, do not disable the service.

You might receive the error message even after you stop or disable all nonessential services and applications. In this case, you might have to manually delete all MSMQ files, the MQIS SQL Server database, and all MSMQ- and MQIS-related Registry entries.

Using MSMQ with Win95
Many systems administrators must manage NT, Win98, and Win95 systems. Using MSMQ with Win98 and Win95 is somewhat different from using it with NT.

Client installation. MSMQ has two client versions for Win98 and Win95: a dependent client and an independent client. You choose which type of client to install, as Screen 1 shows. When you install MSMQ on Win98 or Win95, this screen appears during the application setup routine. When you install NT Option Pack software on a Win98 or Win95 system, the installation application lets you install only software that runs on Win98 or Win95. The Win98 and Win95 versions let applications send transaction notifications, then continue processing without waiting for confirmation that the transaction has completed.

The difference between clients is where they store the message queue. The dependent client relies on a synchronous connection to an MSMQ server that performs MSMQ functions for the client. Because the dependent client does not maintain internal queues, MSMQ cannot function unless it is connected to an MSMQ server that maintains queues. In contrast, the independent client maintains queues on the Win98 or Win95 system and can operate even when disconnected from the rest of the enterprise network. The independent client consumes more disk space and system resources than the dependent client uses. Thus, you need to consider your Win98 or Win95 system's available disk space and system resources when you decide which version to install.

You can obtain the installation files for both versions of MSMQ from the NTS/E installation program or the NT Option Pack. If you intend to install MSMQ from NTS/E, you need to plan ahead, because the choices you make during the NTS/E installation process affect the MSMQ installation process. During NTS/E installation, select Custom, and choose MSMQ installation. The MSMQ installation procedure creates a shared folder named msmqinst. The system then copies the MSMQ installation files to this folder.

When you install the NT Option Pack on an NT system, the setup application installs Internet Information Server (IIS). When you install the NT Option Pack on a Win98 or Win95 system, the setup application detects the OS you are running and does not let you install IIS. Instead, the setup application installs Microsoft's Personal Web Server. When you install Personal Web Server, you must select Custom, then select the Microsoft Message Queue check box to install MSMQ, as Screen 2 shows. After you choose MSMQ installation, the installation program asks you which type of client to install (i.e., dependent or independent). The program also asks for the MSMQ site controller's name, as Screen 3 shows. Regardless of which client you install, you must have an MSMQ site controller.

Troubleshooting MSMQ problems on Win95. You might run into problems while using MSMQ with Win95. If you have trouble getting a Win95 client to talk to another queue, ensure that your client has Windows Internet Naming Service (WINS) and Domain NameSystem (DNS) configured properly. Also, you need to ensure that your HOSTS file contains the correct entries for your MSMQ server, in case you have a name-resolution problem. Finally, take the following steps on each Win95 client with a connectivity problem.

Your Win95 machine has a directory named WINDOWS that contains a file named HOSTS. If the WINDOWS directory does not contain this file, you can use a text editor to create it. The HOSTS file sometimes has a .sam extension and sometimes has no extension. If the file has an extension, you need to rename the file without an extension. HOSTS is a plain text file that you can use any text editor to open. Listing 1 shows an example HOSTS file.

The HOSTS file contains maps of IP addresses to host names. You need to review the file to ensure that it lists the names and corresponding IP addresses of MSMQ servers to which you want to connect.

Because case matters, you need to add a server several times, using multiple cases, to ensure that users find the correct server whether they use uppercase or lowercase letters. Make entries on separate lines. Enter the IP address in the first column and the corresponding host name in the second column, with at least a space between them. You can add comments to this file to provide additional information about servers or IP addresses. To add a comment, enter it on an individual line or to the right of the machine name. The pound symbol (#) designates a comment, as Listing 1 shows.

If your HOSTS file correctly lists your MSMQ server, your connectivity trouble is not a name resolution problem. To further investigate the problem's source, you must know which machine the problem exists on.

You can use MSMQ's native msmqtest sample application, which I discussed previously, to determine which machine is having the problem. Using this application, you can connect an MSMQ machine to other MSMQ machines, one at a time, to determine which machines have connectivity problems between them. If you are running an independent client on Win95, and you can run msmqtest -r and msmqtest -s on the system, try running msmqtest -r on the Win95 system and msmqtest -s on the PEC. Run msmqtest -r and msmqtest -s on the Win95 client to ensure that you receive all the messages the Win95 msmqtest -s session sent.

When troubleshooting connectivity problems, you need to verify that you have network connectivity. Run the MQPing test from MSMQ Explorer on the PEC to ensure that the Win95 client responds. To run MQPing, open MSMQ Explorer, right-click the computer, and select MQPing from the context menu that appears. To verify proper operation, ensure that the PEC is attempting a TCP connection on port 1801 to the Win95 client.

For additional information about troubleshooting MSMQ, visit Microsoft's MSMQ newsgroups. Go to the msnews.microsoft.com server. (If you are using Microsoft Outlook Express as your news reader, select Tools, Accounts to add this server.) Then, access the following newsgroups.

Using MSMQ with Non-Microsoft Products
You can use third-party software to let MSMQ communicate directly with a mainframe host. Level 8 Systems' (http://www.level8.com/falcon.htm) MSMQ Gateway lets MVS, UNIX, and Unisys machines function as MSMQ clients. If you are using MSMQ and MQSeries, you can use Level 8's FalconMQ bridge to exchange messages between the two systems.

MSMQ: A Hero's Tool
MSMQ benefits NT network administrators. The product offers a variety of features that help NT Server support mission-critical applications and make NT a viable alternative to mainstay OSs such as UNIX. MSMQ guarantees message delivery, thus making network applications bulletproof and maintaining the perception of systems administrators as heroes.