Chariot 4.0

Predict performance of network applications

NetIQ’s Chariot 4.0 network-performance-testing software provides a set of tools that emulate specific types of application traffic and help network designers and administrators predict how their networks will perform under various workloads. The software gathers and calculates standard performance measurements (e.g., average throughput, response time, transaction rate) and displays the measurements in realtime. Chariot can help you determine whether your network will be able to handle the workload for a new application and make any necessary adjustments before you deploy the application.

Application scripts emulate various types of enterprise transactions (e.g., Active Directory—AD—transactions, file transfers, database updates) between endpoints. Application scripts tell the endpoints the type and quantity of data to send and receive, when to connect and disconnect, and how to mimic application and user behavior.

My test network used a Boca Research 10Base-T hub and an Intel PRO/Wireless 2011 LAN Access Point wireless hub. I used Ethernet cables to connect Windows 2000 Professional and Linux machines and the wireless hub directly to the 10Base-T hub. I connected a Win2K Server system to the hub through an Intel PRO/Wireless LAN PC Card. I loaded the console on an IBM ThinkPad 560X with 64MB of RAM running Win2K Pro. You can install the console on Win2K, Windows NT, Windows Me, and Windows 9x systems.

I installed endpoints on a Pentium 120 running Red Hat Linux 6.2, a Pentium II 266 running Win2K Server, and the console computer. Endpoint installation was straightforward and well documented for each OS. You must install the endpoint software manually, although you can automate subsequent updates to the software. Because Chariot can run tests between thousands of endpoint pairs, endpoint deployment would be easier if Chariot supported automatic centralized endpoint installation. NetIQ provides instructions for using Microsoft Systems Management Server (SMS) to deploy endpoints, but you can use SMS only on Windows machines.

After I installed the endpoints, I designed a test from the console’s main window. New to Chariot 4.0 is the Visual Test Designer feature, which lets you design tests graphically. As Figure 1 shows, you can drag objects that represent endpoints, endpoint groups, and connections to quickly create a network performance test. However, Visual Test Designer lacks some of the basic functionality of a typical graphical design program, such as the ability to use captions to make comments about the objects in the diagram.

Next, I assigned application scripts. After endpoints have copies of the assigned script, they can begin transferring the data load that the script defines. I used scripts that emulate traffic from Microsoft Exchange Server, Microsoft SQL Server, Win2K Server Terminal Services, and copying and printing a file. For three endpoints, I had my Linux server emulate a Terminal Services server, a SQL Server system, and a print server. (I was a UNIX administrator in a previous life, so having a Linux server role-play SQL Server was too enticing to pass up.) The console machine played the role of an Exchange Server and file server. Figure 2 shows how Chariot presented the results of my test. Chariot can be very useful in determining how combinations of different types of traffic can affect network performance. For example, notice how the Throughput graph in Figure 2 shows that Pair 2 increases performance the moment Pair 4 finishes, at just past 10 seconds.

For another series of tests, I added a wireless connection to the network for another endpoint. When I tried to initialize the Win2K Server system as endpoint 1, I immediately received an error message. The generic error message read Unknown TCP connection error but provided an error number. I thought the wireless connection was the problem, so I went through the usual steps of troubleshooting the IP network with the Ping command. Everything checked out fine. The error message led me to a Help page that said I should call Customer Care. When I called, a support person answered after two rings. Within 15 minutes, we determined that the test failed because port 10115, which is the port Chariot uses to initialize endpoint 1, wasn’t accessible. We found that the culprit was my firewall client software, which was enabled on my server even though my test network didn’t have a firewall. I disabled the firewall client, then proceeded successfully.

Chariot showed me that my wireless LAN wasn’t transmitting at the throughput I expected. I tinkered with my wireless hub’s configuration and achieved a higher sustained throughput and faster response times. The ability to help you detect network bottlenecks is one of Chariot’s benefits. Chariot includes a built-in comparison program that lets you compare the results of multiple tests. And you can run the software from the command line to schedule jobs at off-peak hours.

One shortcoming of Chariot became evident as I studied the results of my tests. NetIQ recommends testing on a live production network if you’re planning to deploy a new application so that you can see the impact of your usual network traffic on the new application, although you probably don’t want to conduct stress tests on a production network during business hours. My small test network had little traffic to compete with Chariot’s test. But on a large, busy production network, Chariot wouldn’t be able to reveal the extent to which usual production traffic was competing for bandwidth with the application Chariot is testing. Chariot doesn’t include a network tool to gauge bandwidth utilization and to detect other types of traffic and determine its affect on throughput. Another NetIQ product, Application Scanner, can listen to a live network and generate custom application scripts that reflect the network’s traffic. You can load those scripts into Chariot on a test network to simulate the traffic on your production network, resulting in a more realistic performance measurement for the new application you’re testing. Application Scanner costs $4500. NetIQ also offers Pegasus, a product that provides network-performance-testing and can monitor traffic; Pegasus starts at $25,000.

Chariot is well designed and well documented; I was instantly impressed with the volume of printed documentation that accompanies the product CD-ROM. The online documentation mirrors the printed information. However, manually installing the endpoint software on very large networks is time-consuming. In addition, until Chariot’s base package includes network monitoring and a way to generate scripts from a production network, the product’s $14,000 price is hard to justify.