Understanding and Using LoadSim 5.0: Part 2

Install and configure this tool to simulate Exchange Server loads

Last month, in the first part of this series, I explained what LoadSim is and what you can use it for. This month, I will walk you through an example scenario, explaining installation and configuration. In the remaining articles of this four-part series, I'll use this example to show you how to install, configure, and customize LoadSim and how to collect and analyze data.

Using LoadSim
In my example, Microsoft Exchange Server is up and running. But before I deploy it and put real users on it, I want to see how the server holds up against the required user load.

For my test scenario, I assumed two major requirements. First, I wanted to ensure that one server sufficiently handles 300 users that closely match LoadSim's Medium user profile. You definitely want performance to be great, and you want to make sure the server doesn't fall over and die when you turn users loose on it. Second, I wanted response times of 1 second or less. This value is standard, and users generally don't get impatient if response time stays in this range.

My example considers the simplest case, in which you have only one server in the Exchange organization. You can test multiple servers with LoadSim, but I focus on the single-server scenario because it covers the majority of the concepts and parameters you need to know about. When you understand those concepts and parameters, you can then configure LoadSim to do a multiple-server test.

Collecting the User Profile Parameters
Last month, I introduced the user profile parameters: Light, Medium, and Heavy. Table 1, page 162, shows the required parameters for a Medium user, the target for my example. The table shows all the parameters you can customize in LoadSim, so if you want to change some of them for your specific requirements, you'll know what to look for. When you've collected the required data for your test user profile, you can install and configure LoadSim.

Notice that I didn't specify any public folder activity in the profile. LoadSim can simulate public folder tasks, but the focus in this example is configuring for email tasks, which are private folder tasks. Fortunately, configuring for public folders is similar and uses similar principles. Along the way I mention specific public folder options so that they won't be unfamiliar to you.

Installing LoadSim
The installation process has two parts and is easy to do. First, you must install the Microsoft Exchange Client software for NT. This step is crucial because LoadSim cannot talk to Microsoft Exchange Server on its own. It must have the client software. Outlook can also serve as the client software, but LoadSim is not simulating an Outlook client. It is simulating an Exchange client. The Outlook software merely serves as the mechanism for LoadSim to talk to Exchange Server.

Next, you install the LoadSim software. LoadSim has no setup program. You simply copy the files to a directory on the LoadSim client's hard disk. (As I mentioned last month, LoadSim is not on the Exchange Server 5.0 CD-ROM. By the time you read this article, you should be able to download a version of LoadSim with full Messaging API­MAPI­support from Microsoft's Web site; this version is equivalent to the LoadSim shipped with Exchange Server 4.0.) Copy the files to a directory such as LOADSIM on your LoadSim client's hard disk. For convenience, you can make a shortcut for LoadSim in a location of your choice in NT. I usually put the shortcut in the same group as the other Exchange client stuff.

Configuring LoadSim
LoadSim configuration consists of five main steps: configuring the LoadSim test topology properties, configuring distribution lists, configuring LoadSim test properties, creating the LoadSim test topology and importing users into Exchange, and setting LoadSim options. To configure LoadSim, you generally follow the Configuration, Run, and Tools menus in order. Some of the menus have a lot of information, but if you work through them in that order, you won't go wrong. Let's look at each of these five steps.

Step 1: Configuring LoadSim Test Topology Properties
First you configure the topology properties. Topology is a fancy name for specifying the test infrastructure, such as the name of the server under test, public folder configuration, quantity of clients and client protocols, and distribution lists. The topology properties form the basis for the remainder of the test configuration.

From the Configuration menu, select Topology Properties. Screen 1 shows the resulting properties sheet, which is blank the first time you open it. You must add the relevant information about the server to be tested. Click the Add button to bring up the Server Properties sheet with the Type tab displayed. You must enter the information in the Server name, Site, and Organization text boxes exactly as you configured Microsoft Exchange Server, as I did in Screen 2. LoadSim fills in the Internet address text box for you as you type.

If you are setting up for public folder tasks, you need to click the Custom Properties button on the Type tab and enter information in the dialog box. Tell LoadSim how many root folders and nonroot folders you want, how deep you want the folder tree to be, and how many messages you want in each folder. LoadSim will use this information when it seeds the public store with folders when you run Public Folder Initialization.

After you complete the Type tab, you can open the Users tab, which provides a place to configure the number of LoadSim users in the topology. Note that the number of users you specify is not necessarily the number of users this machine will simulate. In this property sheet, you are configuring the properties of the topology for the entire test, not just for this machine. You'll get to configure the number of users for this particular machine shortly. I configured my test to use 300 clients with the Exchange (MAPI) protocol.

After you specify the number of users, click OK to return to the Topology Properties dialog box. Screen 3, page 164, shows the entry you just made. It shows the name of the server, the number of users, the site, and the organization. This information becomes important later, because the number of users you enter here dictates the number of users available when you are configuring the test.

Let me give you a brief description of the Security tab. In this version of LoadSim, Microsoft added the ability to configure which NT user account the LoadSim user will utilize when the user logs on. The default is to use a separate account for each user, because in the real world, each user has an NT user account. Sometimes, using one NT account for all LoadSim users will be convenient, especially in huge test runs, so that you don't have to configure a bunch of NT accounts. I recommend using the default to keep things as realistic as possible.

Step 2: Configuring Distribution Lists
Using distribution lists (DLs) is an optional part of any LoadSim test. But because the example user profile says the user adds a DL to the address list 30 percent of the time, you need to be sure to enable use of DLs. You don't configure the 30 percent usage here; you can just turn DLs on or off and configure their sizes.

In the Topology Properties dialog box, select the Distribution Lists tab. The default is for DLs to be enabled, so leave the Use distribution lists check box selected. You'll also find some customization parameters for DLs here.

The DLs per site entry configures how many distribution lists LoadSim will have in the site. Here, the default is 30. A given recipient can be in one or more DLs.

The last three entries—Minimum DL size, Average DL size, and Maximum DL size—configure how many users are on each DL. You can specify whatever you want, as long as the number doesn't exceed the total number of users you have. I used the default, 10, for the average number of recipients on the DL. Changing this number affects the calculated number of emails sent and received in a day. Click OK to close the Topology Properties sheet and move to the next step.

Step 3: Configuring LoadSim Test Properties
Configuring test properties is the central part of configuring LoadSim. From the Configuration menu on the main LoadSim screen, select Test Properties to bring up the Test Properties dialog box shown in Screen 4.

Settings in the Duration of simulation section determine how long the test runs. The default is Forever, which means until you manually stop it. Setting the test to run for a certain number of hours is useful for unattended tests because LoadSim will stop the test automatically. You can choose to Run each task a specified number of times for stress testing. With this option, LoadSim will execute the tasks as quickly as possible the number of times you specify. I rarely use this setting.

Length of daytime specifies how long LoadSim's artificial day will be. In other words, this parameter specifies the span of time in which LoadSim will perform a day's worth of user tasks. Remember, the user profile, such as the one in Table 1, specifies a day's worth of activity. Length of nighttime specifies how long each LoadSim user will rest before starting another day's work. In my example, the LoadSim users will work for 8 hours, take no rest, and start again.

Be careful changing Length of daytime. For Light, Medium, and Heavy users, a LoadSim day is 8 hours. That value means users are supposed to complete their assigned tasks over 8 hours. Thus, if you specify 4 hours for Length of daytime, and leave everything else configured the same, LoadSim crams an 8-hour workload into 4 hours, effectively doubling the stress your LoadSim users impose on the server.

For this example, if you want to run the test over only one LoadSim day, configure the Duration of simulation to be 8 hours, and LoadSim will stop the tests automatically after one full day. Or, you can configure the Duration of simulation to be Forever and stop the tests manually. In both cases, however, the load that LoadSim imposes is correct according to the user load specifications.

Can you change both values to 4 hours and still be OK? No. That configuration would still effectively double the load imposed by LoadSim and just run the test for 4 hours.

If you really wanted to run a 4-hour test, but still maintain the Medium load, you would need to halve the frequency parameters specified in the Medium profile (reduce reading new mail from 12 to 6, reduce sending new mail from 4 to 2, and so on). However, reducing your test's cycle time can decrease the accuracy and reliability of the results. Generally, LoadSim should run for 4 hours, including a ramp-up time of 1 hour. I'll discuss LoadSim testing approaches further in part 4 of this series.

If you are familiar with LoadSim 4.0, you will notice that version 5.0 has no tabs for configuring the many test parameters. You access those parameters via the Customize button at the bottom of the Test Properties dialog box. Also, notice that the User groups section is empty. You need to add information to that section before you do anything else.

To add a user group to the list, click Add to bring up the Edit User Group dialog box. Here you specify the main parameters about the test, as shown in Screen 5.

Server specifies the name of the Exchange server that the test will run against. The server in my example is arrakis.

Protocol specifies which protocol this Exchange client will use. In this case, I've selected Exchange (MAPI). Only the protocols for which you configured users in the Topology Properties configuration are available here. If you configured LoadSim with other protocols, you can specify them here.

User type specifies the LoadSim user profile. My test calls for the Medium profile.

First user specifies which of the 300 users this client will start simulating. Note that the first user in the example is user 0.

Number of users specifies how many users this LoadSim client will simulate. My test calls for this client to simulate 100 users.

The LoadSim software automatically calculates Users covered based on First user and Number of users. This LoadSim client is simulating users 0-99.

The information in the Edit User Group dialog box determines how many LoadSim users are simulated on this particular LoadSim client, regardless of the total number of LoadSim clients. It also controls which users send and receive email.

LoadSim 4.0 lets you specify both senders and recipients. In LoadSim 5.0, the users you choose for a user group send email only to other users in the group. For example, suppose you have 500 total LoadSim users in the Exchange directory, and you configure a user group that contains the first 100 users (0 through 99). If you run the test using just this user group, only users 0 through 99 will send and receive email; users 100 through 499 will neither send nor receive email, with one exception. DLs can contain any of the 500 users, so email sent to a DL can go to users 0 through 499.

Click OK to return to the Test Properties dialog box. Screen 6 shows the result with my user group entry in the list. Some of the headings are cut off because of the size limitations of the dialog box, but you can see a summary of the selections I made.

Step 4: Creating the LoadSim Test Topology
You're ready to leave the Configuration menu and move to the Run menu. You've configured the topology. Now you must create the topology in the Exchange directory.

This step creates a list of users and DLs and attempts to import them into the Exchange Directory. LoadSim does not use any users or mailboxes you might have created in your Exchange server. Instead, LoadSim creates its own user names and mailboxes (and DLs, if you enabled the use of DLs). After everything is imported into the Exchange Directory, the recipients are placed into a separate recipient container called LoadSim in the Exchange Directory. Fortunately, you have to do this step only once.

Open the Run menu, and select Create Topology. No further configuration is necessary; the software generates the file containing the user list and attempts the Directory import automatically. Screen 7 shows a sample output screen, starting with Begin Topology Creation and ending with End Topology Creation.

The automatic import usually works; however, as Screen 7 shows, in some cases, you will get an error message during the import. The most common cause is that your LoadSim client does not have administrative privileges on the Exchange server where LoadSim is attempting the import. If you look in the Application Event Log, you will see the reason for the failure.

If you get an error, don't worry. The user information has been created in a comma-separated variable (.csv) file, so you can finish the job. You'll need to manually import the information into the Exchange directory (as the next-to-last line in Screen 7 suggests).

The topology creation should happen almost instantaneously in this case, because you are creating only 300 users. Even if you are creating more users, the process is still fast because LoadSim is just creating an ASCII file. The actual directory import takes longer.

During the topology creation, LoadSim creates a file and lists its name on the screen. This file is a .csv ASCII text file, and it is in the appropriate format for importing into the Microsoft Exchange Server Directory. The file in my example, dune.csv, is in the directory where I installed LoadSim. LoadSim names it using the convention site.csv. It contains the 300 user names and the 30 distribution lists specified earlier. You will import this file into the Exchange directory.

My preferred method is to copy dune.csv to the exchsrvrbin directory on the Exchange Server machine or share the file over the network. Then you import the directory from the Tools menu of the Microsoft Exchange Administrator. If you use this method, you do not have to create NT accounts for each imported LoadSim user. With this method, you also don't have to worry about setting site permissions for the LoadSim client to allow the import.

If you don't change the test configuration, LoadSim creates an identical output file each time you run Create Topology. For example, if you run Create Topology for 300 users and run it again for 400 users, the first 300 entries for both sets of files are identical because LoadSim uses a consistent algorithm to generate user names. If you examine the .csv file, you see user names derived from the Exchange Server name. I recommend that you generate the maximum number of users you plan to use, import them all, and then run your tests.

Step 5: Setting LoadSim Options
You're almost finished configuring the test, but let's review the LoadSim program options. You might find some of these options useful.

When you select Tools, Options from the LoadSim menu, two tabs appear on the Options property sheet. These tabs let you configure various global LoadSim parameters, and the program stores the values in loadsim.ini in the %SystemRoot% directory.

Screen 8 shows the Logging tab. On this tab, you can specify general operating parameters for LoadSim.

The Output window section contains check boxes that determine what appears on the LoadSim screen during a run. I usually opt for the default selections shown in Screen 8.

Show time is an important option to select because you want time stamps on all activities. Showing process and thread ID information is optional, and you want to select them only when you are trying to debug a problem.

Log to file is useful if you want a log of exactly what is printed on the LoadSim screen during the run. The default filename is loadsim.out. If you don't want to overwrite previous loadsim.out files, select the Archive previous file check box.

Log performance data to file is an important entry. You must enable this parameter if you want to generate logs that can be parsed with LSLOG to produce scores. (LSLOG is the tool you use to compile the performance statistics from the logs. I'll talk more about LSLOG in part 4.) Typically, you want to overwrite old logs so that you don't end up accidentally mixing them between separate runs. The default is loadsim.log. If you don't want to overwrite previous loadsim.log files, select the Archive previous file check box.

The second tab on the Options property sheet is Tasking. Screen 9 shows the parameters you can set on this tab.

Maximum number of LoadSim processes tells LoadSim how many instances of LoadSim processes to start. LoadSim can be resource-intensive, so you want to leave this parameter at a low number, such as 25. If you increase this parameter too much, NT will create too many processes to manage effectively, decreasing the accuracy of LoadSim.

The Threads section is something that you probably won't need to alter, but I'll explain it here for completeness. LoadSim manages threads, allocating a thread for each LoadSim user, so you will usually clear the Allow thread pooling check box. LoadSim will calculate the optimum number of threads it needs, but you can override it with the check box. You might want to share a pool of threads among all the users rather than allowing one thread per user. For example, you might want 100 users to share 75 threads, taking 25 fewer threads from the system. Thread pooling lets LoadSim scale better so that you can simulate many more users on one LoadSim client. However, you want to use thread pooling only for clients that are mostly idle, such as Post Office Protocol (POP) 3 and Network News Transfer Protocol (NNTP). If you start messing around with this setting, you can adversely affect LoadSim's accuracy and spoil your test results.

Do not enable thread pooling for Exchange (MAPI) clients. LoadSim users running the MAPI protocol need to have one thread dedicated to each of them. Thread pooling mostly applies to simulating users with Internet protocols, NNTP, POP3, LDAP, and so forth. Don't change the thread-pooling configuration unless you know what you're changing.

Following Configuration
Now you know all the basic steps to configure LoadSim. Next month, I will cover customizing the LoadSim parameters, which you access by clicking the Customize button in the Test Properties dialog box in Screen 4. In part 4 of the series, I will cover initializing the Exchange database, running the test, and collecting and analyzing the data.