Benchmarking Internet Performance

Tools and tips for testing your 'net speed

I frequently receive questions from readers about which are the best tools and methods for determining the performance of an ISP connection, including those made over 56Kbps-modem, ISDN, T1, frame-relay, Digital Subscriber Line (DSL), cable-modem, wireless, and satellite links. These questions come from corporate desktop and small office/home office (SOHO) users who are looking for ways to perform benchmarking tests under Windows 2000 Professional. I recently had occasion to do some Internet-performance benchmarking from my own Win2K Pro system during an Internet-connectivity changeover.

Metering My Migration
Unfortunately, user-friendly Win2K and Windows NT­compatible network-benchmarking tools haven't always been easy to come by. The last time I needed to perform this type of testing was last year after I installed a wireless Internet connection to my ISP. At the time, I used one of the only inexpensive Internet-performance-testing tools that worked with NT 4.0, VitalSigns Software's Net.Medic. (Lucent Technologies acquired Net.Medic, which now has the name MyVitalAgent.) The software provides data about upload and download speeds and, based on its findings, offers suggestions about where delays might exist in a connection. Many ISPs recommend this software to their customers because it educates customers about why and where Internet-connection delays occur. Customers are less likely to blame their ISP for performance problems that are originating on a connection that's not part of the ISP's network. My overall experience with Net.Medic on that occasion was fairly good and provided me with general upload and download performance benchmarks that I used to gauge the speed potential of my wireless connection.

Recently, my ISP's wireless service and infrastructure provider informed me that it needed to migrate me from my existing connection to a purportedly faster radio connection. I decided to benchmark my connection performance before and after the changeover to ensure that my connection's performance improved or at the very least, didn't diminish. Benchmarking is valuable to me because the functionality and performance of my Internet connection is important to me. I don't handle poor performance or downtime well.

Trial by Benchmark
I found several Win2K-compatible utilities capable of testing an Internet connection's performance. Most of these utilities require you to manually generate traffic across your connection and use the software to analyze the connection's performance.

Alternatively, you can run Internet-connection-speed tests from speed-test Web sites such as DSLreports.com (http://www.dslreports.com), SPEEDUS (http://www.speedus.com), and 2Wire (http://www.2wire.com). Most of these Web sites' testing tools let you pick the Web site's closest regional server to test against, then they use client- or server-side scripting to generate traffic and report the results.

Although speed-test Web sites provide useful diagnostic tools, they don't provide the most accurate level of testing because you have little or no control over the logical distance (i.e., the number of intervening router hops) between your machine and the site's remote servers. Although many sites let you select Web servers geographically close to you, significant latency can still exist on the connection and interfere with the test's accuracy. In addition, how busy the Web server is at the time you conduct your test can affect test results. Because of the possibility of interference and the lack of control most Web site­based tools provide, I elected not to use them in my test.

Instead, I used two products in my testing: AnalogX NetStat Live (NSL) and MyVitalAgent (the new version of Net.Medic), both of which are free downloads. I used two tools from different vendors so that I could compare the results. Two result sets would reveal erroneous or misleading results from faulty testing methodologies and insufficient data samplings.

After verifying that the new radio connection was working properly and my ISP connection was reestablished, I used NSL to run the first post-installation test. (I had run the same tests before the changeover to serve as a baseline for comparison.) If you appreciate no-frills software that provides a "just the facts, Ma'am" display, you'll like NSL. It works with any type of Internet-connected adapter, whether the adapter is an Ethernet LAN adapter connected to a DSL or other router type, an analog modem, or a cable modem. (This support lets you use the utility to also test internal network throughput.) As Figure 1 shows, NSL provides a variety of information regarding the connection, including the current, average, and maximum kilobits per second or kilobytes per second achieved on both incoming and outgoing traffic, as well as the CPU utilization on the local machine. It's also capable of collecting and displaying historical connection-usage summaries.

For my second test, I ran MyVitalAgent, which came up with similar results as those NSL reported. As Figure 2 shows, MyVitalAgent provides a dashboard-style display that illustrates independent upload and download figures. This display also provides useful information about how much time each network transaction took and what percentage of the time spent waiting was attributable to the client, the server, and the network. In addition, this utility can detect and display the type of transaction running across the connection (e.g., an FTP download) and provide a health log that summarizes a connection's problems and offers a potential cause and fix for each problem.

I was happy to discover that the results from both utilities showed that my new connection was capable of about 1.3Mbps downloads and 1Mbps uploads. This performance is a marked improvement from my previous connection, which the utilities reported as capable of 300Kbps uploads and 600Kbps downloads.

To maximize these utilities' benefits and the tests' accuracy, I recommend you take the following preparatory steps before running a test. First, shut down all other applications on the machine, including those running in the system tray. This step will minimize CPU contention and the risk that other applications will interfere with the testing utility.

Second, if possible, try to isolate the test machine so that it's the only machine employing the Internet connection. This step will eliminate potential interference as a result of network and Internet-connection bandwidth usage by the other machines on your network.

Third, to use these utilities, you need to generate traffic across your Internet connection. To test the upload and download speeds of my connection, I use the FTP Get and Put commands, respectively, to send files to and receive files from my ISP's FTP server. Whatever method you use to generate traffic must fully saturate the connection during the test period. Often, throughput spiking occurs early in file transfers and similar activities, so I recommend that you run a test that lasts at least a couple of minutes to get a more accurate average-throughput figure.

If your Internet-connected equipment supports compression and you're curious about how compressed versus noncompressed data transfers perform across the connection, run tests for each scenario. My ISP keeps two handy little files on its FTP server called testfile.compressed.10meg and testfile.uncompressed.10meg that I can download to test download speeds. Because I have FTP file space on my ISP's server, I can also turn around the transaction and upload the files to test upload speeds. (To create an uncompressed file of a particular size, you can use the Microsoft Windows 2000 Resource Kit's createfil.exe utility.)

Finally, when using Internet-connection-benchmarking utilities to test Internet-connection performance, a good practice is to use your ISP's closest server. This setup minimizes the amount of interference that Internet latency causes as a result of the various router hops between your test machine and the remote server.