NT beats Linux again in performance tests

After the controversial Mindcraft benchmark tests this spring, which showedWindows NT 4.0 outperforming OS Linux by a factor of 2 to 3, rival factionshave endlessly debated the tests, the results, and the relevance of suchbenchmarks. More recently, other tests have shown NT beating Linux as well,though not by nearly the extreme scores that the Mindcraft test portrayed.But the biggest sticking point, really, was that the Linux community wasn'ttaking part in these tests, fine-tuning the Linux machines as needed so asto best compete with NT.

Until now, that is.

PC Week Labs released the results of its latest Windows NT vs. Linuxtesting, which was audited by representatives from Microsoft Corporation andLinux vendor Red Hat Software. And like previous tests, Windows NT 4.0 beatLinux handily, despite heaving tuning and tweaking of the Apache Web serverand Samba file-sharing feature used by Linux. And lest the Linux camp cryfoul again (fear not, they will), the PC Week test took into accountvirtually every Linux tweaking tip it received. For example, Linux advocatespoint out that Apache isn't designed for speed and that an alternate Webserver such as Zeus would be more appropriate. So PC Week tested Zeus onLinux as well (and, incidentally, found it almost exactly as anemic asApache, performance-wise).

"In all the areas in which the Linux community cried foul, its assumptionswere wrong," the report reads. "We tested Zeus on Linux and found itsperformance peaked almost exactly where Apache's did."

It turns out that the performance problems in Linux are caused by a poorlywritten IP stack in the Linux networking subsystem, and are not caused bythe Web server at all. Red Hat says the problem will be fixed in the nextmajor revision to Linux, version 2.3.

So let's look at the numbers. For Web serving, Windows NT's InternetInformation Server (IIS) 4.0 handily beat Apache or Zeus on Linux,responding to an incredible 4166 requests per second, compared with 1842 forLinux, an advantage of 226% percent. Linux advocates complained that earliertests used high-end multiprocessing systems that weren't typical Linuxconfigurations, so the test also included single processor systems, whereWindows NT 4.0 once again beat Linux, 1863 requests to 1314. So even oncheaper low-end systems, Windows won by 41%, contrary to the expectations ofmany Linux users.

What's particularly impressive for NT is that a single processor Windows NTWeb server performs equally to a four-processor Linux server. This reversesthe theory that a low-end Linux box can outperform a more expensive,high-end NT box. For Web serving, the exact opposite is, in fact, true.

File serving was another big win for NT: Linux and Samba lost to Windows NTin every test that was run. Linux's best number, 155.9 Mbps, fell well shortof NT's 338.3 Mbps. And an earlier test that showed Linux and Samba beatingNT with NT Workstation clients was turned around, as Microsoftrepresentatives found the problem with the earlier testing: In thiscomparison, NT won yet again.

When it comes down to it, it's not surprising that NT performed so well:It's been in development for over a decade, was designed with networking asa primary component, and has been tweaked and put through the paces by thelargest user base there is. But this isn't a black mark for Linux: For theopen source OS to have performed as well as it did is indeed a testament toits prowess. While the gap may seem wide today, it's clear that the Linuxcommunity will be working on the very components that are responsible forthis gap. In the future, Linux will likely come closer to the performance ofWindows NT. By that time, of course, the heavily modified Windows 2000 willhave shipped, and the game begins anew.

For more information about the tests, including some interesting performancecharts, please visit the PC Week Web site