Benchmark Results and Ketchup

Greetings,
Read on if you're interested in the answer to this thought-provoking question: What do the latest SQL Server world's-best benchmark results have in common with ketchup? Trust me, I'm really going somewhere with this question.

Last week, Microsoft and Unisys announced a new world record for the SAP Sales and Distribution (SD) benchmark. The simulated workload of 10,400 SD benchmark users is the best score ever achieved on a Windows platform for a database server with 16 or fewer processors. A Fujitsu Siemens mainframe holds the current best score of 23,000 SD benchmark users, and the top UNIX score of 19,360 SD benchmark users was achieved on a 64-CPU UNIX database. For more information about the benchmark, see the Unisys press release.

I haven't checked system prices, but I'm pretty sure that a 16-CPU Windows box is a heck of a lot cheaper than a mainframe or a 64-CPU Solaris/Oracle box. However, we've known for years that SQL Server can claim the world's best price-performance ratios, so there's nothing new to report on the cost front. But consider these other facets of the benchmark results. First, SQL Server achieved 54 percent of the best UNIX score with just 25 percent of the processors. Second, the Unisys press release tells us that the new Microsoft score maxed out the 16-CPU SQL Server box with 99 percent CPU utilization. I verified this statement with Microsoft, which assured me that the benchmark results are absolutely CPU bound. In other words, SQL Server had power to spare, but 16 measly Intel CPUs simply didn't have the processing power of a 64-CPU UNIX box. This tells me that SQL Server SAP SD benchmark scores will tend to scale nicely as Microsoft adds CPUs. Do the math, and you'll see that SQL Server achieved 900 SD benchmark users per CPU, whereas the UNIX box achieved 303 benchmark users per CPU. SQL Server was roughly three times more efficient per CPU.

These observations lead directly into the main point of this week's commentary, which, of course, is to explain why SQL Server's SAP SD benchmark score reminds me of ketchup. I apologize to our international readers for yet another lapse into pure Americana, but here goes: "Anticipation . . . anticipa-a-tion . . . it's making me wait . . ." Come on, you know the words! Sing along with me! That's right, it's the old Carly Simon-Heinz Ketchup anthem! And here's the SQL Server tie-in: I've preached SQL Server price-performance benefits for years, but UNIX has been able to top Microsoft with super high-end benchmark scores because UNIX hardware is more advanced than WinTel. However, this SAP SD score is the first "industry-standard" benchmark published for Windows/SQL Server systems running on 16 processors. It's also the first benchmark running Windows 2000 Datacenter Server and the first Windows 16-CPU benchmark running on the Unisys ES7000 platform. Check out the ES7000 specifications, and you'll see that the only way to describe this monster is as a "WinTel mainframe." WinTel hardware is quickly catching up with its upscale UNIX cousins.

I've anticipated 16- and 32-way benchmark scores for months—since last summer's release of SQL Server 2000. Bit by bit, SQL Server systems are closing the gap with the "world's fastest" 64-CPU UNIX database servers. Unquestionably, 32-CPU SQL Server 2000 benchmarks running on "WinTel mainframes" will be an important milestone in the quest for single-server performance parity. Will the gap close completely when someone finally publishes 32-CPU scores sometime in the near future? Perhaps not, but the gap is closing awfully fast. I can't wait to see those 32-CPU scores, but anticipation is half the fun—just like waiting for my favorite, rich tomato condiment to start flowing.

If you'd like to do your own SAP SD research, http://www.xware.net/html/zerti.htm contains a Web interface you can use to view all current SAP Standard Application Benchmark certified results.