SQL Server's World-Record TPC-C Scores: Why Should You Care?

Greetings,
Once again, Microsoft can officially claim to have the world's fastest database. Friday, SQL Server 2000 posted a new Transaction Processing Performance Council (TPC) TPC-C benchmark score of 505,302 transactions per minute C (tpmC) at $20.68 per tpmC, beating the previous best score of 440,979 tpmC and $32.28 per tpmC. The new score means that SQL Server now holds two of the three fastest TPC-C scores, while Microsoft maintains the 40 best price/performance TPC-C scores.

I'm torn between two competing reactions to this news. My first reaction was (to steal a phrase from 1980s television), "That's incredible! How did they do that?" In just 2 months, Microsoft nearly doubled its previous TPC-C score of 262,243 tpmC. And the company didn't release a new product or service pack or bribe the TPC-C auditors. Microsoft simply scaled out its solution from 96 700MHz Intel Pentium III CPUs spread across 12 Compaq servers to 192 700MHz Pentium III CPUs spread across 24 Compaq servers. By doubling the number of processors and servers, Microsoft increased throughput by roughly 93 percent, at an increased cost per transaction of about 2 percent. That's called linear scalability, and it's the holy grail of the database world.

My second reaction to the new world-record score was, "Who cares?" When SQL Server 2000 posted its previous best TPC-C score of 262,243 tpmC, a Microsoft press release said, "At this transaction rate, SQL Server 2000 could handle all of the e-commerce transactions that Amazon.com and eBay.com processed in 1999—and it could do it in just 2 days."

Does this mean that the system that recorded this new tpmC score could handle Amazon's and eBay's 1999 traffic in just one day? Using linear scalability to double SQL Server's best TPC-C score is great, but at this volume, the numbers begin to lose any real meaning. I've learned a lesson from readers over the past several months. Few people care about benchmark numbers this high because the numbers have no bearing on the types of systems most of them run in the real world. And Microsoft's scale-out strategy, which is based on distributed partitioned views, isn't ready for prime-time mass-market consumption. Real sites use distributed partition views to achieve stunning results, but implementation and manageability limitations restrict distributed partition views to a handful of customers.

The only SQL Server TPC-C score that could excite me at this point would be based on a 32-CPU machine running Windows 2000 Datacenter Server, because single-machine scalability is the one area in which UNIX database vendors still have a noticeable advantage. But I'm not worried; I believe Microsoft will soon be able to go toe to toe with UNIX database solutions in the realm of single-machine scalability.

Having said that, I'm convinced that Microsoft's scale-out TPC-C score is important—whether or not you need to handle Amazon and eBay yearly transaction loads in a day. The point isn't whether you need to; the point is that you can. Today, too many people shy away from Microsoft out of fear that the platform will run out of steam when they need it most. If the new scores do one thing, they assure the most cautious customer that SQL Server can finally play with the big boys when it comes to enterprise scalability.

I just hope that Microsoft and IBM don't start playing a game of "I'm king of the TPC-C world," in which they leapfrog each other's best score every other month. TPC-C benchmarks consume tremendous amounts of vendor resources, and most customers don't care if the numbers go any higher. Perhaps Microsoft and IBM could agree to produce the same score, stop their silly race, and focus on solving problems that their customers care about.