Understanding the latest SQL Server TPC-H scores

I recently read Brian Moran’s recent article on the latest Microsoft TPC scores (http://www.sqlmag.com/articles/index.cfm?articleid=96147&puuid=420B5669-1279-9119-120E2F28B99E67C6) with great interest.  I’m a benchmarking junky from way back in my enterprise DBA days and always look forward to the latest news on major benchmarking results from the big database platforms.

Brian’s article points out that Microsoft SQL Server’s latest numbers, once again, outperform Oracle both in terms of the raw number of transactions processed and the cost per transaction.  As Brian points out, this is both exciting for SQL Server professionals and deceptive.  After all, how many of us can afford to buy and equip a $2 million dollar array of hardware for our applications?  And if we do need to spend that much money on equipping our application, A) would the TPC-H test really tell us what we needed to know or B) should we spend money building a much more realistic and representative test?  As you can guess, the answer is B. 

That’s why the TPC tests are, at times, deceptive.  First of all, the TPC tests are constructed to be as realistic as the Transaction Processing Council can make them, but they’re still artificial.  Second, any test whose parameters are fully described can be “kludged” for maximum, if unrealistic, performance gains.  It’s in this second point that I believe every SQL Server professional should take the time to actually read the new TCP test result disclaimers because that’s the only way you’ll ever be able to see “the man behind the curtain”.

So, go to http://www.tpc.org/tpch/results/tpch_last_ten_results.asp or http://www.tpc.org/tpch/results/tpch_result_detail.asp?id=107052101 and read about the current tests scores.  Then go to the bottom of the page and click on “Full Disclosure Report”.  It’s only 509 pages, so it should be a quick read for you, right?  Wrong!

However, I’m going to make life a little easier for you and report on the test over the next few days (as I read it myself) to give you a good idea as to the sorts of tweaks and tricks used to achieve hyper-performance on a top of the line HP Superdome system.