TPC Overturns SQL Server Benchmark Record

The Transaction Processing Performance Council (TPC) has overturned the TPC-C and TPM-C benchmark records that SQL Server 2000 set in February 2000, citing a noncompliance issue that Microsoft rival Oracle discovered. Microsoft has touted the figures since the February Windows 2000 launch event, at which the company first announced that the SQL Server 2000 beta, running on Win2K, processed 227,079.5 transactions per minute (tpmC). At the time, the score was the highest ever achieved on that benchmark. But the TPC says that the Microsoft benchmark, which the company produced on a cluster of 12 Compaq servers, violates a key component of the tests. To meet the criteria, the primary key of a distributed database needs to be updateable, but Microsoft obtained its scores by using a nonupdateable primary key.

"[Microsoft's score] was found [to be] noncompliant," said Jerrold Buggert, the chairman of the TPC. "Given the nature of the problem, the tests need to be redone." Microsoft disputes that claim, stating that the previous record holder, an Oracle database, also failed to implement an updateable primary key. But Microsoft has modified the SQL Server 2000 code to comply, and the final version of the product includes this fix. The results of the modified benchmark aren't dramatically different from the original results, Microsoft claims.

Oracle apparently tipped off the TPC to the benchmark discrepancy. According to sources close to Oracle, the company had been working diligently for months to find a loophole in the Microsoft record, which knocked Oracle 8i from the top spot. Each of the benchmark records is open to peer review.