A Look at SQL Server 2008

When one thinks of Microsoft's platforms, thoughts turn naturally to such products as Windows, Windows Server, Microsoft .NET (including the Web-based ASP .NET technologies) and perhaps Office. But Microsoft's success as a maker of platforms is based on far more than just the platforms themselves. It includes a developer ecosystem rooted in the Visual Studio tools and .NET development languages. And it includes the related data back-end, which centers on SQL Server.

The next version of SQL Server, codenamed Katmai and dubbed SQL Server 2008, will debut in February 2008 alongside Windows Server 2008 and Visual Studio 2008. This is no coincidence of timing. Indeed, these products will most certainly be completed at varying times, with Visual Studio 2008 scheduled for completion in late 2007 and both Windows 2008 and SQL 2008 not due until after the launch event. (SQL Server 2008 will be the last of the three to appear.) Microsoft is launching these products together because they form a cohesive whole, an interconnected uber-platform of sorts in which components of each both rely on and build off of components of the other.

SQL Server today

The SQL Server business has done phenomenally well for Microsoft, thanks to a slow and steady product revision schedule that maps more closely to customer schedules than most Microsoft products and a steady move towards the upper reaches of scalability, capability, and reliability. SQL Server 2005, the most current version of the product at this writing, launched in November 2005 after five years of development and also shipped coincidentally with a major Visual Studio refresh, Visual Studio 2005.

SQL Server 2005 was based around three pillars of functionality, trusted platform, faster results, and better insights. The product integrates with Visual Studio 2005 by allowing developers to build database solutions using the .NET managed code languages--Visual C# and Visual Basic--that they're already familiar with. Microsoft accomplished this by hosting the .NET CLR within the database, giving developers a way to write stored procedures in .NET programming languages instead of Transact SQL.

Since then, Microsoft has augmented that product with two major service pack releases, one of which, SP2, tightened integration with Office 2007 on the desktop, primarily via Excel and SharePoint. Seamless integration between SQL Server and Excel, especially, allows knowledge workers to use familiar tools to manipulate and examine back-end data. Plus, thanks to new Excel Services capabilities, any users can access SQL Server via a familiar Excel front-end using SharePoint in the browser. "That gives users one place to collaborate, share, and search," Francois Ajenstat, a SQL Server marketing executive at Microsoft told me.

Meanwhile, SQL Server sales are phenomenal: SQL Server is the fastest growing database, business intelligence (BI), and data warehouse product on the market, and is number three in market share behind Oracle and IBM, largely because SQL Server's pricing is so much lower and so much more feature inclusive.

Indeed, Microsoft's propensity to ship new functionality in SQL Server rather than ship it as a separate added cost product as would Oracle or IBM is, in my mind, one of its biggest strengths. One only needs to look at BI and OLAP support in SQL 2005 for a typical example.

SQL Server is also making huge gains in performance and scalability. It's now the owner of the SAP benchmark record, offers the best performance at 1 TB TCP-H (ad-hoc queries), and the best price at 3 TB. And since the dark days of Slammer in 2002, SQL has had zero critical security vulnerabilities, compared to over 100 for Oracle. And while Oracle users have suffered through over 300 security vulnerabilities overall in that time period, SQL users have had just 5. That's astonishing given Microsoft's overall security record.

Moving to Katmai

Looking ahead to the next version of SQL Server, Microsoft was motivated by a number of trends it saw developing. The first was the data explosion, where organizations are now storing key data both in and outside of databases. The data inside databases tends to be structured data, but there are now all kinds of unstructured data types being organized in the file system, such as document files, XML files, digital images, and so on. These, too, are corporate assets and need to be managed and queried. So Microsoft decided to expand the database to support these kinds of data.

The second trend is around compliance. There's a lot of regulatory pressure on organizations to store, document, secure, and control corporate assets, and new pressures on IT to control that data.

Finally, storage prices are going down dramatically and organizations are now storing more than ever as a result. And it's not just traditional database servers. There are a wide range of devices with integrated storage, and data is now spread out over laptops, phones, point of sales (POS) systems, and other devices. This data all needs to be stored, managed, and secured. "It's challenging," Ajenstat told me. "Users want to get the right data, in the right tools, whenever they need it. But administrators want to make sure the data is locked down and centrally managed."

These trends guided the scenarios SQL Server 2008 delivers on. Here's what you can expect in SQL Server 2008.

Trusted platform

At a 1997 SQL Server 7.0 reviewer workshop, I was astonished by Microsoft's Terra Server, a room full of data refrigerators that contained satellite imagery and was controlled by a custom SQL Server install. Now, SQL Server can manage multiple terabytes of data as seamlessly as previous versions did a gigabyte, and scalability is at the forefront of core SQL Server work. Part of this effort involves disk compression routines that reduce the footprint of on-disk storage. Part of it is new indexing mechanisms to speed queries of ever-increasing data sets. And part is an ongoing effort to improve performance across the board. For example, SQL 2008 prioritizes workloads intelligently so that administrators can configure service level agreements. That way, a reporting application can't consume all of a server's resources while payroll is trying to print paychecks.

Abilities

While SQL Server is traditionally thought of as a relational database, the product has been consistently improved over time to work with non-relational data. This started in SQL 7.0 with the introduction of multi-dimensional OLAP, and continued in SQL 2005 with XML support. In SQL 2008, Microsoft is expanding these capabilities in the most dramatic fashion yet with support for unstructured data, like documents in the file system, through a new file stream data type. This isn't a replacement for the file system, but rather a replacement for the old blob data type, which had no understanding of the underlying data. Microsoft is supporting this feature with native Windows APIs for querying and indexing the data, and is also formalizing the concept of location data, where you can query locations in a spatial way as with a map. I expect this capability to touch off a new era of custom location-aware applications.

Dynamic development

Expanding on the .NET managed code capabilities in SQL Server 2005, SQL Server 2008 includes support for Visual Studio 2008's LINQ framework, which allows C# and Visual Basic developers to work with business objects rather than tables and columns. This removes another layer of abstraction for developers who need to target databases and doesn't require these developers to be database experts as well. "Developers don't care how information is stored internally," Ajenstat explained. "They want to deal with the data-- whether it's a customer , product , store, whatever. The framework takes care of the backend."

SQL Server 2008 also addresses the occasionally connected scenario with SQL Server Compact, which can run on mobile and embedded devices and desktops. That way, mobile applications can access a database that works anywhere and then sync seamlessly with the server whenever they're connected. It works like Outlook and Exchange Server, providing offline access to data and then making sure everything is synced to the backend when possible. Obviously, it is secure and managed, and will work with whatever compliance policies the organization has created as well.

Pervasive insight

Building on the business intelligence (BI) capabilities of SQL Server 2005, SQL Server 2008 offers better scalability--where you can perform BI and data warehousing tasks against multiple terabytes of data--and better performance. SQL Server 2005 customers told Microsoft that while they liked that product's BI capabilities, the performance suffered at 20 TB of data and above. Now, Microsoft says even 40 TB of data will be comfortable.

Another related improvement concerns availability. In addition to scaling the database to accommodate more data, it needs to handle more concurrent users as well. So Microsoft has improved SQL Server 2008 to provide predictable performance regardless of the number of users.

Finally, SQL Server 2008, like its predecessor, will work well with current versions of Microsoft Office, and this time around there's no need for a software upgrade: It works out of the box.

Scheduling and pricing

As with previous versions, SQL 2008 is on a slow burn, and Microsoft says it has settled on a 24 to 36 month development cycle. "We changed how we build the product to be more predictable," Ajenstat told me. This credo has become reality even during the beta process, which thus far has been marked by a few public CTP releases. Expect two more CTP releases before the launch.

The target release date for SQL Server 2008 is the second quarter of 2008. It will launch alongside Windows Server 2008 and Visual Studio 2008, though each of those products will be finalized at different times from SQL Server 2008. As with previous versions, Microsoft will ship compact versions of SQL Server 2008 for the desktop and mobile devices in addition to its enterprise-oriented offerings.

The best news, perhaps, is that Microsoft is not raising the price. Compare this to Oracle, where every new feature seems to come with a new $10,000 charge. But licensing fees for SQL Server 2008 are identical to those for SQL Server 2005, despite the many additional features. That's good stuff, and a continuation of Microsoft's aggressive value proposition for this product line.