Windows Azure and the Future of .NET Development

Microsoft is still aggressively touting the benefits of its Windows Azure platform. And while the big benefit of Azure today is that developers can push their applications into the cloud, I’ve long suspected that more good would come from Azure than just the immediate, and obvious, benefits that are currently available. I’ll share some of my suspicions and hopes. And look at how they might be starting to take shape with Microsoft’s “top secret” project Orleans.

A Missed Opportunity

A few weeks ago I was going to write about what I thought the addition of Mark Russinovich to the Azure team would mean (long-term) to .NET developers. I had the article drafted, but decided (last minute) to write about Visual Studio LightSwitch instead.

In thinking about Russinovich and Azure, I had outlined what I saw as current pain points with .NET development (in terms of scalability and availability). I was hoping that the inclusion of Russinovich (and other heavy hitters) to the Azure team would help bring about a fundamental change to the way application workloads are handled in truly “distributed” environments. I was hoping that .NET development would finally see some “grid-like” improvements or changes.

Then Mary Jo Foley broke the news of a new Microsoft R&D project she had discovered – called project Orleans. In quickly reviewing what she had discerned about this project, it looked like a dead-ringer for the fundamental changes I was planning to write about. So by not writing out my ideas a few weeks ago it appears that I missed an opportunity to look smart and predict things before they happened.

But the truth is that anyone who’s been developing with both .NET and SQL Server for the past few years could probably come to the same conclusions that I had. I’m excited to see what Orleans will mean for .NET developers, specifically in terms of how Orleans might address some of the pain-points that were on my mind.

SQL Server and SQL Azure Lead the Way

Just as I opined a few months ago about how SQL Azure would (eventually) bring game-changing improvements to future, non-cloud, versions of SQL Server, I’m convinced that Windows Azure will bring fundamental changes to the way that developers write applications.

Case in point: concurrency. In 2005 Herb Sutter released his The Free Lunch is Over article about how software developers needed to make a fundamental shift towards concurrency. For developers, this was a bit of a wake-up call confirming that applications would no longer be able to rely upon faster and faster processors to defray the cost of increasing complexity (and bloat) within their code and applications.

However, if you’re familiar with SQL Server, you know that it has been multi-processor aware for more than a decade now. More specifically, SQL Server has been able to “magically” take advantage of multiple processors to quickly perform operations without any real need for control or monitoring by developers. In fact, SQL Server’s ability to take advantage of multiple processors is so seamlessly implemented that the vast majority of SQL Server users probably don’t even realize that there’s a way (MAX DOP and processor affinity) to control how many processors SQL Server will use to complete a query or operation.

Too bad that’s not the case with .NET development today because even with parallel programming support within .NET 4.0, multi-threading remains non-trivial to implement. (Of course, that isn’t to say that leveraging multiple processors or cores from within SQL Server always works flawlessly; in rare cases DBAs need to tweak, configure, and constrain SMP operations, but in the vast majority of cases, multi-threading within SQL Server just works.)

Concurrency, Availability, and Scalability

It’s important to remember that concurrency (the ability to increase performance by throwing more processors at a problem) is just one consideration when it comes to .NET development. Two other big considerations are availability and scalability. And, interestingly enough, those two considerations are also the hallmarks of cloud computing. (Without the ability to make your applications or solutions more redundant, or fault-tolerant, and without the ability to easily scale your solutions, then all you have are hosted solutions–which are not the same as cloud solutions.)

Because Azure is truly a cloud-based service or offering, it’s only natural to assume that it will address business and developer needs for applications that are fault-tolerant and can more readily scale.

The only question for developers, then, is whether that’s all we’ll get? Or will experiences learned by the Azure team when it comes to abstracting workloads away from their physical implementation hosts be rolled back into future changes and directions for .NET development in general? I’m of the opinion that it’s going to be the latter because Microsoft has already placed such a huge amount of emphasis on addressing concurrency, scalability, and availability over the years.

Clustering Services, the Cloud, and the Need for Orleans

In my estimation, Microsoft has actually done a great job of addressing availability and scalability with clustering. Load-balancing and fault tolerance are key components that have made clustering such a viable option for so many enterprises. The problem with clustering, though, is that it’s really non-trivial to implement and maintain. Plus, clustering is quite expensive. Consequently, I’m hoping that since Azure needs to address these same issues of fault-tolerance and the ability to easily scale, we’ll start to see NEW solutions and paradigms for addressing these pain points from an Azure team that’s working to solve these problems in a real, viable, cloud.

Then, of course, I’m hoping that these insights make their way back into .NET development in general giving an area of focus for the development of the .NET Framework itself. And, frankly, it looks like this is exactly what project Orleans looks to address.

Project Orleans

If you haven’t done so already, take a quick peek at Mary Jo’s overview of what Orleans is shaping up to be. You’ll see that it’s built with increased fault-tolerance and scalability in mind and that these considerations are addressed through indirection or abstraction. More importantly, it looks like the abstraction in question will actually be a new platform one that I hope will be written from the ground up to support .NET solutions.

Think about that for a minute.

And, with that, here’s hoping that the Orleans runtime is everything it looks like it could be: distributed, manageable, versatile, scalable, fault-tolerant, and written specifically with .NET developers (and their apps) in mind.