David Nielsen on the State of Cloud Computing Development

Editor's note: Welcome to .NETRocks Conversations, excerpts from the .NET Rocks! weekly Internet audio talk show. Hosts Richard Campbell and CarlFranklin chat with a wide variety of .NET developer experts. This month's excerpt is from show 669, with David Nielsen, founder and principalconsultant of Platform D, founder of CloudCamp, and author of the bookPayPal Hacks. Richard and Carl chat with David about the state of cloud software development.

Carl Franklin:Platform D, the website, is this your consulting company?

David Nielsen:Yeah. I'm basically a one-man show.

CF:And you're all about the cloud.

DN:That's right. I've been doing web services for the last seven years, and then after that I got my computing kicked into high gear. I've been working inthe cloud base for the past three years.

CF:When our listeners think of cloud, they think of Amazon, they think of Azure. What's the state of these various cloud offerings?

DN:Well, I think of those as well. There's a bunch of others, and it seems like everybody has got a cloud these days.

CF:Is this [cloud] a really overused term these days?

DN:I think it actually might become the new term, the one that we use instead of a thing like the Internet or the web. When you're talking aboutapplications and using applications on the web, I think we're just talking about the cloud.

CF:So it's really the Internet and web for business developers on the Internet.

DN:Yeah, kind of. The cloud implies somebody else is managing things for you. So when you're talking about using a service, they think we're talking aboutthe cloud. Then when we talk about looking up information, we might talk about look it up on the web. So cloud is kind of like a subset of the web in away. I think that's one way to look at it.

CF:Yeah, OK. So you've done some work for PayPal. Was that cloud-based work, or was that just straight-up web development?

DN:No. Actually I was PayPal's first web API evangelist. So back then using web APIs as a part of your website was pretty minimal. Those weren't seen alot. You ran the risk of having your website go down if the other companies didn't respond or something. Then in 2006, when that became kind of therage, everyone was realizing that there's so much value in using somebody else's data or service that it was worth the risk, and that's when we kind ofstarted kicking off this whole cloud computing space where you realize somebody else can provide something pretty critical to your website.

Richard Campbell:Especially when you talk about a payment system, where I don't make money if this doesn't work.

DN:Right. But with payments, you really have no other choice because it's not like you build your own payment processors.

RC:I guess that's true.

DN:You kind of have to do it. There are different types of data that I think just had to do, like real-time code, stuff like that. So you have to rely onsomebody else to some degree. With PayPal, our payments were the ones that you had to do real time, so it was pretty important.

As far as my perspective on cloud computing, I think the most important thing to consider is that cloud computing as a term really became interestingafter 2006 when Amazon EC2 and Salesforce.com both went into beta. It's because those are the first few services that successfully [let] you put yourcode on their servers, where they manage it as a cloud service.

CF:Right.

RC:Now I get that for Amazon. But talk about Salesforce, because I thought it was just a CRM product online.

DN:Well, that's what they're most famous for, but they also have this thing called Force.com, which is

a place where you put-originally you put snippets of code up there, little bits of Apex code, which kind of looks like Java a little bit. You couldcopy and paste there or write and paste that from the website, and then they would manage it, and it would talk to their databases-so that [it] waslike running your code on their platform. And then over time, they open it up. You could run it on any website, not just pass it through Salesforce'sCRM database. So you can have your own panel of database, you could create your own system or maybe a financial system or maybe a little pane of toolsto help pay commissions for your salespeople. You can do anything you wanted really up there as long as it could be written in a scripting language.

RC:I'm with you. Most people think of Salesforce.com for the CRM product, they thought they just sort of jumped on the cloud bandwagon. If you don't knowabout Force.com-this is a full-out cloud development environment now.

DN:Yeah, it is. I mean, they were first to do platform as a service, quite frankly. There were other companies before them, but [those companies] allpretty much failed or...they got lost. But Force.com, they did it well, and they kind of got overshadowed when EC2 launched because with EC2 you couldput your own operating system up there. You could do it with Windows or Linux and do whatever you want, whereas Force.com, as platform as a service,you get a lot of nice things like automatic scalability, but you are limited to that proprietary environment.

RC:So you don't call EC2 platform as a service. Is it infrastructure as a service?

DN:Yeah. It's an infrastructure as a service. Amazon Web Services does have some platform as a service features like simple VB and trying to call ElasticMapReduce, which is kind of a service. But for the most part, it's infrastructure as a service.

RC:And then we got Azure.

DN:That's kind of like right between the two because in a way, it's not really infrastructure as a service and it's not really...I mean, it is platform asa service, but it also is operating system as a service. You get Windows, all managed for you. You don't have to worry about updates or caches orvirus, or that kind of stuff. They kind of take care of all that for you, and then on top of that they manage a platform, sort of a platform ofservices, for you as well, if you want. So [Azure is] kind of both. Just like operating system as a service, you can use it on PHP, Java, C++, but Ithink most people think of it as a platform as a service as if you're writing in .NET.

RC:So which one is better?

DN:Well, I think it depends on what your goals are. If you have an application that needs to run multiple programming languages, then I figure it more asoperating system as a service, if you have different services that you're pulling together to make an application work. But if you're buildingsomething from scratch, you're probably doing it in .NET, and you're probably going to use the platform that they provide you, the workflow and allthat you've mentioned-and then you're really using platform as a service.

CF:We should just mention what services you get when you sign up with Azure. Because when you just hear the word service, you might think, well, OK, do Iget a command line? Do I get a tool, or do I get a remote access to a full-blown operating system? That story changes over time. So tell us what youget.

DN:Yeah. Well, you really are getting everything above the operating system, like you're not going to be able to make changes on the operating system, atleast not much, but you can install pretty much anything you want on top of it. The nice thing about it is that if you are a Visual Studio user, youcan write your code right in Visual Studio. You can even run it on your desktop, and that's it. You don't have to connect to the Internet every timeyou want to run it. Then you can deploy it, and it goes right up on Azure straight from Visual Studio. So you get a really nice integrated environmentfor building your application.

CF:But it's not a user remote access, and I get an instance of a server where I'm running the control panel, and I'm doing all this GUI stuff. That's notit?

DN:No, it's not really focused on that. In fact, in a way, with platform as a service you're trying to [by]pass that. You're trying to reduce those extrasteps that [you] would have to do as an administrator. You try to avoid that and bypass that.

CF:I'm glad you said that because with so many people that's their comfort zone. You know, I log in, I'm remoting in, and I got the GUI, and I knowexactly what I'm doing. But if you think about it, that's just evil because the last thing you want is for some dialog box to pop up on your server, inyour server application that says oh, you have an error or there's some input that's needed at the GUI level.

DN:Yeah. That's definitely stuff you want to avoid. Now you can use PowerShell. I mean sys admins and IT pros, they actually can use Azure using acommand-line language, using PowerShell and all those, and that's actually kind of important because you have to appoint a large system where you havemore than one instance of your application. You do need to appoint them, and you're not going to do that just right from Visual Studio. You do need topass and then deploy multiple instances; you want to scale up using the platform. With Azure, you actually do these instances, which is kind ofinteresting because that's not typical with the platform. I mean they can do it for you, of course, but in many cases you need to deploy your stuff inPowerShell or something like that. It would help.

RC:I'm wondering how much of this stuff is still pretty manual-that you have to build your own scripts for deployment, your own test and evaluation of howwell your system is running. This sort-of dream of automated elasticity, I haven't seen it yet. I don't know that it actually exists. You've got to doit yourself.

DN:You know, I've seen some people demonstrate it, but I think [wrote] the code themselves. There are other companies I'm sure who are in a similar world,and there's more ecosystem, of course, in the Microsoft world. There are third-party full vendors who I'm sure will come up with something.

RC:The thing that's been bothering me the most is starting to see that migrating applications in a cloud is really tough.

DN:Yeah. Well, part of it is because with cloud computing you are scaling out horizontally. What that means is you're using some pretty standard hardwarethat's not necessarily like the supercomputers and the typical web server you might find in any just regular web-hosting company. And in order to scaleyour application out for this to be able to serve 1,000 or tens of thousands of people, you're adding more instances, and your application needs toknow how to use those instances. So you need some sort of load balancer in the front, and then the database is split out from the application. So ifyou're running an application that you typically would only install on one web server on some managed service provider or web hosting company, thatapplication needs to be modified, and it's not easy.

Think of it this way. Most applications are storing to local hard drive. Well, I mean in the cloud, you don't really have a local hard drive. You do,but if you save it on one instance of a local hard drive, then all the other instances don't know about it. Where is that going to store? Whichinstance is it going to store on? So we're going to have to store over in Azure's storage somewhere, and that's not the same code that you're using.It's different. And then the same thing goes to the database. Even though SQL Azure is pretty much the same thing as your regular SQL Server, theredefinitely are differences. There are the limitations that they have in order for it to scale in the cloud, and so not all of your SQL and yourconfigurations will work.

There's much more! You can find the full interview at dotnetrocks.com/default.aspx?showNum=669.

Richard Campbelland Carl Franklin (both Microsoft Regional Directors) are the voices and brains behind .NET Rocks! (www.dotnetrocks.com). Theyinterview experts to bring you insights into .NET technology and the state of software development. They're more than dry technical interviewers-theyhave fun!