.NETRocks Conversations: Anders Hejlsberg on Languages and the Magic of LINQ

Editor’s note: Welcome to .NETRocks Conversations, excerpts of conversations from the .NET Rocks! weekly Internet audio talk show. Hosts Richard Campbell and Carl Franklin chat with a wide variety of .NET developer experts. This month's excerpt is from show 541, "Anders Hejlsberg blows our minds!"

Carl Franklin: I'm very, very excited to introduce Anders Hejlsberg to .NET Rocks! Anders is a Technical Fellow in the Developer division. He is an influential creator of development tools and programming languages. He is also the chief designer of the C# programming language and a key participant in the development of the Microsoft .NET framework. Welcome to .NET Rocks!, Anders.

Anders Hejlsberg: Thank you.

Franklin: Well, there's so much to talk about. I don't know really where to begin. Let me just ask you what you're doing these days first of all, and then maybe we can take it from there.

Hejlsberg: Day-to-day I still work as the chief architect of C#. We are just, this coming week here, shipping Visual Studio 2010, which includes C# 4.0, and that's something we've been working on for the last couple of years.

I also get involved in doing design, API design on the .NET Framework. I was deeply involved in the project to create LINQ, or Language Integrated Query, in C# 3.0 before that, and of course, that wasn't just a language thing. It was also in VB and also included an O/R Mapper and an XML API that I worked on. I sort of get to work on a bunch of different things. These days in addition here to C#, I also sort of oversee our, from a technical standpoint, our strategy for our managed programming languages—VB, and F#, and the IronPython, Ruby Dynamic Languages.

Franklin: What do you think about the language revolution as it's come about?

Hejlsberg: Well, I think the more the merrier, in a sense. New programming languages is how we move the state of the art forward, so I'm a big supporter of that. I think it's interesting to see how the way languages evolved has shifted over time and has changed over time because frameworks and IDEs have gotten so large that it is increasingly important to leverage what's already there. So we're seeing lots of new languages now being built on top of these big managed frameworks or platforms, be it Java or .NET, as opposed to being implemented in their own little world, and I think that's exciting. I mean, luckily we engineered .NET to be a multi-language platform, and that's certainly serving us well now.

Richard Campbell: I guess the question comes down to whether languages will actually get old living in a framework. With Delphi, with the competitors of Visual Basic, the old one, and it's kind of interesting that you're now working with VB, the new one as well, but we can see that Delphi is getting old and it's sort of have been left behind, although I think Embarcadero has announced that it's still out there, it just doesn't have the traction it used to. Is that even possible with .NET Framework, for languages to lose relevance?

Hejlsberg: Anything is possible, but COBOL is in many ways still relevant, and what has it been now, about 40 years, or is it 50? You know, it's yes, they will lose relevance eventually but, boy, it can take a good long time. You know, C++ is still very relevant. I think languages have evolved a lot slower than we all think sometimes and certainly a lot slower than the hardware evolution that we've seen over the past decades, so I think there are many good years left still in .NET and the languages on top.

Franklin: When I look at LINQ, I think it looks to most developers like magic. I mean, is it really simple, or is it incredibly complex, or is both? When I think about what you're doing behind there, I know you've got a parser for the language in a particular structure, and then are we just traversing big lists of lists of lists?  Is that really all we're doing? What's going on in there?

Hejlsberg: What's going on in there? Yes, yes. Well, so LINQ is, I feel, a wonderful pragmatic application of functional programming, actually. I think also that to me one of the exciting aspects of LINQ is that if you boil it down to its constituent parts, it's just some language extensions and some API. You know, we added in C# 3.0 maybe five or six extensions to the language, including Lambda expressions, extension methods, anonymous types, type inference, and a few others. But taken together, these language features make it possible to construct a new kind of API that previously wasn't possible. And so one API that we've built out of those is this thing called LINQ, which really is just an API, an API that composes together to form a query language. And on top of that, we then have a thin syntactic veneer in C# itself, the thing that allows you to write SQL-like queries where you say from C in customers where blah, blah, blah, select blah, blah, blah, but really the compiler just boils it down to calls to methods named where and select and order bind, group bind, and so forth, and they're just implemented as API.

Franklin: So without all the other extensions to C#, LINQ wouldn't be possible.

Hejlsberg: Right, right. Yet all of the things we added to C# are wonderfully applicable to other things. Like Lambda expressions by themselves are super useful for other stuff and extension methods, I mean we see people doing all sorts of interesting things with those and typing inference. Hey, that's just goodness, right. So they all sort of bear the hallmark of a good language extension. They are applicable in a multitude of different ways.

Franklin: And that's where it gets interesting, because you have LINQ to XML, you have LINQ to SQL.

Hejlsberg: Right.

Franklin: How long did it take for those teams for the light to go off and say, aha, we can use this?

Hejlsberg: Well, I think part of the thing that was interesting about the LINQ project was that it's expanded a bunch of different disciplines. To wean the language design, there was doing the, if you will, the theory behind LINQ or like doing the third execution and pipelines and whatever. Then there was the implementation of multiple different LINQ APIs, so LINQ to Object, LINQ to XML, LINQ to SQL, and all of them we did sort of together, and people worked on different aspects of it. Personally I ended up doing a lot of the coding on LINQ to XML and some of the coding on LINQ to Object, and Matt Warren, who works on the C# compiler, ended up doing the majority of work on LINQ to SQL.

Franklin: Yeah.

Hejlsberg: So we did both and that, I think, is what made it great. It was that we were in there on both sides of the fence, and we could have the implementation in form, the language design, and vice versa in a sort of very tight loop, if you will.

Campbell: I have to wonder if the LINQ to SQL part of this didn't sort of complicate the whole issue. I love your position on this because there have been LINQ to XML, LINQ to Objects, LINQ to anything that is a relational database.

Hejlsberg: Right.

Campbell: To me, it is incredibly compelling because all of those things have different querying methods that were really a pain.

Hejlsberg: Yup.

Campbell: And then LINQ to SQL is somehow different, that it is a relational database, and we ended up with a subset to some degrees, and there's always an issue with the DBA on the back end. How do you see this?

Hejlsberg: You know, I was always fascinated first of all with the impedance mismatch, if you will, between databases and general-purpose programming languages. And when I talk about LINQ, I would often say like I'm old enough to remember the good old days when data was easy….

Franklin: Say that again.

Hejlsberg: Where we have products like DBase and whatever where data was ephemeral, it was an integral part of the language that you were programming in.

Franklin: Or we could even write our own databases.

Hejlsberg: Sure. But then in the name of scalability and "enterprisiness," and, I don't know, consistency in transactions and what have you, we sort of fork the world, and some of it went into relational database land and some of it into sort of general-purpose programming languages that were more expressive and whatever. But we somehow lost the deep connection between the two and in a sense regress into talking to our database through the spunky query APIs, where you sort of have another programming language inside strings called SQL, that it doesn't get type-checked and by the way has a completely different type system and all sorts of just irritating things that you have to deal with and a strange asymmetry like, hey, I can query my data, but only if I put them in the database.

I got one side taken out of the database, I can't query it anymore. Why is that? How is that? How is that helpful? And so, I felt we could do a lot better, and so I think, you know, I understand how you can sort of view LINQ to SQL as being an outlier in that it's different than its relational database, but I think LINQ would have gone nowhere had it not been for LINQ to SQL. I mean, I don't think you can really talk about data without covering the relational domain.

There's much more: To listen to or read the full interview, go to www.dotnetrocks.com.


Richard Campbell and Carl Franklin (both Microsoft Regional Directors) are the voices and brains behind .NET Rocks! (www.dotnetrocks.com). They interview experts to bring you insights into .NET technology and the state of software development. They’re more than dry technical interviewers—they have fun!