Hillel Cooperman and Tjeerd Hoek Talk Longhorn (Part Two)

Back to Part One...

Paul: In terms of the user interface, how do you balance subtlety with learnability? The obvious example is Paul's favorite example, the Mac OS X "jumping icon" on the Dock [where an application that wants to notify you causes its Dock icon to jump up and down, indefinitely, until you address it]. Tell me when this gets annoying [bounces hand up and down in front of his face].

Hillel: Ah, this is like the guy who made the video ... "Jack - f#$%ing terrier"!

[Laughter]

Paul: Exactly. But if you make it too subtle, will a less experienced user even notice it?

Tjeerd: Yeah, it's a very hard trick. We have to think a lot about the types of things we want to put on the screen, and what can go into the peripheral vision areas. We get into discussions about features: How important is this, really? What's the failure scenario? Even with things like toast [the MSN-style alert windows that appear in the lower right corner of the screen in Windows], there are issues about how you want it to appear, what it will look like, and when it will disappear. Some things you can test--you can actually bring them into a lab--and some things you have to do by experience.

Hillel: Have you seen our user experience labs?

Paul: No.

Tjeerd: Microsoft as a company has made a huge investment in user experience labs.

[At this point, Hillel demonstrates his group's internal tracking system for Longhorn user experiences. The demo highlighted the success rate of every single task in a photo scenario, measuring how the company is doing in each Longhorn build, and was amazingly detailed. For obvious reasons, I can't say much more about it right now.]

Hillel: This is us actually monitoring the quality of every single user experience scenario we care about, with every key milestone of the build. The way we measure this is to walk through every single thing, every screen ... we're a little crazy about this ... but we have to be.

Paul: You guys really don't get enough credit for the amount of work you do in this area.

Hillel: Well, that's because we don't tell people. In the past, the way we measured these things was to measure discoverability, task completion, and time on task. But that didn't really get as deeply as we needed to in order to understand how people feel about our products.

Tjeerd: Also we would test things in isolation. Like a single dialog: How is it functioning?

Hillel: Right, so what you saw before isn't just about acquiring pictures. Who the heck just wants to acquire pictures? You want to organize and manage them, you want to print, you want to share, and you want to post them on a Web site. You have to measure the thing in total to understand the end-to-end scenario. And these are the things we started doing in XP and will really expand on hugely in Longhorn.

Tjeerd: With this tool, we can see the quality of the product, broadly, as it applies to these scenarios. As a feature is designed, we do very targeted testing, usually with a prototype that we build in Flash, HTML, whatever. We do very targeted tests and produce deep reports detailing how these features might work.

Hillel: The difficulty for us is that the more [information] goes on the screen, the less simple it feels. We can prove to people that we're easier to use in many cases than anyone else, but yet it doesn't always feel that way. It's a very difficult trade-off to make. For Microsoft, it's in our DNA to say, look, I don't care how it feels, look at the numbers. We're getting 9 out of 10 users [completing tasks successfully] and they're getting 5 out of 10 users. Who's better? That said, we gotta be a little more... We gotta do both. It's not good enough for us to say "We're 9 out of 10 and you're 5 out of 10." We gotta be able to say that we're 9 out of 10 or we're 10 out of 10, and by the way, you know what else, our users feel great about using it.

Tjeerd: That gets back a little bit to what you were mentioning earlier. When you use a wizard, you make it really easy for the people who are unfamiliar with something. Whereas if people are completely familiar with something, and they do this thing every single day, [you want to get out of the way]. In the past, we had often bogged down the latter folks with the easy-to-use UI that steps you through it, and guides you and tells you all about it. People are saying, "I know this, I know this ... I burn CDs every day; don't keep telling me how to do it." And so we're trying to do a better job now.

Paul: This is the Mac myth. The Mac is supposedly easier to use, but in reality it's only easy to use when you already know how to do everything.

[Laughter]

Hillel: Well, everything is easier when you know how to do it!

Tjeerd: [Apple is] great at enabling the optimal scenario, enabling the optimal path. But as soon as you deviate or you have some problems, it gets a little harder. You start seeing people fail.

Paul: Right.

Tjeerd: We've done usability testing and we know, for example, that, yes, [Apple's] UI is very clean and simple, but even the most basic things sometimes are really hard for people to actually discover. It's almost like a game, you know?

Paul: Yep. There is no discoverability in Mac OS X at all. That's just a fact.

Hillel: No.

[At this point, Hillel invites Paul to visit Microsoft's campus-based usability labs in action. I'm hoping to write about that visit sometime this spring.]

Hillel: It's very exciting stuff.

Tjeerd: There are many components to usability.

Hillel: Absolutely. In our research team, we have the highest concentration of PhDs and master degrees [when compared to the wider User Experience team or Windows division].

Paul: This came up at the PDC. There is a lot of stuff you are doing [on the User Experience team] that people just don't know about. It's probably pretty accurate to say, if you go back before XP, that things were just kind of slapped together. You get the idea that whoever was working on Windows back then said, "hey, this dialog works, we're done."

Hillel: Right. But I want to caution you about that. The system definitely wasn't as involved as it is now, and frankly, since we have yet to ship, and prove that what we're saying makes sense, I'd be loathe to claim credit for anything beyond what [Windows] used to do. That said, remember that in most of the industry, there are no resources for thoughts around testing ease-of-use. In the fact that, pre-XP, Microsoft had been making investments in usability for ten years.

Look, we're the number one kid on the block. If that means we get a target painted on our back, that's OK, we'll bear that burden and not complain. But a surprising amount of thought went into [the user interface in pre-XP versions of Windows]. I was shocked when I came here because I was a Mac guy. And I was assuming that everyone up here was dopey, or didn't care.

Tjeerd: The problems seemed simple.

Hillel: Right. Actually, that was the shocking thing about coming here: The problems weren't simple. The number of people that I met who were humble, that cared deeply and passionately about making the user experience better for customers and weren't just saying, "Hey, here's a new API"--not that I don't love a new API--was surprising. These people were really deeply concerned about how they could make customers' lives better and had thought a lot about this, way more than I had. I was blown away. I just couldn't believe it. Part of the reason I never understood Microsoft before was that I was a Mac guy. It's a very self-referential community, in a good way. It's fun; you're doing the Mac thing.

Now I get in trouble [at Microsoft] for saying we're bad at anything, but I'll just say, we're very bad at telling this story. We're very bad at telling the story about what we do.

Paul: Oh, I agree totally. And I can help with that.

Hillel: I appreciate that.

Paul: People are fascinated by this stuff. This time last year I came out to the campus for a bunch of meetings with the Windows Server guys and talked with some of the folks who were involved with NT in the early days. And people are just amazed to discover that way more went into this stuff than they ever would have imagined.

Hillel: We make it hard on ourselves because our style is not to push a single personality as the genius behind all of it.

Paul: Are you sure about that?

[Laughter]

Hillel: No, when it comes to the UI ... Look, we certainly have a single personality when it comes to the guy that is running the company. But even there, there are a lot of people on stage during keynotes, and it's not just people doing a demo for Bill Gates. I mean, that was my job, but ...

I'm talking about, from the UI perspective, this is a real team effort. The bench that we have around the UI is so exciting, but you're only seeing two of us today. When you come back in April, you have to meet everyone else.

Here's the truth. The reason we've never been great at telling this story is that ultimately, if we have to choose between making it as great a product as possible and getting the story out, we'll always choose the former. We don't really care about the credit. We've only started to care recently because we've realized that it sets the tone for what users expect from the product. So it's not so much that we really care about getting credit, but if we're going to talk about what we're trying to accomplish, the credit goes to a broad group of people.

Paul: Honestly, even with XP, the perception in the wider world is that Microsoft isn't looking at feedback, that they're just shoving this thing in our face. It's actually even more negative than that. People don't just think you don't think about it, they think you ...

Hillel: Oh, they think about it. They think we're screwing them.

Paul: Exactly. Hey, let's change the location of everything in this release. It's funny. The notion that some thought actually went into the design of these products, and its way more than they could have possibly have imagined, is to me what's interesting.

Greg: You know we had these conversations. And we decided it just wasn't interesting cover. We just never tried to tell the story. You're unique in the sense that ... not everyone gives us the same credit.

Hillel: Let's put it this way. We try, but we can do better. We can agree on that. And I think we will do better this time [with Longhorn] getting out way ahead, and being more open about what we're doing than ever before.

Paul: How come you're gradually moving into a fetal position as we talk?

[Laughter]

Paul: Freaky ex-Mac guy running the ...

[Laughter]

Hillel: We care about getting the word out to that people say, wow, some real thought went into this, and I recognize that Microsoft's trying to make my life better, and they're listening. And now they can get involved earlier in the process, and tell us what they think. We turn away no opinion.

Paul: People are so resistant to the notion ... actually, I have a term for this; I call it "general knowledge." Its things that aren't true, but everyone believes them to be true. People are very to the notion that the Windows UI is in any way helpful or works better than the competition. I wrote this article about iterative UI in Windows XP, and it's interesting. Every once in a while I'll actually get through to somebody. But it's so hard to overcome this widely held perception. It's frustrating to me. Not because I'm a Microsoft sycophant as is so often charged, but because I know what's really going on.

Hillel: The reason we're going to get names associated with work now, and faces, and I'd like to get as many as possible out there, is because I've found the only way to combat [those misconceptions] is to meet us. That's the only way to do it.

Paul: Exactly.

Hillel: I meet people on planes, and I talk to them. And they're like, wow, you're so nice, you couldn't possibly work for Microsoft. You're not as evil as I'd imagined. You're not crazy or arrogant. That's why I'm excited to get the names and faces of as many people on the team out there as possible. Because you've got the two of us, but there's a whole bunch more back at home in the other building, who are busy working right now.

[Plans are made for the next campus visit, which will include a group session with the User Experience team, a tour, and so on.]

Paul: OK, so what's happened since the PDC?

Hillel: We're in this funny position, because we came out so early... Honestly, I feel a little empty-handed here. I wish I had more new stuff to share with you, but I don't. But I will confess to that, one of the things I love about Microsoft is that it's the most self-flagellating company I've ever seen. The people here are just ... I think we overdo it a bit, actually. We're very down on ourselves all the time. Oh, we suck... Oh...

We had this meeting a couple of weeks ago with our team, and I was like, is there any way we can reflect on some of the good things we've done, before we go back to beating ourselves up? And once in a while, we try to take a moment to remind ourselves we're OK. Could we be more efficient? Absolutely. Could we do things a little faster? I sure hope so, because there's gotta be a faster way. But once in a while we do need to remind ourselves that the thing we're doing is very difficult. And I don't say that to pat ourselves on the back, or to exalt our efforts, but because, frankly, it's challenging. It's because it's the Deep Think. We could do the shallow UI much faster. And I think we've done it. I don't mean "shallow" in a bad way. Go look at MSN Explorer. We shipped those things in 9 to 12 months, boom-boom-boom. All of a sudden, you had a brand new shell for MSN. We're proud of it, and think it's very cool. It's a beautiful UI, it's simple, and discoverable, and all the right things. But that was this self-enclosed MSN thing. We're doing the Windows thing now. It's ... man. We're hopefully realistic, but we have high aspirations.

So that's why we really don't have anything new for you per se, we've been working on making all that stuff you saw [at the PDC] real.

Tjeerd: What we showed at the PDC was real, of course. We've always done a lot of design work, coding, and so forth that still hasn't gone into the product yet. What we showed at the PDC was real, but it was new to a lot of people internally too. A lot of what we're working on doesn't make it into the wider builds. It doesn't make it outside of our team.

Paul: Yeah, you guys have gotten a lot better with that, which I don't appreciate by the way.

[Laughter]

Greg: We really got out ahead this time, in terms of disclosure. And maybe right or wrong we set an expectation that continuing that level of disclosure. Maybe we could have done a better job of setting expectations.

Paul: Well, you did this so far in advance for good reasons. But suddenly everything we have now sucks. And we want more.

Greg: That's the other edge of the sword. You want to make it better, you want to give the industry a platform to build off of. The difference between the MSN shell and Longhorn is that, in Windows, you have all the other parts that need to build off of it, or rely on it. MSN is just an application.

[Hillel displays a recap video highlighting the design work that led up to the Longhorn user experience Microsoft briefly showed off at the PDC.]

Tjeerd: Most people just see the end result; they never see the work that went into it. Even internally we have that problem.

Hillel: It still has a ways to go, and it's not close to final.

Tjeerd: We have this process, where we initially define what it is we're trying to do with the design.

Hillel: About a year ago, the design team did heavy aesthetics investigations. There were thousands of images made to try and figure out what the direction was that we would take. Ultimately, we wanted something that felt would be appropriate for a couple of years from then, because we knew it wasn't shipping any time soon. So we had to define state of the art.

Tjeerd: We're really trying to hit the professionally designed, beautiful aspect... And we're very specifically designing for very high resolution, wide aspect ratio monitors, with incredible quality.

Hillel: At first, our user experience explorations were very wide, but once we picked a direction, we started to go very deep.

Tjeerd: The largest part of the product design thinks about how thing work, what's the flow, and then we have a number of people thinking specifically about look and feel. The early work was all static--screenshots. Then there's a whole separate effort around how these things move, what kind of animations are really appropriate, what do we want the movement to say to the user? There are functional aspects to shared relationships in terms of our brand, or the emotive aspects, and the user experience. We have prototypes that really get into the animations, where a lot of the feel comes to play. You really get the sense that you're interacting with these things onscreen.

Paul: Now I have the same queasy feeling I had at the PDC.

Hillel: What's the queasy feeling?

Paul: The stuff that's coming is very interesting, and this renders what I'm using now less interesting.

Hillel: Well, that's part of the reason we're loathe to really get it out there. For most people--there are still a ton of people running Windows 9x today. We feel like XP is such a huge improvement compared to that, and we want them to move to XP before we hit them with this stuff.

There's nothing we'd like better than to just go nuts telling you every detail about every single thing we're doing, get your opinion, get your feedback, all that stuff. It's just that...

Paul: Greg's here...

[Laughter]

Hillel: You think if Greg leaves the room he doesn't have cameras, microphones?

Greg: There's a ridiculous number of people still running Windows 98 that have no idea how much better Windows XP is, that suffer through stuff every day that we already fixed in XP. And we're still trying to tell that story.

Hillel: We don't want them to miss that step of enjoyment.

Tjeerd: But I know your frustration because part of what motivates us as well is that, no matter how good XP is, there are still so many holes, or places where you just realize it could be so much better.

Paul: We've also settled into a nice evolutionary path over the last ten years. I was an Amiga user and I was never excited about Windows until Windows 95. That was the first Windows version that I thought was worth even thinking about.

Tjeerd: I was a Commodore 64 user...

Hillel: I had an Atari 800.

Paul: We can have that debate later, but I had a C64 as well...

[Laughter. Conversation predictably moves off-topic to 80's computer systems.]

Tjeerd: What I was going to say was that, XP is so great in its underpinnings and its foundation, and there are good interactions in it, and we did a whole bunch of things very well. But in terms of how the product really comes across in its aesthetics, and how well-integrated and carefully produced it looks, it just doesn't hit the quality bar. And so when you see these pictures [of the Longhorn UI prototypes in the video], you say, yeah, that's the kind of quality I expect from the best operating system in the world. That's really what I think this thing should look like.

Hillel: Although, again, compared to Win9x, XP is beautiful.

Tjeerd: Yeah. Right.

Hillel: XP was the first ... XP made me stop using my Mac. Right? I worked on Windows for a while here, but I still had both Mac and Windows at home. I was using both. I did all my photos and stuff on my Mac, and I did other stuff on my PC. But my PC was just faster. And then XP came out and I was like, wow. And I stopped using my Mac. That was it.

Tjeerd: You're not the only one.

Paul: I know a lot of Mac users who gave it up when XP came out.

Tjeerd: Look, [Apple] did a great job on OS X. They did some good stuff there. We like the ... competition.

[Laughter]

[Talk turns back to the annoying "bouncing OS X dock icon" effect.]

Tjeerd: The whole [Mac OS X] dock thing is just too toyish to me ...

Paul: Well, even XP was going to have a similar effect. The taskbar buttons were going to flash an orange color forever until you dealt with them. But during testing, people complained that it was too annoying, so now it just flashes three times and stops.

Hillel: It's so funny, we get a little myopic because we want to make sure the user can see it, and we don't want them to miss an opportunity, and we don't want to get in a bad place, and ... you gotta stop being so paternalistic. You can't overprotect. Everyone's a grownup here. Give them the tools, let them do their own thing. It's a hard balance.

Back to Part One...