Testing, Testing 1 2 3

Concentrating on testing is one of the most challenging tasks for developers. I've lost count of the number of times a developer has raced into my office to tell me that a piece of software is done, only to race back out of my office when I ask, "Have you tested it?" However, that isn't as bad as the response, "Well I'm not sure this code can be tested before we send it to integration, because ...." The fact is most developers are lousy testers.

Don't get me wrong. I understand that testing isn't exactly the most exciting activity, especially since in the past, most developers accompanied each DLL or back-end component with a custom Windows Form that could act as a test harness. What occurs then is the creation of custom code to call the methods, set up the conditions, and ensure the code works correctly. Unfortunately, even this type of testing is only as good as the person writing the tests, so the axiom that most developers are lousy testers bears repeating.

I also understand that it takes a long time to develop tests and that most organizations fail to treat even automated test frames with the same level of source control as the code. Thus, unit testing is often something that is cut in each project. Most of the testing effort is instead focused on integration testing after the software is "complete." However, this all-too common approach results in applications spending too long in the first round of integration testing because bugs unrelated to integration must be resolved. And if the bugs aren't caught during integration testing, they might surface during maintenance--and a fix at that time might actually break an existing application.

Fortunately, software developers are beginning to implement better testing approaches. For starters, some developers now follow a test-based development model. Just as there are developers who start the development process with Use cases or Joint Application Design (JAD) sessions (which involve end users), these developers start by defining a set of tests, then use that test set to develop software. This description is an oversimplification of the process. If you want to learn more about this model, go to http://www.testdriven.com.

I subscribe to a different model in which software design includes, in most cases, test planning. I still find that most companies are surprised when I ask to have members of the testing team participate in late-stage design discussions. The idea is that testers in the room will be less interested in what a feature should do than how they'll determine whether it's broken. Thus, testers tend to ask a lot of "What if?" questions. When captured, these questions result in requirements and design specifications that are much more detailed. The testers aren't focusing on what features they want but rather what the exceptions will need to be.

Talking about testing is good, but using tools to help automate the process is even better. Some organizations have begun to develop automated test tools. A good example is Nunit. NUnit is freeware designed to automate the testing of .NET applications.

Microsoft has also developed automated test tools for testing its own applications before they're shipped. Most of these automated test tools are custom in-house applications that Microsoft has maintained over the years.

Because Microsoft performs unit testing on its own products, it isn't surprising that Microsoft's Team Edition for Developers (TED) product suite includes tools for unit testing. What might be a bit surprising is that TED's unit-testing tools are somewhat similar to those you'll find in NUnit. In fact, there's even a converter to take your NUnit tests and convert them into the Visual Studio Team System Test format.

To unit test an application in Visual Studio 2005 beta 2, you need to find a new menu called Test. On the Test menu is an option to create a new test. When you select this option, you receive a dialog box that looks amazingly similar to the New Project dialog box. It's similar because, under Team System, tests are collected in Test Projects.

I recommend that you start by using the Unit Test Wizard to automate the generation of most of your initial tests. The wizard is a great way for you as a developer to jumpstart your knowledge about automated testing. After answering a few questions, Visual Studio 2005 will create your new test project and populate it with a collection of test classes. These classes contain the code that fully exercises (i.e., tests) your project. Using this generated code as a starting point, you can then add custom test examples. Let me be clear--as with any generated code, the generated test code is generic. For your tests to have meaning and value, you need to customize them. A good place to learn more about the customization of tests is from the Microsoft Developer Network (MSDN).

Testing goes beyond just developing tests; it includes running tests and analyzing test results. With Visual Studio 2005's new test projects, you have windows that let you review a project's "code coverage" and review test results. "Code coverage" refers to the percentage of code that has actually been tested. In other words, it lets you know which code in your application has been exercised and which methods haven't been exercised. This metric is important because new features can be added to existing code and tests might not have been updated to account for the new methods. The fact is that with the Team System testing tools, you can get a lot of information about your application's readiness. A good resource for learning to leverage these test tools is the VSTS Quality Tools blog.

As I mentioned previously, most developers are lousy testers--and the best developers are avid testers. In this column, I've talked about unit testing, but that's not the only type of testing you'll need to perform on your application. In Visual Studio 2005 beta 2, you have access to all the Team System test tools, including tools for load testing. Of course, this will change after the beta period, when TED users will have access to only the unit test tool. I covered the load test tool, Application Center Test (ACT), in the column "ACT Up." The new version in the Team System has several new features that definitely deserve coverage. I'll cover those features in future column.