Build a Black Box Testing Solution Using Visual Studio 2008

Every application includes some code that sooner or later will require a refactoring or an upgrade—maybe simply because the business requirements changed or perhaps because users report that a particular section of a website is too slow to respond. Whatever the reason for the change, it's important that some automated tests exist that allow a smooth refactoring or upgrade process. Black box testing is, in my opinion, an important factor to consider in such situations because it allows the aforementioned transition to occur painlessly.

In this article, I'll explain how to set up a continuous integration and build management system with JetBrains TeamCity 4.5 Professional Edition, Selenium Remote-Control (RC) 1.0 beta 2, NUnit 2.5, and Microsoft Visual Studio 2008. Used together, these products let you perform black box testing for an ASP.NET MVC application. I have been using Selenium RC for more than a year and, although still in beta version, it has proved to be stable. The testing environment that I will present is based on Microsoft operating systems; however, both TeamCity and Selenium RC run on other OSs as well.

Installing and Configuring TeamCity Professional
TeamCity Professional is a free system for up to 20 users that doesn't require any server license for trial or perpetual use. For purposes of this article, I used Windows Server 2008 R2 Release Candidate to run both TeamCity and Selenium RC. Also, before installing and starting to work with TeamCity, I downloaded and installed the latest version of Microsoft Visual Web Developer 2008 Express Edition SP1. This component is required by TeamCity for building the project (build management). However, it is also possible to exclusively use MSBuild, NaNT, or other equivalent products.

You can download TeamCity Professional at www.jetbrains.com/teamcity/index.html. Once the product is downloaded, simply install it by executing the .exe file. By default, during the setup process, TeamCity installs and configures an agent, which is the component that lets TeamCity connect to a repository, such as Subversion, build the application, and run the related unit tests against the latest committed code.

There is no need to further configure the agent. However, it might be a good idea to write down some information, such as the server's port, which is required later to manage the system. I also recommend saving the agent's configuration information, which is displayed at the end of the installation process. Also, the TeamCity's installation wizard suggests the user profile's path as the path for the server. A better approach, however, is to define a custom path that is not related to any specific user.

Once you've installed TeamCity Professional, you can start managing it by opening a web browser and typing the URL displayed at the end of the installation. In my example, the URL is http://localhost:8080/login.html. As soon as the browser is started with the default URL, TeamCity lets you create an administrative user for the system and create general settings for the project. When you reach the Version Control Settings screen, pay particular attention to creating and attaching TeamCity to the correct version control system. In this article, I'm using Subversion as the version control system; to connect to it I enter the information that Figure 1 shows.

The next step is to inform TeamCity which Build Runner to use (Build Runner enables TeamCity integration with a specific build tool.) For this article, I used Visual Web Developer 2008 Express—so I selected sln2008, which indicates Visual Studio 2008, and entered as the solution's pathname the location of the .sln file. I also selected NUnit 2.50 in the NUnit Test Settings section. After you save these settings and select Projects and the link to the project has been created, a screen similar to that in Figure 2 is displayed.

Other settings in TeamCity let you, for instance, create users and administrators of the system and configure TeamCity to send them email notifications. For example, when a compilation fails, an email message will be sent to all project members advising them that a compilation failure occurred. The same applies to errors encountered while running unit tests. I advise you to review the product's manual to learn about this and other useful settings available in TeamCity.

Recording with the Selenium RC Extension
Now that TeamCity is ready, the next step is to get familiar with Selenium RC and automatically record a test, through a Firefox extension. To do so, open Firefox, select Tools, then Add-ons, and click Get Add-ons. Now type "Selenium IDE", click Add to Firefox, and follow the on-screen instructions to finish installing the extension. Finally, restart the browser to enable the extension.

Configure the web application in Visual Studio 2008 to use the Visual Studio Development Server and ensure you start and browse it with Firefox. To do so, right-click the Default.aspx page of the web project, select Browse with, select Firefox, and click Browse. Once the first page is displayed, open the Selenium extension to start recording: Select Selenium IDE from the Tools menu in Firefox. From now on, the extension will record everything you do with the web application and display the generated commands. In the example provided in this article, the test consists of selecting a company and accounting year to manage, creating a transaction, and finally ensuring that the transaction is correctly displayed in the account's details page and the balance updated accordingly.

Once the recording has finished, click Export Test Case As and select C# - Selenium RC; doing so will create a file that contains C# code for the recorded test. This newly created test class will need to be included in a test project that also references some Selenium DLLs. For the purpose of this article, I added references to the NUnit and Selenium libraries. Figure 3 shows an excerpt of the entire unit test implementation.

Also, since the application uses AJAX requests to save the ledger entry, it's important to instruct the unit test to wait for the AJAX requests to complete. To do so, you can include the following line of code:

selenium.WaitForCondition("selenium.browserbot.getCurrentWindow().jQuery.active == 0; ", timeout.ToString())

This statement instructs Selenium to wait until all AJAX requests are completed. The timeout is used to ensure that Selenium doesn't indefinitely wait for the aforementioned to be true, as Figure 4 shows.

In this article, the IsTextPresent method is used to detect whether the application could correctly store the new ledger's entry. For our purposes, this method does its job. However, in other situations it might be more appropriate to check the generated HTML code—that is, by using the GetBodyText method. Additionally, you might want to extend the various tests using the generated HTML. In our example, there is no check that confirms whether or not the new ledger entry could be successfully saved, but in a real-world scenario, it would likely be important to know if this happens. In the example presented in this article, the test could be extended to look like the following: 

Debug.Assert(_selenium.IsTextPresent("31-12-2009Dtest selenium100.0000100.0000    (D)"), "the returned text is the following: " + _selenium.GetBodyText());

 By doing so, it would be easier and faster to analyze the result of TeamCity. In fact, should an error occur, instead of displaying only the expected message, TeamCity would also include the entire page's HTML code. In the case of a timeout, for example, you wouldn't need to look for the problem that caused the test to fail. If you have hundreds of unit tests, this approach could save quite a lot of time.

Downloading and Preparing Selenium RC
Now it's time to prepare Selenium RC to test the application. Go to the Selenium RC website, seleniumhq.org/download, and download the software (the file to download is called Selenium RC and appears in the projects list). Set up a directory locally called Selenium, copy the downloaded file, and extract its content. The directory structure should be as indicated in Figure 5. Note that it's important to keep the naming of files and directory as previously indicated, since the Windows service that we will set up later will use this data.

Create a new batch file under Selenium; name it StartSelenium.bat and add the following content:

 cd C:Seleniumselenium-remote-control-1.0selenium-server-1.0
java -jar selenium-server.jar -trustAllSSLCertificates

 This command will start Selenium RC, and the option -trustAllSSLCertificates forces the Selenium proxy to trust all SSL certificates. At the time of this writing, you can find the explanation of the various options at the Selenium Documentation page.

Coding a Windows Service to Automate the Process
Last, but not least, you need to code a small Windows service that will ensure Selenium RC is always up and running, even if the continuous integration machine is rebooted. This article assumes that you know how to properly set up a service and ensure that it can always be stopped and started. (There are also a number of online articles showing how to set up Selenium Server as a service; just search on "running selenium server as a service," and you'll find them.) You'll need to add code in the Windows service specifically to kill the Selenium RC's associated process. Figure 6 shows an extract of the service's code that performs the process's cleanup.

It's important to always restart the Selenium RC service as soon as the web application is deployed on the test web server; otherwise Selenium RC will not run against the latest application's version. You can further automate the process by writing some code that automatically restarts the Selenium service as soon as the unit tests start.

Think Inside the Box
I've showed you how to set up and write a simple test in a continuous integration scenario. As you've seen in this article, Selenium RC enables programmers to elaborate automated black box testing even for websites that use the very latest technologies, such as Microsoft ASP.NET MVC 1.0. By using the approach I've described, you, too, can start writing black box tests.

Michael Lusenti ([email protected]) is a software consultant based in Fribourg, Switzerland. He specializes in Microsoft .NET technologies and spends most of his time architecting and building various web solutions and software applications for large and medium-sized companies.