Parasoft SOAtest 5.0

asp:review

Parasoft SOAtest 5.0

A (Mostly) Fantastic Tool for Testing Web Services

By Matt Dinovo

One aspect of testing that is frequently overlooked is thetesting of Web services. Lots of folks test Web services in the context ofbusiness operations, but not many test the raw service itself. This should be acritical part of performance testing any service oriented-based application(SOA), as it s vitally important to know how the service fa ade reacts undermalformed messages, intense load, and other conditions.

I have employed various tools even going so far as toroll my own to fire messages at service implementations both in a functionaland load scenario. When I was presented with the opportunity to review SOAtest5.0 from Parasoft, I was eager to find out if, finally, a tool exists that wasflexible enough to handle the multiple testing scenarios I mentioned above. Theanswer is ... almost.

Before I get to that, let me describe some of the featuresof SOAtest 5.0. SOAtest s primary function is to test Web services in a varietyof ways: it can emulate the client or the server (by stubbing out services);validate WSDL documents; act as a unit, functional, or load test agent; andmore all in an easy to understand user interface. Additionally, SOAtest hasrich test-case management capabilities. Individual tests can be grouped intotest suites that can be executed in sequence, allowing for various scenarios tobe validated. Each test can have separate methods for determining inputparameters and validating responses. When a test is executed, either by itselfor as part of a test suite, any failures are clearly identified in the lowerpane for analysis and remediation (see Figure 1).

Figure 1: The SOAtest test executioninterface.

SOAtest is very strong in interpreting WSDL documents andprovides several mechanisms to assemble requests based on WSDL. Once a WSDL isinterpreted by the tool, you can provide input to the discovered Web servicesby using raw XML, either entered via the UI or from a file, or by using adynamically generated user interface to interactively enter the inputparameters for the request (see Figure 2). In addition, SOAtest supports databound parameters in a variety of formats including a built-in spreadsheetparadigm to allow for pseudo-randomly generated inputs at execution time.

Figure 2: The dynamically generatedparameter input screen.

One of the greatest capabilities of SOAtest with regardsto test suites is the ability to extract values from service responses viaXPATH, then use the result as inputs in subsequent service calls. In this way,a typical session from a client making several service calls in a row can beelegantly represented in the tool. Test suites can contain many different typesof tests; however, for the .NET developer, SOAP client tests will be the mostused test type. Responses can be validated, as well, and SOAtest providesvalidation wizards to validate individual parameters (via static comparisons orregular expression matching), the entire message, or the structure of theresponse.

Above functional testing, all tests can be executed in a loadtesting scenario. This functions in much the same way that load tests areexecuted in Visual Studio Team Test. You select the tests and/or scenarios youwant to execute as part of the load test and then select the performancecounters you want to capture as part of the test execution. During execution,SOAtest will capture the request/response metrics from the load test, as wellas any performance metrics attached to the execution, and present the resultsin a unified view. SOAtest can simulate 100 virtual users on a single client.

WCF and SOAtest

As part of my review, I created several differentservices: traditional ASMX Web services and WCF services using basicHttpBindingand wsHttpBinding. I expected ASMX Web services to work flawlessly, and that sexactly what I experienced. When I flipped over to WCF services, the experiencechanged somewhat. Basic HTTP binding (SOAP 1.2) worked as expected, but servicesexposed through wsHttpBinding did not. I tried several permutations to attemptto get SOAtest to successfully send requests to WCF services exposed viawsHttpBinding, but to no avail. SOAtest was able to interpret the WSDLcorrectly and present me with an interface to set parameters for the request,but the actual execution of the request failed. wsHttpBinding supports more WS*standards than SOAtest, and I believe this is where the incompatibility lies.While this was disappointing, it was only a minor inconvenience I could stilltest WCF services using basicHttpBinding and a suitable workaround was to usemultiple endpoints or to switch the binding when you deploy.

Conclusion

SOAtest is an impressive tool for testing Web services ina variety of capacities. The product boasts an impressive list of capabilities.However, because it is a Java-based application, I couldn t test certainfeatures (but those were few). Over and above the everyday testing scenarios,SOAtest also allows for the establishment of policies that act as a kind ofFxCop for WSDLs, penetration testing for security considerations, and even QoSand health reporting. Parasoft has committed to increasing SOAtest s .NETsupport, so I hope the issues of incompatibility with some WCF HTTP bindingswill be resolved.

Having done performance testing on SOA applications, Itruly appreciate the difficulty in being able to test only the serviceinterfaces to see how they impact performance. SOAtest makes this task veryeasy as long as you keep things simple from a standards perspective (at leastfor .NET developers). If you are using ASMX or basic WCF services, you owe itto yourself to give SOAtest a try to see how it can help increase the qualityof your system. You can try out SOAtest by downloading an evaluation version(88MB) from the Parasoft Web site.

Matt Dinovo is aSenior Solution Developer at Avanade Inc., a Seattle-based integrator forMicrosoft technology that is a joint venture between Accenture Ltd. andMicrosoft. You can reach him on his blog at mailto:mattdinovo.spaces.live.com.

Rating:

Web Site: http://www.parasoft.com/jsp/products/home.jsp?product=SOAP

Price: EnterpriseEdition, desktop licenses, and training for a team of five costs around US$50,000.