Web Security Scanning: David vs. Goliath

It's glaringly obvious that Web technology is getting more popular by the minute. I think it's safe to assume that the majority of companies with one or more network-enabled products are looking at how they can leverage the Web to increase the revenues generated from those products--if they haven't already jumped into the Web services market.

Add to that lot the huge number of e-commerce sites, toss in all the new so-called Web 2.0 sites that are popping up, and we find ourselves at a new level of frenzy in technological evolution. Naturally, one issue on nearly everyone's mind is Web security.

"McAfee's Latest Acquisition: ScanAlert" tells about McAfee acquiring ScanAlert, a company that provides Web security scanning services to e-commerce sites. Those who pass muster get to display ScanAlert's logo, which helps builds consumer confidence. McAfee will pay around $51 million in cash for ScanAlert, if that gives you any idea how important the Web application security market has become.

Recently, I was made aware of an interesting comparative review conducted by Larry Suto, an application security consultant. Suto examined three commercially available Web application scanners: NT OBJECTives' NTOSpider (at the first URL below), Watchfire's AppScan (at the second URL), and SPI Dynamic's WebInspect (at the third URL). As you might know, NT OBJECTives is an independent company--"David" when compared to "Goliaths" Watchfire, which is owned by IBM, and SPI Dynamics, which is owned by HP.

http://ntobjectives.com/products/ntospider.php

http://www.watchfire.com/products/appscan/default.aspx

http://www.spidynamics.com/products/webinspect/

The review results stirred quite a bit of debate. According to Suto's test results, NTOSpider outperformed the two other tools by a country mile. NTOSpider crawled more links, found more vulnerabilities, and returned zero false positives in terms of vulnerability detection. In short, David beat Goliath quite convincingly.

Suto tested the scanners against three applications: a closed-source application, an open-source blogging platform, and an open-source customer management platform. He also used Fortify Software's Fortify Tracer (at the URL below) to determine how effective the scanners were in exercising the tested applications. Suto said that during his tests, "Each scanner was run in default mode and not tuned in any capacity to the application. The importance of this lies in how effective the default mode [is] so that scalability of scanning is not limited by manual intervention and setup procedures which can be very time consuming."

http://www.fortifysoftware.com/products/tracer/

That aspect of the tests is what stirred so much debate. Some people argued that fine-tuning each scanner before conducting tests would yield better results. Others agreed with Suto's methodology. Still others had a lot of "sacred cows" that skewed their perspective.

Regardless of the various opinions, the test results are useful. In addition to the real surprise--that NTOSpider beat both AppScan and WebInspect--the tests reveal once again (as other independent tests have shown) that AppScan is most likely better than WebInspect.

More good news for NTOSpider is that Veracode, which offers a Web security rating service (at the URL below), recently chose it as the company's tool of preference after closely examining many competing tools. Veracode was cofounded by Chris Wysopal, formerly at L0pht and @stake, aka "Weld Pond."

http://www.veracode.com

You can get a copy of Suto's entire report (in PDF format) over at ha.ckers.org. Be sure to read the comments at the site too (at the first URL below). You can also read a bit more at Jeremiah Grossman's blog, at the second URL below.

http://ha.ckers.org/blog/20071014/web-application-scanning-depth-statistics

http://jeremiahgrossman.blogspot.com/2007/10/best-web-application-vulnerability.html