Splunk 4.0

IT pros are constantly bombarded with information—we face a labyrinth of event data in the form of firewall, server, router, switch, and other log files every day. When something goes wrong, we must descend into that labyrinth, seeking the root cause of the problem. Splunk 4.0 collects, sorts, and correlates all that event data for you ahead of time, making your log file explorations faster and easier.

This product parses any and all types of log files or diagnostic output streams using an intelligent regular expression (regex) engine, and then it makes the output searchable from a web interface. Splunk input includes message queues, Windows event logs, registry hives, packet captures, intrusion detection alerts, UNIX syslogs, web access logs, Netflow streams, and more. Splunk parses timestamps and information fields for each source before organizing all of it in a searchable decentralized index. If Splunk makes a minor error, you can correct it by training its parsing engine using the built-in data input control panels. For more complicated inputs, you can use the configuration files, which allow for custom regex definitions.

It offers a comfortable, quick web interface that delivers simple yet powerful search-augmentation tools laid out in a straightforward workflow. Splunk’s query engine uses typeahead, search history, and its knowledge of parsed fields to help you construct a meaningful, accurate search. Key to the process, however, is the addition of iterative search terms that refine your results. These iterations leverage booleans, wildcards, and extracted fields to help narrow your data set. Events pop up that are physically near, and usually related to, the incident in question. Once you’ve determined the time frame for the problem, the key advantage of universal timestamping becomes apparent. This contextual vantage point gives you the ability to trace more complicated interactions back to their source, as Figure 1 shows.

Splunk’s price depends on the amount of data you want to index on a daily basis. Splunk Free lets you index up to 500MB per day, and Splunk Enterprise lets you index an unlimited amount of data per day, with licenses starting at $5,000. After investigating Splunk alternatives, I found that Splunk Enterprise’s price isn’t as exorbitant as I first thought, given the product’s target markets and the fact that its feature set appears to be more complete and cohesive than those of its competitors. Nonetheless, I do wish there were a price point tailored for data-rich small-to-midsized businesses (SMBs).

The Enterprise edition offers role-based access controls and enterprise dashboards, which let users share useful searches and reports with their team. The crown jewel of the Enterprise edition is its distributed searches, which are bolstered by load-balancing and failover mechanisms. Splunk Enterprise also lets you architect data flows from your data hubs (called “forwarders” in this role) so that they feed indexed data up a hierarchy at regular intervals. And the Enterprise edition offers sophisticated monitoring and alerting functionality.

Despite its power, Splunk’s interface has a few small cracks. There were some interface bugs, which were likely a result of the intricacies of maintaining a consistent experience on multiple web browsers and their associated OSs. These issues are primarily cosmetic, and rarely affect typical use, but they do underscore the importance of sticking to clean browser installs to reduce conflicts with Splunk’s otherwise slick interface.

Splunk pulls together disparate reports and unifies them in a clean, searchable manner. Although its pricing scheme could use a tier for SMBs, the product can still help you manage at least the core fraction of your IT data files. If you’re stuck in a virtual cell padded with the remains of unused or unusable records and log data, Splunk will help restore your sanity.