Enterprise Search Being Replaced by More Effective Cognitive Search

Enterprise search is getting a major boost in performance as the outmoded technology is being overshadowed by newer, faster and much more efficient methods known today as "cognitive search."

That's one of key highlights of a new 12-page report from analyst firm Forrester Research, which concludes that enterprise search is no longer the most effective method for businesses to organize and retrieve their huge stores of data. The shortcoming of traditional enterprise search, the report states, is that it is still based on an old-fashioned process known as "indexing," which uses keywords to find needed data when it is needed.

Instead, Mike Gualtieri, an analyst with Forrester and the author of the report, "The Forrester Wave: Cognitive Search and Knowledge Discovery Solutions, Q2 2017," told ITPro that cognitive search capabilities from companies like Attivio, HPE and Sinequa is bringing new energy to business search by using natural language processing and machine learning to vastly improve and accelerate how information is sorted and evaluated.

Cognitive search is still a developing technology and market, according to Gualtieri, but it offers the promise of much faster searches of much larger amounts of business data for users. When indexing was used alone in the past, the searches weren't as relevant as they needed to be, he explained. In response, research led to the introduction of natural language processing and machine learning – two important artificial intelligence tools – into the mix to help accomplish faster and better search results, he said.

An example of this idea are technologies like Amazon's Echo or Apple's Siri, which respond using AI when users ask a question or give other inputs, he said. "Think of it as that answer back is being powered by search."

In the same way, cognitive search will quickly help enterprises get similar fast and responsive results to searches of their documents, databases and applications, said Gualtieri. 

"Knowledge workers are often wasting time trying to find what they need," he said, with more than 54 percent of global information workers reporting they are interrupted from their work several times a month or more as they try to find and retrieve needed information within their companies and files.  

Already cognitive search is being used in enterprises, including life sciences companies which are building search into research science applications and large banks that are using the technology to watch for money laundering operations inside large datasets of customer transactions, according to Gualtieri. 

"Search can be a productivity tool for very specific business processes," he said. In the cognitive search marketplace, progress is evolving quickly as companies work to take leadership roles and add more capabilities for users.

"In the enterprise, where the typical search engine is just based solely on indexing, it's terrible" getting the best results today because keywords are no longer enough to provide adequate answers, he said. "Everyone wants great search like Google, but you can't get that with traditional search."

That's where cognitive search comes in, bringing benefits of more analytical searches by looking not just at individual words, but also at how the words are strung together and the deeper meanings they have when grouped together in sentences.

"Enterprises need to use these advanced techniques like natural language processing to figure that out," he explained. "Look at a sentence on a written page. Natural language processing looks at them not individually but together, so it can see what is being discussed. It uses a combination of statistical and some machine learning methods to extract meaning from the sentence."

With that accomplished, cognitive search can then extract a specific topic and can look for it instead of just seeking keywords. 

The leaders and strong performers in the marketplace today, including Attivio, HPE, Sinequa, Coveo, IBM and Mindbreeze, saw the value of using natural language processing to improve enterprise search, giving them a head start on other competitors, he said. Google has been using machine learning in its search for several years.

Interestingly, two open source contenders in the enterprise search market, Elastic.co and Lucidworks, which offers support for Apache Solr, are both oriented toward indexing very large data sets, but are not very good at natural language processing and machine learning, according to Gualtieri. "So enterprises have to be a little cautious" even as they look to cut costs by using open source applications. "This is one of the few times when the commercial vendors have out-innovated the open source community."