Five Strategies to Navigate the Black Box Problem of AIOps

By Anthony Cote, Senior Enterprise Product and Solutions Marketing Manager, NETSCOUT

By now, most are familiar with the old data science adage "garbage in, garbage out," which refers to the phenomenon where bad data fed into an AI system can lead to less-than-satisfying results. 

Unfortunately, most organizations only realize their data is bad after it has been integrated into their systems or when these become little more than unpredictable 'black boxes,' which produce unsatisfying results. And there are a lot of black boxes out there. 

Peering Into the Black Box

When an AI system becomes a 'black box,' its logic, analysis, and decisions are often erroneous and challenging for users—and even the developers who built the models—to understand. This lack of transparency is hazardous to organizations because it makes it difficult to trust the system's entire output, especially when the data used to train the model is flawed, biased, or incomplete. This can also create a single point of failure (SPOF) if an AI-driven process or platform misinterprets or overlooks critical issues, making the AI an organization-wide vulnerability.

Building prior knowledge into a machine learning (ML) model can make it more reliable. But figuring out what the model needs and how to include that isn't always straightforward. Models may do well with training data but can struggle with real-world data that’s noisy, incomplete, or outdated. Real-world data is rarely as clean or consistent as in controlled training environments. This can cause big problems for sophisticated AI technologies, including inefficient automation and inaccurate predictions.

Related:The Impact and Future of AI in Financial Services

For example, an AIOps and cybersecurity platform relying on incomplete logs might misinterpret a credential-stuffing attack as a normal server performance increase that typically happens during a seasonal traffic spike. This could potentially allow attackers to compromise sensitive systems undetected. Alternatively, suppose an automated AI program analyzes fragmented data from departments like IT, finance, and customer service. In that case, it might produce conflicting automated actions, such as different automated responses to customer complaints. This would make the AI program hard to trust, turning it into a black box—where outcomes are more likely to be questioned than followed.

Unboxing AIOps Platforms, Automation, and High-fidelity Data

AIOps and cybersecurity platforms, including those from Cisco, ServiceNow, Palo Alto Networks, and Splunk, rely heavily on advanced AI and ML algorithms to analyze machine-generated data and make predictions and recommendations. AI relies on large datasets, making the quality of data provided to these platforms crucial. High-fidelity AI-ready data—accurate, trustworthy, clean, and granular—enhances the ability of AIOps and cybersecurity platforms to deliver precise and actionable insights.

Related:Artificial General Intelligence: Are We There Yet?

False positives are very common when companies rely on less granular, accurate, and clean data to run their AI models, especially those using deep learning algorithms, which have complex layers that make it hard to see how decisions are made.

To avoid the black box problem, organizations should consider adding a few strategies to their AI roadmaps, whether they are using third-party AIOps and security platforms or building custom AI systems, including:

Related:AI and the Green Market Revolution Will Intertwine

By adopting these strategies, organizations can bring more transparency and structure to their AI-based initiatives. This can help them avoid the problem of “garbage in, garbage out,” reduce false positives, and create more reliable outputs from their AI models.

About the Author

Anthony Cote is Senior Enterprise Product and Solutions Marketing Manager at NETSCOUT.