State of Observability 2024: Embracing OpenTelemetry for Better Insights

There's a lot of value to be gained by improving IT observability.

That's one of the core themes in the annual State of Observability report from Splunk, the cybersecurity and observability leader recently acquired by Cisco. The comprehensive study, conducted in collaboration with Enterprise Strategy Group (ESG), surveyed 1,850 IT professionals across 10 countries and 16 industries.

The report highlights the evolving role of observability in complex IT environments and its impact on organizational success.

Key findings from Splunk's 2024 State of Observability report include:

A key challenge that the report reveals is visibility across dispersed environments.

"Our report showed that only 24% of respondents can correlate all or almost all their data across their application environment," Patrick Lin, senior vice president and general manager of observability at Splunk, told ITPro Today. "To build a leading observability practice, that organization needs complete visibility across any environment and any stack, through shared insights across ITOps, engineering, and security teams to see how a problem originated."

Related:Observability Takes Center Stage as IT Braces for Technological Shifts

OpenTelemetry Adoption Sees a Spike

One of the biggest surprises in the 2024 report for Lin was the extent to which OpenTelemetry has been adopted. OpenTelemetry is an open source technology that is part of the Cloud Native Computing Foundation (CNCF). The project was formed via a merger of the OpenCensus and OpenTracing projects in 2019.

"We always believed that this was the best thing for the customers, and invested in it accordingly," Lin said. "Anecdotally we've seen a lot of interest in it, but to know that it's so well correlated with maturity and success - that was a pleasant surprise."

There are several reasons why OpenTelemetry has become such a critical part of observability. Lin noted that observability has become increasingly important over time, and leading organizations are realizing that being locked into their vendor's agents means they are now dependent on those agents for something that is critical to their business.

"OpenTelemetry puts the ownership of data back where it belongs — in the hands of the customers," he said. "OpenTelemetry is vendor-agnostic and is rapidly displacing vendor proprietary agents."

Related:When to Use OpenTelemetry and eBPF for Modern Observability

The majority of vendors have begun adopting OpenTelemetry, making it the industry standard for getting data from applications into observability systems, Lin added.

How to Become an IT Observability Leader

The report introduces a new maturity framework for observability practices, categorizing organizations into four stages: Beginning, Emerging, Evolving, and Leading. This classification helps understand the varying levels of observability sophistication across industries.

Leading organizations demonstrate significant advantages in issue resolution and code deployment. They can identify application problems within minutes or seconds of an outage, 2.8 times faster than beginning organizations. Moreover, 76% of leaders can deploy the majority of their application code on demand, compared with only 30% of beginners.

"To build a leading observability practice, it's about empowering your ITOps and engineering teams with the right tools and best practices to help them be more efficient and productive, and free up their time to focus more on product innovation," Lin said. "An organization can do this by ensuring they have a dedicated platform engineering team to empower ITOps and the engineering team to do what they do best — building and shipping code — rather than wrangling toolchains."

Another key component of a leading observability practice is leveraging AI to alleviate alert fatigue. Lin said alert fatigue is a real issue that ITOps and engineering teams face today, and the report found that AI and ML tools excel at detecting anomalies and correlating events, making them well-suited to addressing the issue of alert fatigue. AI helps to predict potential problems before they turn into customer incidents, and this helps ensure a seamless experience for customers.

Another important step in becoming an observability leader is recognizing and empowering OpenTelemetry champions within one's engineering teams. OpenTelemetry has numerous benefits — it gives organizations better control over their data, it prevents vendor lock-in, it can reduce observability costs, and more.

"However, our report showed respondents say implementation can be challenging, mainly due to lack of staff familiar with OpenTelemetry," Lin said. "To get ahead of this, we recommend mobilizing your existing workforce to learn more about OpenTelemetry to take advantage of all its benefits and, better yet, build an internal training program on it."