Applied Observability: Converting System Performance to Business Performance update from October 2022

Gartner identified Applied Observability as a Top Strategic Technology Trend for 2023. I penned down a few ideas "on the back of an envelope" as the co-author of the trend with Frances Karamouzis — who was the original brains behind it.

Gartner-trends

As organizations launch new digital products and services, the need for fine-grained visibility into product usage is important not only for operational support but also for customer success, product management and sales teams. Building observable systems helps both business and IT stakeholders understand what features customers care about the most, what they struggle with and how they can improve the service.

Observability also offers a new approach to managing business change within organizations — instead of monitoring business applications and processes, which is fundamentally reactive, observability relies on instrumenting processes with the necessary data and control mechanisms to enable proactive and preemptive actions. Applied observability combines telemetry data obtained across multiple systems to make data-driven business and IT decisions. IT leaders must go beyond using observability for system reliability and build an observable digital business.

Examples include:

Related:Observability vs. Monitoring: Who Needs Which When?

Context

Observability provides a measure of how well the internal state of a system can be inferred from its external output. Applied observability extends this principle and applies granular visibility across multiple technology domains such as applications, infrastructure, data, network, and security as well as business processes to provide valuable business insight. This ability to harness observability data enables increased innovation, improved resilience and enhanced customer adoption, engagement, and experience.

Historically, organizations found it difficult to either plan or learn from past failures due to insufficient data. However, we have an exact opposite problem today — it is not the lack but an overabundance of data that make it difficult to extract the right signals from noisy data. Applied observability is about aggregating, correlating, and analyzing observability data across multiple technology tiers and domains and making that data available for both business and technical roles to inform manual and autonomous decisions.

Extend Observability Benefits to Multiple Roles

Applied observability involves harnessing the value of observability data to meet the needs of different roles across business and IT. For example, application teams responsible for customer experience will benefit from customer observability data. Similarly, I&O teams responsible for managing service levels will need IT observability data such as events, logs, metrics, and traces. Likewise, finance departments managing operational expenses may want to know the reason behind increased cloud spend and have enriched data to make informed decisions.

Observability data is only as useful as its ability to convert knowledge of the health and performance of individual systems to overall business health and performance. Connecting individual parts using telemetry across all layers of the business and IT systems topology is key to realizing the full benefits of observability. The core observability domains that underpin this trend include:

Applied observability extends the use of observability to address business intelligence issues. For example, tracking customer adoption and sustained use of a newly released feature requires continuous observability into user experience and usage patterns. The ability to obtain answers to exploratory questions using observability data helps to create new knowledge and bridge knowledge gaps for both engineering and business stakeholders in the organization.

Using Observability at Strategic Points of Value

Observability data in and of itself has no value unless it augments decision making at different "points of value." Points of value can be viewed as the critical junctions in business workflows that affect service levels, user experience and business decisions. Therefore, it is crucial to ensure that observability data is easy to use across teams — those that are driving the change and others impacted by the change.

The increased complexity of distributed system architectures creates the need for end-to-end observability to ensure that systems are reliable and resilient. Distributed systems are characterized by multiple points of failure, complex web of interactions, and the inability to predict the impact of component failures on the system (as a whole). As consumers use digital services as the first and in many cases the only touchpoint to meet their needs, it dramatically increases the need for digital resilience. Optimizing resilience metrics such as MTTR and RTO requires accurate observability data for troubleshooting and root cause analysis.

Recommendations

Introduce observability as a core ingredient of application design in addition to practices such as observability-driven development. Much like security, integrate observability into the complete software development lifecycle.

Improve the user experience of digital products and services by instrumenting business workflows and applying observability to every layer in the technology stack to discover, learn, and improve the usage of IT services.

Upskill and equip teams with capabilities to design and architect for observability with a goal to converting system performance to business performance.

This article originally appeared on the Gartner Blog Network.