Engineering Metrics: The Elephant in the Room

There's a story, probably apocryphal, about a royal leader who presented an elephant as a gift to those who had fallen out of favor. The unlucky recipients were left with a large and expensive problem. They knew they had a valuable animal, and they knew their health depended on the elephant's health. But the cost of feeding and caring for the creature exceeded the value that they could realize. Ultimately, they went bankrupt, surrounded by piles of … err, tech debt.

This story came to mind when I talked with a client recently. Their complaint was that "engineering" and "the business" didn't understand each other. Engineering was the elephant: bulky, expensive to keep, hard to move, and incomprehensible. Business was the poor beneficiary, not understanding how to create value from the incredible resource — they weren't even sure what they had.

Framing the problem this way is a recipe for failure. Engineering is part of the business, often a large part. Engineering leaders must recognize that they need the rest of the business to be healthy, or the elephant will starve. Leaders of the rest of the business must give clear direction to the elephant so that it's doing useful work for the organization and not just wandering off on its own.

Related:6 Software Development Trends to Watch in 2025

GenAI: Was That an Earthquake?

Developer productivity is on everyone's mind these days. We've all heard that generative AI promises to increase developer productivity by 40% or more. There's a lot of hype, and leaders need to cut through the hype to find reality.

Business leaders both inside and out of engineering have been coming to me with the same question these days: "How can we determine if our developers are more productive with genAI tools?" My response usually doesn't go over too well: "Take whatever you're using to measure productivity now, add genAI, and see if those measures go up."

The truth is that measuring developer productivity is hard. Back in 2003, Martin Fowler gave up, saying that "we have no way of reasonably measuring productivity." Metrics such as lines of code have always been meaningless, and they're even more meaningless when you can add a prompt like "make this twice as long."

Business Is What Matters

In many cases, metrics are a form of vanity. "Our team is DORA-elite" doesn't mean much if your customer doesn't want what you deliver or if you've got overwhelming turnover costs due to developer burnout. Just the act of measurement will change what happens at your organization, so a light — and balanced — touch is needed. Save the time and motion studies for processes that get repeated and automated.

Related:Top 10 Software Development Stories of 2024

The elephant knows how to lift the log and enjoys doing it. Team up with the elephant so you can both succeed. Create alignment — make sure that the elephant understands where the log needs to be — clear out the obstacles, and let the elephant figure out how to get it there. To learn how to do that, Forrester clients can connect with me or read my report, Your Focus On Developer Productivity Is Killing You.

— Andrew Cornwall, Senior Analyst

This article originally appeared on Forrester's Featured Blogs.