Beware the Cult of Numeracy
As companies grow I see a tendency to focus more and more on data. While data can be a fantastic tool to understand behavior and measure impact, we should be mindful of its limitations. Many important things in life and business can't be measured. A good related thought experiment:
Imagine living your personal life such that you never took (or were allowed to take) any action unless you could first quantify or measure its impact. Now you understand one of the pathologies affecting the tech industry.
How does something that seems so crazy in a personal context become the norm in the workplace? In short, we love tangible wins. This is especially true for high achievers. As Clay Christianson write in How Will You Measure Your Life:
High achievers want to optimize achievement. So they spend all their energy on near term tangible achievements. However, this can lead to negative long term consequences.
In his latest book he explores some of these long term consequences:
People who are driven to excel have this unconscious propensity to underinvest in their families and over-invest in their careers — even though intimate and loving relationships with their families are the most powerful and enduring source of happiness.
Bringing this back to business, the proliferation of data is putting more and more attention on short term and easy to measure metrics. In the short run this leads to stellar OKR updates. However, in the long run it often frustrates users, depletes trust, and reduces brand loyalty. This is especially true in competitive and fast changing environments where consumers flock towards simple and user friendly experiences.
Running 20 A/B tests to optimize button clicks may seems like a win. However, this hyper focused metric optimization attitude will more often than not degrade your experience over time and prevent you from the types of large scale innovations that lead to step function growth. As Evan Spiegel said in an interview with ReCode:
For me, I think the big red flag for all of us should be when we put more weight on things that can be counted instead of the things that can't be. Because the things that can't be counted are the things that make us human, and the things that are the most important to protect.
Data is not bad. In fact, data is a very useful tool to reduce uncertainty, validate hypotheses, and measure impact. However, we need to make sure we recognize its limitations. I love the way Andreas Eldh puts it in Human Driven Decisions:
Because data is sometimes great, we think it’s always great... [but N]umbers don’t show nuance, uncertainty or complexity. They give us an illusion of exactness. Data can tell us what is happening, but not why. Data can describe the problem, but not the solution.
The tail shouldn't wag the dog. And data, especially near term metrics, shouldn't be the main driver of our decisions. Rather, it should be one of several inputs alongside values, intuition, and empathy. Squarespace's Optimize Towards Ideals core value does a good job of articulating this balance [1]:
While metrics are critical for tracking and testing the performance of our business, they are merely a reflection of our ideas and execution in the market. Our values and ideals are our decision making guide.
Consumer expectations are rising. We expect packages to be delivered in days and digital experiences to be fast, simple and easy to use. While we shouldn't stop using data, we should acknowledge the unintended consequences of overemphasizing it.
So here's to recognizing the limitations of data and bringing our attention back to the nuances and intangibles that make us human. Less focus on metric optimization. More focus on understanding customer needs and delivering value.
[1] Between writing and publishing this post they have evolved this value. You can check out the updated version here.