Everyone says they live for metrics, but are your metrics really helping you focus on the right things? Do you use metrics just to brag, or to fill out slides on your manager's presentation deque? Did you use your metrics to make a decision today? What is the difference between good and bad metrics anyway?
In Lean Analytics, Croll and Yoskovitz spell out some rules of thumb for what makes a good metric:
- A good metric is comparative. Being able to compare a metric to other time periods, groups of users, or competitors helps you understand which way things are moving. "Increased conversion from last week" is more meaningful than "2% conversion".
- A good metric is understandable. If people can't remember it and discuss it, it's much harder to turn a change in the data into a change in the culture.
- A good metric is a ratio or a rate. Accountants and financial analysts have several ratios they look at to understand, at a glance, the fundamental health of a company. You need some, too.
There are several reasons ratios tend to be the best metrics:
- Ratios are easier to act on. Think about driving a car. Distance travelled is informational. But speed - distance per hour - is something you can act on, because it tells you about your current state, and whether you need to go faster or slower to get to your destination on time.
- Ratios are inherently comparitive. If you compare a daily metric to the same metric over a month, you'll see whether your're looking at a sudden spike or a long-term trend. In a car, speed is one metric, but speed right now over average speed this hour shows you a lot about whether you're accelerating or slowing down.
- Ratios are also good for comparing facts that are somehow opposed, or for which there's an inherent tension. In a car, this might be distance covered divided by traffic tickets. The faster you drive, the more distance you cover - but the more tickets you get. This ratio might suggest whether or not you should be breaking the speed limit.
- A good metric changes the way you behave. This is by far the most important criterion for a metric: what will you do differently based on changes in the metric?
- "Accounting" metrics like daily sales revenue, when entered into your spreadsheet, need to make your predictions more accurate. These metrics form the basis of The Lean Startup's innovation accounting, showing you how close you are to an ideal model and whether your actual results are converging on your business plan.
- "Experimental" metrics, like the results of a test, help you to optimize the product, pricing, or market. Changes in these metrics will significantly change your behavior. Agree on what that change will be before you collect the data: if the pink website generates more revenue than the alternative, you're going pink; if more than half your respondents say they won't pay for a feature, don't build it; if your curated MVP doesn't increase order size by 30%, try something else.
If you want to choose the right metrics, you need to keep five things in mind:
- Quantitative versus qualitative metrics
Qualitative metrics are unstructured, anecdotal, revealing, and hard to aggregate; qualitative metrics involve numbers and statistics, and provide hard numbers but less insight.
- Vanity versus actionable metrics
Vanity metrics might make you feel good, but they don't change how you act. Actionable metrics change your behavior by helping you pick a course of action.
- Exploratory versus reporting metrics
Exploratory metrics are speculative and try to find unknown insights to give you the upper hand, while reporting metrics keep you abreast of normal, managerial, day-to-day operations.
- Leading versus lagging metrics
Leading metrics give you a predictive understanding of the future; lagging metrics explain the past. Leading metrics are better because you still have time to act on them - the horse hasn't left the barn yet.
- Correlated versus causal metrics
If two metrics change together, they're correlated, but if one metric causes another metric to change, they're causal. If you find a causal relationship between something you want (like revenue) and something you can control (like which ad you show), then you can change the future.
What are you measuring?
The specified item was not found.