Metrics and dimensions
A metric is a number (sessions, clicks, revenue); a dimension is the criterion you break it down by (channel, page, device, month). The value lies in combining them: “revenue from the email channel in October”.
Always compare with something
- With the previous period, to see the recent trend.
- With the same period last year, to remove the effect of seasonality.
- With the target, to see whether you are on track with the plan.
Beware of small samples
A rate calculated on twenty visits can change a lot with just one more event. That is why we show the confidence interval alongside rates: it tells you how reliable the value is. Below the minimum data thresholds, we prefer to say so openly.
The most common traps
- Confusing correlation with causation: two numbers rising together does not mean one causes the other.
- Looking only at the average: a few very high values push it up; the median is more robust.
- Changing the definition halfway through (what counts as a conversion, what counts as an active customer).
- Reacting to a one-day change: look at trends over longer periods.
A number is a question
When a value changes, the next step is to understand why: a channel, a page, a campaign, an external event. That is what monthly reports are for.
An example: August versus July
A hotel sees direct bookings fall by 30% in September compared with August and gets worried. Comparing with September last year, it finds they have grown by 12%: the drop was just seasonality. That is why you always compare with the same period a year earlier.
