App Growth Measurement: When Your Numbers Disagree
Subscription apps rarely lack data. The harder problem is that the platform dashboard, the mobile measurement partner, the product analytics tool and the finance ledger each answer a different question — so they will not produce the same number. This guide shows how to decide which figure to trust for which decision, how much disagreement is normal, and where the blended business view fits in.
Why the numbers will never fully agree
Each team has improved its own slice of measurement. User-acquisition teams read platform models and MMPs. Product teams see richer behavioural analytics. Finance keeps its own revenue ledger. Subscription teams see cohort and renewal data. Those systems are not built to output the same total; they are built to answer different questions.
Apple's App Tracking Transparency (ATT) arrived with iOS 14.5 in April 2021 and pushed the industry toward aggregated, privacy-preserving measurement. SKAdNetwork postbacks, conversion values, anonymity thresholds and shorter windows reduced user- and creative-level visibility. The industry adapted with first-party data, modelled attribution such as Meta's Aggregated Event Measurement (AEM), web funnels, incrementality tests, cohort analysis and marketing mix modelling. The result is more measurement than before ATT, but often less agreement about what the numbers mean.
A useful example: for one client, Meta reported less than a third of the conversions visible in App Store Connect over the same period. That gap is not a reason to assume campaigns became 3.5x worse, and it is not worth a week spent trying to force the two systems to display the same total. The productive questions are different: is the gap broadly stable, does it widen or narrow when spend is deployed differently, does Meta still rank campaigns and creatives correctly, and what happens to business-level numbers when spend scales?
Define your normal disagreement
Reliable measurement does not mean identical numbers. It means the team understands why two figures differ, roughly by how much, and what would count as an anomaly.
Set a baseline tolerance. If Meta typically reports about 35% fewer trials than your baseline tool, that is your normal disagreement: 100 trials on the baseline against 65 on Meta is not alarming. A sudden move to a 70% gap is worth investigating. A stable mismatch and a new mismatch are different events and should trigger different responses.
The question to ask is not "which dashboard is right?" but "what decision am I trying to make with this number?"
Build the measurement system around decisions, not dashboards
When numbers conflict, a common reflex is to build another dashboard that pulls everything into one place. If the inputs are already inconsistent, that dashboard only displays the inconsistency together.
A less exciting but more useful starting point is to list the decisions your team repeatedly makes, for example:
- Should we increase channel spend next week?
- Which ad set gets the next $1,000?
- Did the paywall test improve the business?
- Is the US cohort genuinely better than the UK cohort?
- Can we afford to acquire a customer at $60?
- Is this channel adding customers or mostly taking credit for demand we already had?
For each decision, define four things:
- The fastest useful signal — what can move quickly enough to help you act?
- The downstream guardrail — what stops you declaring a false win?
- The business anchor — what blended outcome should ultimately move if the strategy is working?
- The evidence threshold — how much confidence is warranted, scaled to how large and how hard-to-reverse the decision is?
The decision-to-evidence map below sets out how those four elements apply to four common decisions.
Match the strength of evidence to the size of the decision
Once you accept that no single metric can do every job, the next question is how much evidence a decision actually requires.
A $500 creative-allocation decision tomorrow does not need the same evidence as moving $3 million of annual spend between channels next week. For the small decision, platform data can be adequate if you understand the usual reporting gap and you are watching the blended business outcome. For a paywall or pricing change, a controlled experiment plus downstream subscription guardrails is more appropriate. For a large channel reallocation, cohort quality, blended performance and, where possible, incrementality evidence are the right inputs. For financial planning, realized financial data and explicit assumptions matter most.
Directional data is not dirty data. If Meta consistently finds one creative at half the reported cost per trial of everything else, you do not need a year of data before putting more budget behind it — but you do want to know whether that creative brings a cohort that converts and retains sensibly, and whether blended economics stay healthy as spend increases. The standard of evidence rises with the cost of being wrong.
A worked example: one metric, used deliberately
Opal, a seven-person team, reached $5 million in annual recurring revenue and has since doubled that figure. Rather than building an enormous measurement framework, the team focused on Day 8 return on ad spend (ROAS) as a practical growth signal. Kenneth Schlenker described the approach as "just focusing on this one metric: Day 8 return on ad spend."
That single figure did not settle every question about the business. Its value came from two properties: it arrived fast enough to inform a decision, and it sat close enough to the economics the team actually cared about. The distinction that matters is between an imperfect metric and a useless one: growth teams get into trouble when they confuse the two.
Use blended metrics as the economic boundary
When attribution gets noisy, return to the boring numbers: how much did you spend, how many customers did you acquire, and how much revenue did those customers generate? That blended view is your anchor.
If the business spent $200,000 in a period and acquired 4,000 new paying customers, blended acquisition cost is $50. There will still be debate about which channels caused those customers, which captured existing demand, and how organic growth contributed — but the blended figure sets the economic boundary the models have to fit inside.
The three layers do different jobs:
- Platform data helps you make decisions inside the platform.
- Attribution helps you understand where growth appears to be coming from.
- Blended performance tells you whether the business as a whole is becoming more efficient.
Blended reporting does not solve attribution and does not tell you whether Meta caused customer #4,382 to subscribe. It does tell you what went into the machine and what came out. It also gives you a sanity check on channel stories: if every ad platform claims more conversions than the business acquired in total, the channel reports cannot all be read literally.
A useful mental order is to begin with the whole pie — its total size and economics — and only then use attribution and segmentation to explain how that pie is being carved up. Begin with the slices instead, and you can end up pouring budget into a channel that shines on its own dashboard while the total business hardly shifts.
The catch: blended truth still hides averages
Blended CAC is a strong anchor, but it is not a segmentation strategy. Averages can hide important differences.
Consider the familiar split between 'tourists' and 'locals'. Tourists may subscribe quickly and churn quickly; locals can become deeply engaged, long-lived customers. Combined into one average, the resulting number may represent neither group. The same pattern appears across subscription growth: a blended trial-to-paid rate can hide one country converting well and another badly; overall LTV can hide very different monthly and annual subscriber behaviour; average CAC can hide an audience that is cheap to trial but expensive to pay.
Time adds another layer. Subscription analytics tools commonly split lifetime value into two figures: realized LTV, which counts revenue already banked, and predicted LTV, which projects revenue still to come. A Day 30 realized cohort is measured but still young. A 12-month predicted LTV covers more of the customer's life, yet it remains a projection rather than a record. Each has a job — finance often wants contribution after fees and refunds, while growth may need an earlier gross-revenue reading to decide whether to scale tomorrow.
The hierarchy is not 'blended good, segmented bad'. Begin with blended metrics to establish whether the business economics hold up, then break the numbers down to see what is driving the result, where it is deteriorating, and which levers to pull next.
What good measurement actually looks like
Good measurement is unglamorous and rarely as clear as it seems. A lift in trial conversion can look impressive until those same users turn out to cancel sooner. A channel can appear efficient in isolation until the wider blended economics are visibly heading the wrong way. Neither observation makes the original figure worthless — it simply means no single metric is entitled to answer every question.
The goal is not to make every system agree. The goal is to know:
- what each number is useful for;
- when to sense-check it against the blended picture;
- which numbers help you operate day to day;
- which ones tell you whether the business is healthy;
- which decisions are important enough to require extra evidence.
The measurement stack will keep changing. The industry moved through ATT, SKAN, disappearing user-level signals and several years of being told attribution was dead, and there will be another version of this problem. If everyone can look at the data and know what to do next, the measurement is doing its job.
| Decision | Fast signal | Downstream guardrail | Business anchor |
|---|---|---|---|
| Allocate creative spend | Platform CPT / CPA | Trial-to-paid, early value | Blended CAC / revenue |
| Roll out onboarding change | Activation / trial start | Paid conversion, refunds, retention | Cohort revenue |
| Change price or paywall | Purchase conversion | Plan mix, renewal, refund rate | Realized revenue / contribution |
| Reallocate major channel budget | Attributed CPA + cohort quality | Incrementality / longer-term value | Blended CAC, ROAS, total revenue |
This map is a planning aid, not a validated benchmark. The specific signals, guardrails and anchors should be adapted to your own business model and decision cadence.
Frequently asked questions
How much disagreement between Meta and my baseline tool is normal?
There is no universal number. The practical approach is to measure your own typical gap and treat that as your baseline. If Meta usually reports about 35% fewer trials than your baseline tool, then 100 trials on the baseline against 65 on Meta is within normal range. A sudden shift to a 70% gap is a new mismatch and warrants investigation. What matters is whether the gap is stable and whether the platform still ranks campaigns and creatives correctly.
What does blended CAC tell me that attribution does not?
Blended CAC shows what the business put in and what came out, without needing to assign each customer to a channel. If you spent $200,000 and acquired 4,000 new paying customers, blended acquisition cost is $50. Attribution can then help explain which channels appear to have contributed, but the blended figure sets the economic boundary those attribution models must fit inside. It also flags impossible claims: if every ad platform reports more conversions than the business acquired in total, the channel reports cannot all be read literally.
Should I use realized LTV or predicted LTV?
It depends on the decision. Realized LTV reflects value already generated and is observed data, though a Day 30 cohort is still immature. Predicted LTV estimates future value and is more complete over a 12-month horizon, but it is an estimate rather than observed data. Finance may prefer contribution after fees and refunds; growth may need an earlier gross-revenue signal to decide whether to scale tomorrow. Both are useful, and neither replaces the other.