Price Testing for Mobile Apps
A price change rarely shows up in one number. It can lift purchases while lowering the revenue each buyer eventually brings in, and the difference only becomes visible when trials, paid conversion, retention and revenue are read as one chain. This guide sets out how to structure that chain, which metrics belong to each stage, and how to avoid declaring a winner on a purchase-rate figure that hides a weaker outcome later.
Why a purchase count is not enough
The familiar A/B test splits an audience at random, shows each group a different version, and counts how many people complete one action. That model is a binomial experiment: each trial either succeeds or fails, and the result is a success rate you can compare between groups. Significance then tells you how likely the gap is to be more than chance.
Price experiments break that model in a useful way. The two cohorts are not performing the same action at different rates; they are paying different amounts. A cheaper option can produce more purchases and still generate less revenue. Once subscriptions enter the picture, the question widens further: did the user start a trial, did the trial convert to a paid plan, and how many renewals followed? A single yes/no purchase metric cannot carry that weight.
That is why the unit of judgment has to be lifetime value per customer over an extended horizon. The horizon is a business choice — a year, three years, or open-ended — but the principle is constant: the value of a subscriber is not knowable on day one, because renewals arrive later.
Three stages to read in order
Group the metrics of the subscription lifecycle into three buckets and read them in sequence rather than in isolation.
- Initial conversion. This is where a customer starts a free trial or accepts another offer. It is the stage that reacts first and most visibly to a price change.
- Paid customers. This is the crossover point where a trial-taker becomes a revenue-generating subscriber. For most subscription businesses it is the stage that decides whether the price change helped.
- Revenue. Money generated during the experiment is the realized portion of lifetime value for those cohorts. It is a snapshot, not the final figure, because retained subscribers keep adding value after the report is read.
The growth rate of that revenue can differ by variant, and it can also differ with the mix of paid customers — for example, how many are annual rather than monthly. Treat the revenue column as a moving picture and note the date you read it.
Metric map: which number answers which question
The table below lists the metrics available for each of the three buckets. Read down a column to follow one stage, and read across a row to see how the same idea is expressed at different stages — for example, a count, a rate, and a per-customer value.
A practical reading order is: start with initial conversions and trial conversion rate to see whether the offer attracted people; move to conversion to paying and active subscribers to see whether they stayed; finish with realized LTV per customer and MRR per paying customer to see what each cohort was actually worth. A variant that wins the first stage and loses the third has not won.
Reading results over time and by platform
A single cumulative figure hides the shape of a trend. Tracking each metric day by day shows which movements are steady and which are noisy, and it shows where a deeper look is warranted. Results can be filtered by platform, because performance often differs between them, and the full daily history of each metric can be exported for further analysis.
Side-by-side tables of cumulative performance per variant make the change drivers easier to isolate. Most metrics can be expanded to show performance by product, which is often where the real explanation sits: one product may be carrying a variant while another drags. That tabular view can also be exported.
The point of all this is confidence. A pricing decision made without knowing what drove the movement is a guess dressed as a result.
These views — platform filtering, per-product expansion and daily export — are capabilities of an experiment tooling layer, not something a bare A/B calculator provides. When you evaluate a tool, check that it exposes each of them for the metrics you plan to judge on; otherwise the breakdown work falls back to you.
A worked example using the supplied metrics
Suppose variant A keeps the current price and variant B lowers it. At the initial conversion stage, B shows more trials started and a higher initial conversion rate. At the paid customer stage, B's conversion to paying is similar, so active subscribers rise roughly in line with trials. At the revenue stage, however, B's realized LTV per paying customer is lower, and its MRR per paying customer is lower too.
The correct conclusion is not "B wins on conversions" and not "B loses on revenue." It is that B trades revenue per buyer for volume, and the decision depends on whether the extra volume closes the gap in realized LTV per customer over your chosen horizon. Because revenue keeps accruing from retained subscribers, the gap may narrow or widen after the report date. Re-read the same cohorts later before locking in the price.
Checklist before you call a winner
- Confirm both variants ran on the same metric definitions and the same time window.
- Check the initial conversion stage first, then paid customers, then revenue — in that order.
- Compare realized LTV per customer, not only purchase counts.
- Note the report date and plan a later re-read, since renewals keep changing the revenue figure.
- Check whether the paid-customer mix (annual versus monthly) differs between variants.
- Filter by platform and expand by product before attributing the result to price alone.
- Export the daily history if you need to separate a steady trend from a spike.
- Write down the horizon you are judging against before you look at the numbers.
| Initial conversion | Paid customers | Revenue |
|---|---|---|
| Initial conversions | Paid customers | Realized LTV (revenue) |
| Initial conversion rate | Conversion to paying | Realized LTV per customer |
| Trials started | Active subscribers | Realized LTV per paying customer |
| Trials completed | Churned subscribers | Total MRR |
| Trials converted | Refunded customers | MRR per customer |
| Trial conversion rate | MRR per paying customer |
The table reproduces the supplied metric names exactly. It is a map of available metrics, not a claim that any particular result was observed.
Frequently asked questions
Why can't a standard A/B significance calculator settle a price test?
Those calculators assume a binomial experiment, where each user either takes one action or does not. In a price test the two cohorts are paying different amounts, so a higher purchase rate does not imply higher revenue. The outcome has to be judged across trials, paid conversion, renewals and revenue together.
When should I read the results of a price experiment?
Read them at the point you planned, but treat the revenue figure as provisional. Realized LTV grows as retained subscribers renew, and the growth rate can differ between variants and by the annual-versus-monthly mix. Re-read the same cohorts later before making a permanent pricing decision.
Which metric should I look at first?
Start with the initial conversion stage to see whether the offer attracted people, then move to paid customers to see whether they stayed, and finish with revenue metrics such as realized LTV per customer. A variant that leads at the first stage and trails at the last has not won.
Can I break results down by platform or product?
That depends on the tool you run the experiment in. Platform filtering, per-product expansion of most metrics, and daily export of each metric's history are capabilities an experiment tooling layer can provide; a bare A/B calculator typically does not. Confirm which of these your tool exposes before you rely on a breakdown for a pricing decision.