MaxDiff for paywalls: find the benefits users actually value
Most paywalls list benefits the team believes in. A MaxDiff study asks users to choose the most and least important option in a set, so you learn which benefits carry weight and which are filler. This guide covers when MaxDiff is the right research method, how to plan the sample, and how to turn the ranking into segmented paywall copy.
When a MaxDiff study fits a paywall decision
MaxDiff, short for maximum difference scaling, presents users with sets of benefits and asks them to pick the one that matters most and the one that matters least. Because every choice forces a trade-off, the method reduces the 'everything is important' bias that flattens ordinary rating questions. The output is a relative ranking of benefits, not an absolute score.
Use MaxDiff when you already have a working paywall and need to know which benefits to lead with, or when you suspect different user groups want different things. Skip it when you have no candidate benefit list yet, or when you only need to test two copy variants — a straightforward A/B test is cheaper and faster.
Decision checklist before you commit:
- You can name at least 8–12 candidate benefits to compare.
- You can define meaningful audience segments (life stage, use case, tenure).
- You can reach enough respondents per segment to trust the ranking.
- You are prepared to act on the result, including building more than one paywall variant.
If any of those is missing, fix it first. A MaxDiff study on a vague benefit list produces a confident-looking ranking you cannot use.
Planning the study: audience, segments, sample and incentives
The planning question that matters most is not 'how many people can we reach?' but 'how many responses per segment do we need before the ranking is trustworthy?' Work backwards from that number.
Applica's Dogo study shows the arithmetic. Four segments were defined — non-dog parents, puppy parents, adolescent dog parents and adult dog parents — and the target was 150 completed responses in each, 600 in total. Because participation can range from 10% to 70%, the invited audience was scaled to the expected response rate rather than to the target count. Both potential and existing users were included, so the ranking reflected the full addressable audience rather than only current subscribers.
Incentives affect completion, so keep them consistent across segments. The study offered extended free trials to potential users and subscription discounts or renewals to existing users; unequal rewards between segments would tilt the comparison.
Survey design field map:
- Segment field: captures which audience group the respondent belongs to.
- Benefit set: the list of value propositions shown in each task.
- Most-important selection: one choice per task.
- Least-important selection: one choice per task.
- Incentive field: records which reward the respondent received.
- Completion status: flags partial responses so they can be excluded.
Keep each task short enough to read in a few seconds. A survey that feels like work produces careless answers, and careless answers distort the ranking.
Reading the results without over-claiming
A MaxDiff ranking tells you relative order, not absolute demand. Two benefits can sit close together and still be effectively tied for your audience; treat small gaps as noise until an A/B test confirms them. The bigger signal is when a benefit the team assumed was universal drops to the bottom for one segment — that is the case for segment-specific paywalls.
The Dogo results split cleanly by life stage. Puppy parents put potty training and smart training plans at the top; adolescent and adult dog parents instead led with new techniques and tricks plus enhanced bonding. That divergence, not the average ranking, is what justified building two paywall variants: the puppy version kept all four benefit screens including potty training, while the adolescent/adult version dropped the potty training screen and led with the remaining benefits.
Applica reported the following outcome figures for the Dogo study:
- Puppy segment: 19% uplift in ARPU.
- Adolescent segment: 17% uplift in ARPU.
Those numbers describe one app's experience during the study period. Treat them as a directional reference and check your own baseline before assuming a similar lift.
From ranking to paywall copy
The ranking is only useful once it changes a decision. Two decisions matter most: which benefits belong in the free tier versus the premium tier, and which benefits appear on the paywall for each segment. If segments diverge sharply, that is the signal to consider segment-based pricing tiers as well as segment-based messaging.
A practical sequence:
- Rank benefits per segment and note where the order changes.
- Assign the top-ranked benefits to premium and check the free tier still has a reason to exist.
- Draft one paywall per segment, leading with that segment's top two benefits.
- Run A/B tests on the new variants against the current paywall.
- Keep only the changes that hold up in the test.
Failure distinction: if every benefit scores similarly, the problem is usually the benefit list, not the audience — the options are too vague or too overlapping to force a real trade-off. If one segment shows a clear leader and another shows a different one, that is genuine divergence and worth acting on.
Takeaways for your next paywall test
MaxDiff is a research step, not a substitute for testing. It narrows the field of candidate benefits so your A/B tests start from a better hypothesis. Pair the ranking with clear visuals and concise copy on the paywall itself, because a well-ranked benefit still has to be communicated well to convert.
If you are weighing whether to run the study at all, start with the decision checklist above. If you can act on a segmented result, the study is worth the sample cost. If you cannot, a simpler preference poll or a direct A/B test will get you further for less.
Frequently asked questions
What is MaxDiff analysis in simple terms?
MaxDiff, or maximum difference scaling, shows respondents sets of options and asks them to pick the most and least important in each set. The forced trade-off produces a relative ranking of options instead of a flat list where everything looks important.
How many responses do I need for a MaxDiff study?
The Dogo study targeted 150 responses per segment across four segments, for 600 total. Because participation can range from 10% to 70%, the invited audience was scaled to the expected response rate. Your own target depends on how many segments you define and how precise the ranking needs to be.
Should I run MaxDiff or just A/B test the paywall?
Use MaxDiff when you need to know which benefits matter before you write copy, especially if you suspect segments differ. Use a direct A/B test when you already have two concrete variants to compare. MaxDiff narrows the field; A/B testing confirms the winner.
When should I build separate paywalls per segment?
Build segment-specific paywalls when the MaxDiff ranking shows a clear order change between groups — for example, one segment leading with a feature another segment ranks low. If the rankings are broadly similar, a single paywall with the shared top benefits is simpler to maintain.