Skip to content
AppDues
Log in
Contact salesSign up

App User Testing: A Practical Guide for Mobile Teams

Surveys and heatmaps can tell you where people drop off, but they rarely explain why. A short round of app user testing — watching someone install, open, and move through your product — usually surfaces the friction that analytics only hints at. This guide covers what makes mobile testing different from web testing, which tools are worth the effort, and a five-step process for running a round and turning what you see into a testable backlog.

What makes app user testing different from web testing

The foundations are the same as web research: recruit relevant people, give them a task, and document what happens. The differences sit in five places that change how you plan a session.

Platform spread. On the web you might test mobile-first with a few desktop cases. In an app, mobile splits into iOS and Android, and you may also have a web app or heavy tablet use. The more surfaces you support, the more participants you need. A starting range of 8–10 users is common, but variation across platforms can push that toward 30.

Behavior does not transfer. A feature that works on the web does not guarantee the same consent, opt-in, or notification behavior in the app. Decathlon's insight team found that even with strong app trust, consent rates for tracking and personalized notifications were lower on the app than on the web. Treat web learnings as hypotheses to re-test in the app, not as conclusions.

Remote sessions are harder. Not everyone is comfortable sharing a phone screen, particularly older participants. Context matters too: people use apps on the move and amid distractions, so an in-person session in a cafe, station, or coworking space can be more realistic than a quiet video call.

Install and update steps. Unlike most web tests, an app test may require the participant to download, install, or update the app first. Budget extra time or send clear pre-session instructions. A 25–30 minute session is workable for a focused task; if you also need to cover install and sign-up, plan for 40–45 minutes.

Push and in-app notifications. Web push exists but is less common. In an app, notifications can appear mid-session and are harder to control. Add a dedicated notes section to record any pushes a participant receives and how they react.

Tooling: what you actually need

A realistic context often beats a fancy tool stack. In one app startup test, the researcher approached dog owners in a park, offered a free year of membership in exchange for a few questions and a try of the app, and recorded responses on a standard iPhone voice recorder. The dogs played around them while participants installed and explored — close to the distracted conditions in which many people would actually download the app.

That said, tools help with four specific jobs:

  • Recruiting. Respondent is useful for niche audiences and typically works out to roughly $60–$80 per user once recruitment costs and rewards are included. Userfeel and Userbrain are alternatives for unmoderated testing.
  • Scheduling. If your recruitment tool does not include scheduling, a simple booking link (for example Calendly) reduces drop-off before the session starts.
  • Prototyping. Figma is common, but Marvel App is a lighter option for non-designers who need something testable quickly.
  • Recording. Zoom or Google Meet can record sessions (Google Meet recording requires a premium account). AI notetakers such as Fireflies add notes alongside the recording. Lookback is worth considering if you test frequently and want viewers, note-taking, and organized findings. Eye-tracking tools like Tobii exist but are unnecessary for most organizations.

Whatever you use, check GDPR or other data-protection requirements before storing or sharing recordings.

Step 1: Decide what you want to learn

Start with quantitative data to pick the area with the biggest impact. If onboarding shows a sharp drop across the first two steps, your goal might be to understand what drives that drop. Write down your assumptions — but treat them as inputs to task design, not as conclusions, to avoid confirmation bias. Then define the task itself. The core rule: tell participants what to achieve, not how to achieve it.

Step 2: Recruit and screen participants

Once you have a task and a short question script, recruit. Beyond paid platforms, options include a targeted segment of your email list, friends-of-friends who match the audience, approaching people directly, and relevant online communities (respecting their rules).

Screen participants upfront, especially when they are not existing customers. A weak screening question invites bias. For a strength-training app aimed at women, "Would you like to improve your strength?" mostly produces yes/no noise. A better question — "What is your most important fitness goal?" with options such as improve endurance, lose weight, get stronger, improve flexibility, feel better, or other — reveals whether the person matches your audience. For app tests, also ask which device and operating system they use so you get a relevant mix of iOS and Android.

Offer compensation. A common benchmark is at least £30/$40 for 30 minutes. For existing customers, a gift card often works better than a free subscription, which tends to attract only superfans. Send a reminder the day before the session; many booking tools can automate this.

Step 3: Prepare the session

Before the first call, settle four logistics questions:

  • Will you record, and with which tool?
  • Will a second person join to take notes?
  • What is your no-show process? A common pattern is to message after five minutes and again at fifteen to decide whether to reschedule.
  • Where will findings be documented?

A rainbow sheet — a visual grid of observations across participants — is a practical way to spot patterns. Set it up before the first session so notes go straight into a consistent structure.

Step 4: Run the conversations

Open with a short introduction: who you are and what the session covers. A few Jobs to Be Done questions help you understand the participant, but keep them brief and avoid re-asking what the screener already covered.

Hand the task over and step back into a guidance and note-taking role. Useful prompts during the session include:

  • "Could you explain why you clicked?"
  • "What made you stop scrolling?"
  • "What is going through your mind?"
  • "I noticed you reacted <X>. Can you tell me more about it?"

When participants ask for help, resist the urge to solve the task for them. In one web-to-app signup test, participants struggled to work out how to download the app; the researcher encouraged them to keep looking and only stepped in once, after five minutes, when a participant seemed ready to give up entirely.

Set expectations before the task so people narrate freely without feeling judged. A simple framing: "You know that inner monologue we all have? This is your chance to say it out loud. I may ask questions, but that is not because you are doing anything wrong — it is to understand what you are thinking."

Step 5: Analyze results and build a testing backlog

Refresh the rainbow sheet after every session, while the details are still easy to recall. A pattern worth acting on usually shows up across at least five testers, and a typical round will surface ten or more such patterns. Treat each one as a hypothesis rather than a build order. Group the patterns by opportunity area: if onboarding feels unclear and drop-off follows, there may be several possible causes and several ways to test a fix. Apps are harder to A/B test than web pages because updates pass through App Store or Play Store review. Third-party tools can shorten that loop for parts of the product such as paywalls. A subscription platform may focus on A/B testing for pricing, packaging, and paywalls, while Optimizely and Apptimize cover broader app experiments. Qualitative findings and A/B results can disagree. In one illustrative case, a team tested a deliberately plain paywall design that nobody expected to win, and the simpler version outperformed the polished one for their audience. The lesson is to validate qualitative insight with a test rather than assume it. If you do not have enough traffic for A/B testing, prototype the change and run another round of user tests. Iterative rounds can use a smaller group, for example five participants.

A pre-session checklist

Use this before each round to avoid the most common gaps:

  • [ ] Goal tied to a specific drop-off or question in your analytics
  • [ ] Task written as an outcome, not a set of steps
  • [ ] Screener includes device and operating system
  • [ ] Mix of iOS and Android participants appropriate to your audience
  • [ ] Install or update instructions sent ahead of time
  • [ ] Session length set: 25–30 minutes for a focused task, 40–45 if install and sign-up are included
  • [ ] Recording tool chosen and consent captured
  • [ ] Notes structure ready, including a section for push notifications received during the session
  • [ ] No-show process agreed
  • [ ] Compensation and reminder scheduled
  • [ ] Data-protection review completed for recordings

Usability failure vs. participant discomfort

Not every struggle in a session is a product problem. Two failure types look similar on video but call for different responses:

  • Usability failure: the participant cannot complete the task because the interface, wording, or flow does not support them. This is a product signal and belongs in the backlog.
  • Participant discomfort: the participant is uneasy sharing a screen, unfamiliar with the device, or distracted by the environment. This is a session-design signal. Fix it by changing the format — in-person instead of remote, a different device, or a quieter setting — rather than changing the product.

Separating the two prevents you from redesigning a screen that was never the problem.

Session notes field map

A consistent notes structure makes analysis faster. A workable set of fields:

  • Participant ID and recruitment source
  • Device, operating system, and app version
  • Task given (outcome wording)
  • Step-by-step observations with timestamps
  • Direct quotes
  • Moments of hesitation or confusion
  • Push or in-app notifications received, and reaction
  • Whether the participant completed the task unaided
  • Hypotheses raised
  • Follow-up test or prototype idea

Frequently asked questions

How many users do I need for app user testing?

A common starting range is 8–10 participants. If you support multiple platforms, also have a web app, or see wide device variation in analytics, you may need closer to 30 to cover the variation. Iterative rounds after a prototype can use fewer, for example five.

How long should a session last?

A focused task usually fits in 25–30 minutes, which tends to improve uptake. If you also need to cover install and sign-up, plan for 40–45 minutes.

Is remote testing good enough for apps?

Remote sessions work, but they are harder for participants who are not comfortable sharing a phone screen, and they miss the distracted context in which many apps are used. In-person sessions in everyday settings can be more realistic. If resources are very tight, screen-sharing app mockups is a fallback.

Do I need paid tools to start?

No. A voice recorder and a notes sheet can carry a first round. Paid tools help with specific jobs: recruiting niche audiences, scheduling, prototyping, and recording or note-taking. Choose them only where the manual approach is slowing you down.

How do I turn observations into action?

Look for patterns that appear across at least five testers, group them by opportunity area, and write each as a hypothesis. Validate with an A/B test where traffic allows, or prototype the change and run another small round of user tests.