Overcoming Challenges in Testing Apps with International Users
Published on 31 Aug 2023 in
Launching a mobile app in multiple countries at once can help companies reach a larger market. However, ensuring that your app matches the needs of stakeholders in different regions, using different languages, can be a challenge.
Remote usability testing is extremely useful for validating your application’s use in multiple locations at once. Testing app usability online rose in popularity during the pandemic as a way to match the world’s exponential digital transformation, while keeping in line with Covid-19 restrictions.
Of course, despite its benefits, remote usability testing still has its unique challenges, many of which we encountered ourselves when performing UX research when developing digital products for our clients. Nevertheless, each challenge is its own opportunity to learn new processes.
Why run remote usability testing?
Testing user experience is a crucial step in the development of every digital product. Usability testing is the process to measure if your product is easy-to-use for the people it is designed for. Furthermore, usability testing mitigates the chances of your product team receiving critical complaints and having to fix key features after delivery.
Remote usability testing can help UX researchers complete more sessions under a short timeframe, while allowing you to collect behavioral data from anywhere in the world, in real time. During video calling sessions, you can observe facial expressions and reactions as you run through wireframes, directly from the real people who will be using the app.
Challenges in running usability testing online
The shortcomings of remote usability testing is that many factors are outside of our control.
In the past, our UX researchers have found that more users turn up to a testing session than initially planned. This usually happens because we are unable to physically prevent anyone from entering a room when conducting remote usability testing. The best usability testing is one-on-one, with a maximum of two participants to represent each stakeholder. This allows us to record the most accurate data possible. However, when there are too many participants, it can make it difficult for researchers to focus on observing how a user progresses through a wireframe.
Sometimes, C-Suite and management level people turn up to a usability test intended for their employees. Unfortunately, employees may not give honest feedback when under scrutiny from their managers, or they might overly rely on instructions from their supervisors to give the ‘right’ answers. However, in real-life situations, employees would have to navigate through an application or platform themselves. Consequently, any usability testing that is attended by managers will cause a company to miss out on valuable feedback.
Additionally, our UX researchers have occasionally found that a large number of participants can derail the function of a usability test. It isn’t easy to have tens of people run through a single sitemap together. In a case where tens of people have attended a usability test, we have inevitably been asked to demonstrate a runthrough of the sitemap. This is more similar to a training session, which is not the purpose of usability testing.
Our aim is not to force people into complying with a digital product. UX research is an opportunity to adapt your digital product to easy, intuitive user flows, and to create a better experience for your users. After all, user-friendly apps lessen the need to conduct expensive training sessions.
Steering towards smoother remote usability testing
At Morphosis, our ethos is that all mishaps are an opportunity to improve. Thanks to some of the obstacles and challenges we’ve experienced during our UX research, we have developed a process to avoid speed bumps in remote usability testing.
Here are a few steps you can take to ensure successful, accurate and productive remote usability testing:
1. Run stakeholder interview sessions to align your business goals, important features, and target users.
2. Generate tasks and scenarios relevant to the users’ jobs-to-be-done and the context of how they will use the app.
3. Prepare conference call tools that are suitable for your team and users. We recommend recording the session with the participants’ camera and microphone on. Make sure that participants can share their screen, as this will allow UX researchers to capture real-time interactions and perceptions with your product or prototype.
4. Assign a specific notetaker, so that your moderator can devote their whole focus to the participants. Alternatively, record the session so you can analyze and define the user’s patterns of behavior later.
5. Encourage your participants to think out loud. Ask them to tell you where they are looking, why they are confused, and where they are going to click. Vocalizing a user’s thought processes can be especially helpful if you are testing mobile apps, as you will not have a cursor to track users.
6. At the beginning of each session, state clearly that there are no right or wrong answers. If a person running through our sitemap cannot complete their task, then it has been a successful usability test! We are here to learn, not judge anyone’s performance. Usability testing – remote or otherwise – is not a pop quiz. It is an assessment of the app’s usability and intuitiveness.
7. Never lead users to the next screen or tell them where they should click. (As mentioned above, this is not a training session.) Let participants try and fail a few times. If they give up or become frustrated, then end the task and explain how the design actually works.
8. Be ready to improvise follow-up questions. Sometimes, participants can surprise you by clicking indirect paths or looking at elements that are not related to the tasks set. Investigate their intentions to discover valuable insights and unexpected solutions.
9. Ask your users to compare your app with competitors, if any. Learn what your users like or dislike about your product when compared with your competitors. This way, you can come up with unique selling points for your business.
10. Perform task analysis. The best way to visualize usability scores is to include them in a table. Record which features perform well or poorly. Our UX researchers recommend three score criteria: green, yellow and red.
- Green: the user can complete the task intuitively.
- Yellow: the user tried a few times, clicked the wrong button and went down the wrong path at first, but eventually managed to complete the task.
- Red: the user cannot complete the task even after multiple attempts.
Get insights on your next digital product
A good usability testing report can identify the most critical problems in your digital product. Moreover, usability testing helps you improve your apps, avoid dead-end flows, and increase user retention.
While running usability testing remotely has its own challenges, good communication, strong organization, and adequate planning can help minimize obstacles. As with all things, it is important to remain agile and adaptable to any circumstances that might happen.
People behave in unpredictable ways, which provides brand-new ways to learn and improve your apps.
Are you unsure on how to start validating your digital product or brand-new mobile app? Find out how our team of UX researchers can support you here.
Subscribe to our newsletter.
Here are some related articles
UX Research
Let us will help you open new business opportunities by giving you a new perspective on your digital product you may not have considered before.