Remote mobile usability testing is better but not perfect
The times they are a changin’, and that goes for remote moderated mobile usability testing, too. Technology improvements are making it easier to actually see and record your participant’s mobile device screen in a remote testing situation. Collaborative meeting platforms like Zoom and GoToMeeting now offer screen sharing capabilities on mobile devices.
HOW REMOTE MOBILE USABILITY TESTING WORKS
Here’s how it works. You send the participant the instructions to download the collaborative meeting mobile app from the App Store or the Play Store. You can also send a link to the tutorial video and written instructions on how to set up screen sharing.
GoToMeeting instructions for IOS: https://www.gotomeeting.com/screen-share/iphone
GoToMeeting instructions for Android: https://www.gotomeeting.com/screen-share/android
For iPhone users, you ask them to confirm that they have version 11.0 or higher. If they do not, you ask them to update their software.
Your next communication with participants is to confirm that they have been able to successfully download the collaborative meeting app and have gone through the setup process to share their screen with you.
Then you should be ready to go.
PROBLEMS WITH REMOTE MOBILE USABILITY TESTING
Sounds easy, but we have learned from recent remote moderated mobile usability testing studies that there are several problems associated with this process: some are user-generated, and some are technology-generated.
We know you shouldn’t blame the user, but we also know that it’s human nature not to follow instructions. No matter how much we reminded our confirmed participants to download the screen sharing app and set it up on their mobile device, some did and others did not. Or, as was often the case, most downloaded the app to their phones but didn’t set up the screen- sharing process.
This meant that we had to take the time at the start of the session to help them get set up for screen sharing. The process usually took 5 minutes or longer, eating up precious minutes in our study session. Because we couldn’t see their screen at this point, we tried to facilitate the process by sharing our screen with them, showing them where they needed to go for each step in the process. With our support, they were usually able to set up screen sharing, but not always.
If the screen sharing didn’t work, we asked them to go through the process one more time before we abandoned screen sharing and reverted to the old way of having them tell us what they were doing on their screen while we followed along on our phone. Not ideal.
In some cases, screen sharing worked fine . . . at first. Until it crashed. When this happened, we once again took our participants back through the screen sharing setup to see if we could get it to work again. This took more time away from the task plan and also clearly increased the frustration level of our participants.
Partway through the sessions we discovered what might have been causing the problem: if they had their phones configured to time out after a period of minutes, when the phone timed out, the screen sharing crashed. Learn by doing!
For all remaining participants, we added instructions to set their phone to never turn off so that we could get through the session without interruption. That helped in some cases, but not in all.
The other discovery we made is that GoToMeeting turned out to be more stable than Zoom for screen sharing, so we switched over to GoToMeeting for the remaining remote sessions we conducted.
LESSONS LEARNED IN OUR REMOTE MOBILE USABILITY STUDY
Because we love that we can now see our participants’ screens in remote moderated mobile usability testing sessions, we want to do more of this and do it more consistently. These are the lessons we have learned:
- If time permits, do a quick walkthrough with your participants ahead of your scheduled session—maybe the day before—to confirm that you can see their screen.
- Make sure that participants set their phones to stay on for at least the duration of the session so that their phone doesn’t time out.
- Remind them to turn off notifications, so that they are not interrupted by these and to ensure that these notifications do not become a part of the session recording.
- If the platform you have selected for your study causes any technical problems, have a backup plan to use a different meeting app. If that isn’t feasible, you can ask participants to show you their phone screen by holding their phone up to their web camera during several points in the session.
The need for screen sharing on mobile devices is growing and technology solutions are coming along to support this need.
Our takeaway is this: It’s getting easier and better to conduct remote moderated mobile usability testing. It’s not perfect, but definitely worth pursuing. Just have a backup plan.