Prototype and test your ideas

Prototype Testing: A step-by-step guide to efficiently testing ideas through prototyping, with a concrete example.

16 min readAug 20, 2022

Teams now understand the relevance of testing prototypes, especially for daring innovation projects. It’s pretty evident that testing a 1-week prototype and iterating with users before developing it costs a lot less than spending three months developing something and then realizing that it doesn’t meet users’ needs at the end of the tunnel.

However, some teams have questions about the approach: Where to start? Which people to interview? What questions to ask? How to confirm ideas? What tools to use? How to handle user feedback?

I worked with a team that asked all these questions. In this article, I’ll walk through the 7-steps process I implemented to run usability testing interviews efficiently:

Step #1: Define a goal
Step #2: Write a research plan
Step #3: Recruit users
Step #4: Write an interview guide
Step #5: Build a prototype
Step #6: Conduct the interviews
Step #7: Analyze the collected data

I’ll use the project I led as an example: the project aimed to develop an internal Asana-like project management tool for employees.

Step #1: Define a Goal

Testing a prototype is like fast-forwarding into the future. Before you invest more time and resources, you can see the overall design, copy, user flow, and UI elements through users’ eyes. Beyond that, it helps you understand their behaviors, pain points, and ways of working.

However, before you jump on the first user and show him a prototype, ask your team the right questions. This article by Michael Margolis is a list of questions you might ask to understand what the team needs. The central question is:

What are we trying to learn?

To answer this question, you can lead a workshop that enables the team to identify the riskiest assumption:

https://cdn-images-1.medium.com/max/1600/0*yhgQlWD9ogaj7Btb.png — ***Riskiest Assumption Canvas*** *from Design a Better Business*

Team members write down their beliefs and place them in the middle of the box where they think is best.
Each team member explains their thoughts and risk assessment.
Then, the team votes on which one is the riskiest.

For the project taken as an example, this workshop allowed me to identify the following hypothesis as the riskiest: users will adopt the tool for their daily work and not just as a reporting tool. So my goal was quite simple: find out if project owners can use the tool for their daily work.

Make sure that the goal identified is:

Specific enough to keep interview questions focused.
Simple: one objective at a time.
Related to the outcomes, results, or KPIs that have the most significant impact on the business outcomes.

Once you identify the goal, you’ll need to define the plan and share it with the rest of the world.

Step #2: Write a Research Plan

A research plan is a concise one-page outline to tell stakeholders everything they need to know about the test without boring them with details. It also helps you focus and better prepare for the study: it’s a living document you complete as you confirm things with stakeholders.

This article by Tomer Sharon gives the sections of a good research plan. For the project I lead, the latest version of my research plan was as follows:

Research Plan

Title: Facilitate project management tasks for project owners

Author: Adam F.

Stakeholders: (name of project stakeholders)

Date: April 2021

Background: since March 2021, a new project management process has been defined to improve performance. We needed to evolve our project management tool to become the central place for managing these projects. Prototypes have been defined but so far without involving project owners.

Goal: understand if project owners can complete their daily project management tasks using our tool

Research questions:

What are the current ways of working for project owners? (frameworks, collaboration, tools, and challenges)
Would project owners understand how to complete actions to manage their projects on the new designs?
What capacities are missing to enable them to manage their projects effectively?

Method:

Number of interviews: 5
Location: remote via MS Teams
Duration: 45 min
Prototype: (a link to the prototype)

Participants: project owners from different entities

Schedule:

Recruiting: end of March
Study: 3rd week of April
Results: end of May

Make sure everyone on the team agrees on the research questions. If it’s their first time participating in this approach, make sure they don’t have inconsistent expectations:

Testing a prototype is not a way to get designs approved by users. Users don’t approve designs but react to them in a certain way that will help the team improve them.
Testing a prototype is not a way to ask users for solutions or future behaviors. Users don’t know what they want and aren’t good at predicting. All you can do is observe their current behavior and understand their pain points and needs.

The next step is about an essential part of a prototype test: users.

Step #3: Recruit Users

You might want to get feedback from various participants to understand how a broad audience perceives your product. But being selective improves the quality of the results and avoids wasting your time.

Jakob Nielsen demonstrated that a team could uncover 85% of usability problems from testing no more than 5 participants. This result is all the more true when carefully selecting these participants against well-defined criteria.

This article by Michael Margolis details how to be selective in recruiting users. The steps are as follows:

Brainstorm with your teams who you want to talk to and who you don’t. It depends on your testing goal. For example: “I want to exclude people new to the company.”
Convert this information into precise and quantified criteria. For example: “anyone who has less than six months of experience in the company.”
Identify the questions to ask to verify compliance with the criteria while ensuring not to bias the answers. For example: “on what date did you take up your position in the company?”
Create a screener, a form made up of the identified questions.
Share the screener with many people to collect data and identify relevant people to interview.

For the project I led since it was aimed at internal employees, I had enough information to target them without asking too many questions. I sent them the following email:

Email to Participants

Subject: 45 min of your time to save you hours of busywork

Hello Laure,

I’m Adam, and I joined the company as an external consultant.

As part of the transformation process, the current tool is evolving to become your one-stop shop for managing your projects. One of the tool’s main objectives is facilitating your project management tasks.

We would like to show you some mockups related to project management. Your feedback would be very valuable to us and may influence the upgrade. We would appreciate it if we could plan a 45-min call for this discussion.

Please reply to this email with time slots that suit you next week.

Thank you very much in advance for your help!

Adam F.

Here is a checklist of good practices for recruiting internal employees by email:

Don’t write too much text, be concise and direct.
Introduce yourself.
Presents what the interview will bring the user in return for her time. Highlight the impact that her feedback can have on the product or roadmap.
Provide information on the method (duration, date, remotely or in person, etc.)
End with a clear and engaging call to action.

Now that you have started your user recruitment, the next step is to write the interview guide.

Step #4: Write an Interview Guide

An interview guide helps you think about how to phrase questions, explicit the essential details to handle during the interviews, and plan a time slot for each step. You don’t have to go through it word for word, but it gives you structure and consistency from interview to interview.

I rely on the five-act interview by Jake Knapp and Michael Margolis for the structure.

The Five-Act Interview

Act #1: Friendly Welcome

You can’t show your prototype right away and ask questions: you would collect little reliable data. It’s rare for people to tell a stranger what they think. So you’ll have to go gradually, starting by creating a relationship of trust with the user. It’s the purpose of the first part of the interview.

Objective: put the user at ease and describe how the interview will work.

Guide checklist:

Thank the user for his participation.
Describe the context and why the interview is taking place.
Indicate the plan of the interview.
Reassure the user by reminding him that you don’t test him — there is no right or wrong answer — but that his feedback will help improve the designs.
Request permission to record the conversation and start recording.
Finish this first part by asking: Do you have any questions before we begin?

Here is what it gives for my project:

Friendly Welcome (5 min)

Thanks for your time today! I’m Adam. I joined the team as an external consultant two months ago. My goal is to understand your needs and expectations about the project management tool.

We’re constantly trying to improve our tool. So, getting your feedback is a really important part of that.

I’ll ask you many questions, but I’m not testing you. There are no right or wrong answers.

I’ll start this session by asking some questions about your working methods. Then I’ll show you a prototype and ask you to do some tasks.

If it’s ok for you, our interview will be recorded to help us better analyze your feedback. The recording will not be shared outside of our project team.

Do you have any questions before we begin?

Act #2: Context Questions

The user now understands the context of the interview and the game’s rules. You can continue with questions that are still a little general to understand better users’ existing behaviors and attitudes related to the subject of your study.

Objective: learn more about the user and collect some background information

Guide checklist:

Ask open and non-leading questions. “Open” means that it prompts the user to go into more detail. “Non-leading” means that it doesn’t imply or hint at desired answers which may bias what the user is saying.
Start broad with easy-to-answer questions to help build rapport and trust, then move on to specific questions related to the test goal.
Focus on discovery insights you want to gather about the specific testing goal you determined.

Here is what it gives for my project:

Context Questions (10 min)

Can you tell me a little about the kind of work you do at the company?
How long have you been involved in project management?
Are you and your team currently managing projects using the new process?
How do you get people to work collaboratively on your project?
How do you communicate with your team members?
How do you share progress with your managers?
How do you store and share documents?
How do you assign tasks to team members?
How do you submit your work for approval?
How do you assess project performance?
What are the main challenges or pain points you’ve met regarding project management?

Act #3: Introduction to the Prototype

Again, it’s important to re-emphasize the game’s rules to ensure the maximum amount of reliable data is collected.

Objective: make the user comfortable giving you frank and candid feedback about the prototype

Guide checklist:

Present the implications of testing a prototype (everything doesn’t work perfectly, the team hasn’t yet developed the designs, etc.)
Specify your expectations from the participant (thinking aloud, explaining what she is doing, etc.)
Encourage the participant to give honest feedback

Here is what it looks like for my project:

Introduction to the Prototype (5 min)

Now, I’d like to show you some prototypes of ideas we’re experimenting with. I’ll give you the link to a prototype [send the link through Teams chat].

I’ll ask you to do some tasks on it. If you share your screen and think aloud, it will really help us see it through your eyes [the participant shares her screen].

It’s a prototype, so it’s just pictures of screens. Even though they look real, they won’t work completely. There is no way to break it.

Again, I’m not testing you. I’m testing the prototype. If you get confused, it’s not your fault. It helps us identify the problems in the designs that we need to fix.

Since I didn’t design this, you won’t hurt my feelings or flatter me. In fact, frank and candid feedback is the most helpful.

To begin, please just take a look at this page. What is this?

Act #4: Tasks

A task is an action or activity you want a participant to complete on the prototype. So we come here to the heart of the interview.

Objectives: Give users tasks, and observe what they do on the prototype.

Guide checklist:

Keep the ordering of tasks in mind to guide the study participant from start to finish. Start with broad tasks and move in a logical flow.
Write a brief instruction for each task.
Check that there is no UI term in the instruction of tasks (button name, component name, etc.) Using UI terms would mean that the statements are too directive.

You don’t need to prepare specific questions, as they are always the same. These questions encourage participants to think aloud (discussed later in this article).

There are two types of tasks:

Broad, open-ended tasks: you give participants minimal explanation about how to perform the activity. It helps to learn how users think. This type of task is proper when you’re seeking insights.
Specific tasks: you give participants well-defined activities with clear guidance. It helps to learn how users use your product. This type of task is proper when you’re seeking usability optimization.

Here is what it gives for my project:

Tasks (20 min)

Let’s pretend you work on this project. You want to add a task and assign it to your team member. How would you do that?

Now, let’s pretend all the tasks have been completed. You want to submit your work for approval from your manager and directors. How would you do that?

Finally, the project was approved, but with some modifications to be made. You want to update project information and share it with stakeholders. How would you do that?

Act #5: Quick Debrief

You end the interview by asking the participant to summarize the essential points. It’s also an opportunity to open the discussion before ending.

Objective: synthesize discussions and open up opportunities.

Guide checklist:

Ask the participant to summarize the essential points.
Ask questions to identify opportunities not discussed during the interview.
Thank the user for his participation.

Here is what it gives for my project:

Quick Debrief (5 min)

We are coming to the end of our interview. In conclusion, what do you think is the most important in what we saw today?
If you had a magic wand, what would make the tool better for you?
Do you have any specific expectations that we haven’t had the opportunity to address during this interview?
Do you have another person in mind that we should contact?
This has been incredibly helpful [Summarize key points of the interview].
Your input is really valuable for us. Thank you for taking the time to answer all of my questions. We will continue to update you on our progress. Thanks SO much!

Once you write the guide and contact the participants, you have to prepare the prototype, and you’ll be ready to start the interviews!

Step #5: Build the Prototype

If you work with a designer on the team, you can share your guide and co-build the sequences on a prototyping tool.

It was the case with my project. The designer used the guide to design the screens and create the links to stick to the tasks. He then gave me a link to a high-fidelity prototype that looked like an actual site.

If you don’t have a designer on your team, no worries. There are plenty of other solutions. Depending on your skills, you can:

Build prototypes by hand:

Print screen templates and draw prototypes freehand.
Print a wireframe kit and have fun building the screens by cutting and gluing UI components.

Build prototypes using tools:

Use an office software that can do the trick (MS PowerPoint, Keynote, Paint, etc.)
Use a prototyping tool. Figma is my favorite as it comes with kits dedicated to wireframing (more tools at the end of the article).

Have someone make the prototypes for you (Fiverr, etc.)

Make the participant believe that a live website is a prototype:

Use your current live website or your development environment.
Use your competitor’s website as a prototype. Yes, it’s legal. It can save you time and help you collect lots of great feedback.
Build a quick & dirty website using templates from no-code tools (Umso, Bubble, Webflow, etc.)

To prototype a physical product, you can either use an existing product and modify it slightly or prototype the marketing of your product (landing page, poster, press article, etc.)

No excuse not to prototype your ideas and test them with users.

Now that everything is ready for the interviews, let’s start the fun!

Step #6: Conduct the Interviews

#6.1: Run a Pilot Session

In a pilot session, you perform a test and collect data the same way as an actual test. The difference is that you are testing your test to identify potential issues with the guide, prototype, recording tool, etc.

It helps you determine whether there are any flaws or confusing instructions and allows you to make adjustments before launching the test thoroughly.

For my project, since we didn’t want to waste our target users’ time on a pilot session, I did the pilot session with someone from the team to ensure everything was working fine.

#6.2: Involve Team Members and Stakeholders

Conducting interviews on your own may lead to conclusions not accepted by the team members and stakeholders. They may feel like it’s falling on their heads out of nowhere.

That’s why it’s essential to involve them in observing the interviews and talk about what they learned.

Even if they aren’t actively participating in the interview, you can coach them to be good observers and give them predefined roles (note-taking, timestamp capturing, etc.) to accelerate the next analysis step.

Here is a checklist of good practices for involving participants:

Before the sessions:

Communicate about the sessions in advance to encourage them to participate.
Explain the rules of the game.
Give them each a role to have during the interviews.

After the sessions:

Send them a link to a form where they can write down what they have learned.
Share the links to the recordings if they want to review a session or cannot participate.
Schedule a debriefing meeting.

#6.3: Prepare for an Interview

Before an interview, you have to prepare the material and the attitude.

Get into character by adopting a curious, open, and objective attitude. Take a deep breath. Smile. Try to see the world through your participants’ eyes. And resist the temptation to judge or dismiss what they say. — Michael Margolis

#6.4: Run the Interview as a Therapist

The attitude to adopt during the interviews always made me think of how therapists behave: being open to listening, answering a question with a question, nodding, and frequently asking, “what do you think?”

This impression was confirmed when I found this excellent guide which indicates what to say depending on the users’ behavior.

I also have my own interviewing attitude reminder checklist:

Show interest in what the participant saws.
Don’t take notes.
Let there be silence.
Don’t explain a question.
Don’t be distracted by my thoughts.
Don’t pitch the product.
Ask follow-up questions.

Follow-up questions are improvised questions that enable you to understand what’s behind some participants’ answers or behaviors:

Why?
So what happened there?
Was that what you expected?
How do you think it would work?
…

You collected a lot of data during the interviews. The next step is to analyze them and define the actions to be completed on the project.

Step #7: Analyze the Collected Data

The challenge is to convert user responses and behaviors into actionable things. You have to take some distance and not jump from what users are saying to conclusions too quickly.

#7.1: Transcribe the Interviews

I proceed in 2 steps:

First, I write down what I learned from an interview in general terms just after the session.
Then, I listen to the interviews again and transcribe them — using tools — to make sure I have understood users correctly and that I didn’t leave anything out.

For this second step, I write the verbatims on a spreadsheet file. I often discover things that seemed trivial during the interview but were very interesting.

#7.2: Convert User Quotes into Actions

I then paste those verbatims on prototype screenshots and share the findings with the team to identify the best solutions.

We convert participant quotations little by little into tangible actions:

We made adjustments on the prototype directly so it can go into the development or another 5-user testing plan.
We listed our outstanding questions for subsequent test sessions.
We integrated impactful requests that cannot be dealt with immediately into the roadmap as future discovery topics.
We updated personas with our new knowledge gained on target users.

As our target users were employees on my project, the insights collected were synthesized and shared on a document with the participants. We wanted to show them how the team handled their feedback to trust our method and give us their honest opinions.

These tips may make you think that prototype testing is tedious, but you shouldn’t be discouraged. It’s not necessary to set everything up perfectly from the start. The more you do it, the easier and more natural it becomes!

Takeaways

Prototype testing may seem tedious at first glance, but it saves time as you get user feedback before developing your ideas. The more you do it, the easier and more natural it becomes.
Start by defining your testing goal: What are you trying to learn? One objective at a time.
Write and share a research plan (a brief one-page outline) to align stakeholders.
Be selective about who you recruit for your tests to improve the quality of the results and avoid wasting time.
The best results come from testing no more than five users and running as many small tests as you can afford.
There is no excuse not to prototype your ideas and test them with users with today’s prototyping solutions.
Involve team members and stakeholders to observe the interviews and talk about what they learned. Coach them to be good observers, and give them predefined roles.
Don’t show your prototype right away to users during the interview. Think of the testing session as a semi-structured discussion, starting with discovery questions to build trust and moving gradually to specific tasks.
You have to make the user comfortable to give you frank and candid feedback.
Run the interview as a therapist. Listen, let there be silence, answer a question with a question, ask a follow-up question, etc.
Convert user quotes into actions collaboratively with your team. Adjust the prototype, update the roadmap with the subsequent discovery topics, and the personas with your new knowledge about target users.