User Experience Research
UX research isn't a step-by-step instruction manual. When you begin, expect to loop back and forth between different parts of the process. The early stages of planning and initial research will likely lead to new questions or unexpected findings. These discoveries will then inform how you conduct further research, analyze data, and even refine your original goals. Think of it as a cycle of learning and adjustment where each step informs the next, and sometimes the previous ones too. You'll uncover valuable insights throughout, not just at the end.
How to get Started:
1. Define your research goal
A well-defined research plan acts as a roadmap, ensuring your efforts are focused and efficient by outlining objectives, methodologies, and participant criteria. Establishing clear research goals provides a benchmark for success, allowing you to measure the impact of your findings and make informed design decisions.
Create a dovetail project nested in your domain's folder, complete with a research plan in place. See an example here:
2. Choose your methods of research and go
The initial stages of a UX project often feel the most ambiguous and challenging to navigate. Questions like "Who should we interview?", "How many participants do we need?", and "Should we conduct user tests or surveys?" frequently arise, and there aren't always straightforward answers. However, a structured approach can effectively guide the process of uncovering essential user requirements.
A recommended starting point is direct user engagement. If you lack a clear understanding of how users interact with the current system, conducting one-on-one user interviews with at least five participants is a valuable first step.
For established products, integrating user testing into your workflow can be particularly useful. Regardless of the chosen method, maintaining ongoing communication with users through alternative techniques such as field studies and moderated sessions will yield a deeper understanding of their pain points.
3. Provide Insights
Sharing the insights gained from these interviews with product owners, business stakeholders, and development teams is crucial for gaining alignment and defining the project's direction.
This tends to be the thing that helps get UX a seat at the table. When we can speak more clearly for our users and define their pain points, provide video back up of the pain points happening and define future solutions, then we can help steer the direction of the product.
Often, we may even create quick prototypes of how we could solve issues to bring the user's journey from poor to usable. Ultimately it's our goal to build delightful experiences.
Research Methods
Personas
Who can create Personas?
On some teams, product owners are the ones who create the personas. Other teams rely on the UX professional to create this data. We operate in an organization where many of the products we build are for a distinct set of users.
Example: Local Company Presidents, Warehouse, Outside & Inside Sales. We also build for internal users of our own products.
Example: WCMS (Wholesaler Credit Management Services).
In most cases, we understand who our users are so diving deep into Personas is objective to each team's needs.
What is a Persona?
A persona is a fictional, yet realistic, description of a typical or target user of the product. It is used to promote empathy, increase awareness and memorability of target users, prioritize features, and inform design decisions. The following resources define personas in depth and compare personas to other related artifacts.
Personas Make Users Memorable for Product Team Members
What Are Personas and Why Should I Care?
How to create a Persona
The following resources will help you determine the scope of your personas, understand the process for creating them, and avoid common pitfalls during the creation process.
3 Persona Types: Lightweight, Qualitative, and Statistical
Statistically Generated Personas
Competitive Analysis
When to create a competitive analysis
Data on what works well or poorly on other sites saves you from implementing useless features and guides UX investments to features that your users need. Competitive evaluations let you assess if your design is better or worse than your competitors and discover the relative strengths and weaknesses of competing designs. They allow you to take an in-depth look at how others solve the same design problems.
Goals of Competitive Usability Evaluations:
The goal of any competitive evaluation is to see what competitors are doing, how they’re doing it, what’s working, and what’s not.
• What design challenges are you trying to solve?
• What features of your competitors seem interesting or appealing?
• What features on your site do you want to compare to others?
Defining the Competition:
A typical competitive evaluation focuses on 4 to 6 competitors’ sites or applications. If you have many competitors, do an initial review of several of them to determine which sites:
• Offer similar content and functionality to your site or to what your site strives to provide
• Provide the best overall user experience
• Use innovative designs that set them apart
• Are your strongest or most important competitors
• Are the competitors that your customers are most likely to compare you against
Conducting a competitive analysis goes beyond merely comparing your product to major competitors. The goal is to extract valuable insights that can elevate your user experiences. While larger competitors may seem like natural benchmarks, valuable insights can also be gleaned from examining smaller or more innovative companies.
Additionally, exploring businesses that are only tangentially related to yours can offer unique perspectives, as they may employ similar practices but with variations influenced by traditions from other industries or market segments.
Competitive evaluations go beyond just finding badly designed products and claiming that yours is better. While you can learn what doesn't work from poorly designed sites, it's equally important to study sites with better usability than yours.
Journey Maps
A journey map is a visualization of the process that a person goes through in order to accomplish a goal. In its most basic form, journey mapping starts by compiling a series of user actions into a timeline. Next, the timeline is fleshed out with user thoughts and emotions in order to create a narrative. This narrative is condensed and polished, ultimately leading to a visualization.
What Makes up a Journey Map?
The Person – This is usually represented as a Persona or an Archetype. This provides a point of view for your users/people.Environment or scenario – This describes the journey that the user will be going on. An example of this could be “John Smith is looking to buy a new car”. This lays the expectations of the user and based on which persona/archetype they are, we can make assumptions on user behavior.
Journey Phases – This describes the high-level stages of the user’s journey. In the previous example, we can use stages like Research -> Test Driving -> Purchasing -> Initial Experience.
Actions, Mindset, Emotions – These are behaviors that the user will encounter throughout their journey and can be mapped while they go through the different stages. John can have felt overwhelmed during the research stage and have a student-like mindset, but when he gets to the test-driving stage, he could experience doubt and a second-guessing mindset.
Opportunities – These are insights that come as a result of the mapping. The opportunities allow the team to identify and answer questions like “How can we improve the workflow experience?” and “Where are the biggest opportunities to improve the experience?”.
At Winsupply we can tweak these phases to meet the needs of our domains. For example in payroll we used Awareness -> Consideration -> Decision -> Implementation to map the journey of our users through the people management ecosystem.
User Flows
User flows are diagrams that depict the path a user can take to complete a task while interacting with a product. A user flow focuses on the user's needs and the most efficient way to meet them.
What Makes up a User Flow?
For products that are already in use, user flow charts help determine what’s working, what’s not, and what areas need improvement. It helps to identify why users might be stalling at a certain point and what you can do to fix it. Does one screen flow into the next? Does the pattern of the screens make sense?
Mapping out the movement within an interface in a blueprint type fashion helps you see what options the user has on each page and if the routes available help the user accomplish a task innately and without wasting time.
User Flows can help identify the total amount of steps in a process, and how we could reduce those steps in their workflow before we ever begin the design process.
Building User Flow Diagrams
Some teams utilize Figjam as well as Figma for user flows. You are welcome to do either. Figjam has prebuilt user flow components and the arrows automatically connect from each box.
User Interviews
A research method where the interviewer asks participants questions about a topic, listens to their responses, and follows up with further questions to learn more.
What Makes up a User Flow?
Stephanie Walter’s “A Cheatsheet for User Interview and Follow Up Questions” is a great starting point to produce the questions that will help you get the most out of your interviews. For steps regarding the process from start to finish, see NNG’s “How to do a user interview.”
Step 1: Create Research Plan
Step 2: Create Interview Questions
Step 3: Moderated Session Guide for Interviews or Tests:
Your objective on a general user interview is to:
Understand at a high level what workflow the user does now.
Understand pain points.
Identify automation opportunities.
Document key findings and pass along information to other product teams where appropriate
How to find participants:
1. Utilize your IT Committee’s panel per domain
2. Find local company employees who can be great participants like Local Company Presidents, and office, admin employees who use the
Other ways to find participants:
Contact the presidents of local companies to request permission to interview them or some of their users. Please avoid requesting interviews with counter sales staff, as they may be very busy during the times you are available.Reach out to Area Leaders to ask about users they would recommend speaking with, or use the defined users from your IT panel.
Before you reach out to an LCP:Consider weather or natural disasters that may be affecting them. Avoid scheduling calls with individuals who may be experiencing a crisis. (It’s happened before!)
Tips and Tricks:
Instead of responding affirmatively, ask the user why they answered the way they did. Respond to a question with a question. It’s your job to keep them talking and on track within the guardrails of the objective. It can easily fall off, so know when to return the conversation to the main priority.
Unmoderated User Testing
Information Architecture:
One of the best ways to test the information architecture of your product is to utilize testing like Tree Test (Treejack) or card sorting to gain insights on how real users identify the proper labeling of your information. To further refine your results you may consider an A/B test to get user preference. At Winsupply we utilize a combination of these tests for many
Card Sorting:
A research method in which study participants place individually labeled cards into groups according to criteria that make the most sense to them. The set of cards can include pages, links, images, or descriptions of pages that users would navigate to on your website.To run a card-sorting session, present each participant with the unsorted set of cards. Then ask the participant to sort the items, information, or concepts into groupings.
NNG - Card Sorting
Tree Test Studies
An evaluation of a hierarchical category structure, or tree, by having users find the locations in the tree where specific resources or features can be found.To conduct a tree test, you don’t need to create any prototypes, design layouts, or visuals, nor do you have to write any content.
You need to prepare only two things: The tree, or hierarchical menu, which will be displayed as a series of accordions that represent the site’s navigation categories, without any visual design or content.
The tasks, or instructions that tell study participants what they should look for in the tree
As the participant clicks on categories, they expand to reveal subcategories. The participant clicks through the tree until they have found the location that they believe contains the information specified in the task.
NNG - Tree Testing
A/B:
A quantitative research method that tests two or more design variations to determine which variation performs best according to a predetermined set of business-success metrics.
In an A/B test, you create two or more variations of a design in a live product. Most commonly, you’ll compare the original design A, also called the control version, and one variation B, called the variant. Ideally, the variant should differ from the original design only in one design element alone, such as a button, an image, or a description.
During the A/B test, the incoming traffic of real users to your product is split, so that each visitor will be directed to only one of your design variations. This split of traffic can be such that each variation receives the same share of traffic.
Once the traffic is split to your design variations, you collect a set of metrics to determine which design variation encourages desired user behaviors and thereby better supports your business objectives.
NNG - AB Testing
Moderated User Testing
Identify the Purpose of the Test:
• Usability testing – Are users able to navigate the product efficiently? Do they understand how to complete key tasks?
• Feature validation – Are specific features functioning as expected? Do they meet user needs?• Experience evaluation – How do users feel while interacting with the product? Are they frustrated or satisfied?
• Comparative testing – How does your product perform compared to competitors or an older version?
Define Key Questions You Want to Answer:
To ensure focused testing, outline the core questions you need answers to. These should be aligned with the goals of your test. Some key questions include:
• Can users complete critical tasks without external help?
• Do users understand the purpose and functionality of key features?
• What are the main pain points users experience?
• Do users prefer certain design choices over others?
Write Your Problem Statement:
A problem statement clearly defines the challenge you are investigating in your moderated testing session. It helps guide your research by framing the issue, why it matters, and what you aim to understand.
You need to prepare only two things: The tree, or hierarchical menu, which will be displayed as a series of accordions that represent the site’s navigation categories, without any visual design or content.
Ask yourself:
• What specific difficulties do users face?
• What feature or process is causing friction?
• How does this problem impact their experience?
Problem Statement Example
Clearly specify which user group(s) are experiencing the problem. If different user types have unique challenges, mention them.
"The company's current manual sales tax compliance processes for its 770 locations are inefficient and unsustainable, particularly in light of frequent tax law changes and the potential for rapid business growth. The lack of automation increases the risk of non-compliance, financial penalties, and operational disruptions. This research seeks to identify automation and domain improvement opportunities to streamline sales tax compliance, reduce costs, and enhance operational efficiency."
Tips for Writing Test Questions:
Be neutral and non-leading – Avoid giving hints or assumptions about what the "right" action is.Try not to use the words within your labels of UI elements in the question. Encourage verbal feedback – Use follow-ups like "What do you expect to happen next?"Tell me why you clicked on that? What made you take that step? Measure time and effort – Observe how long users take to complete tasks and note areas where they hesitate (this can be done in Dovetail after uploading the recording and tagging different sections for the insight report). (If done correctly in Lyssna, this can be done for you automatically)
Setting up the Test with Lyssna (Moderated Testing):
When conducting moderated usability tests, Lyssna is a highly recommended tool for capturing participant interactions across various methodologies like Tree Tests, Card Sorting, and Prototype testing. The process involves scheduling a live session, providing a concise overview, obtaining recording consent, and then presenting the test questions. Crucially, withholding the questions until the live session preserves the authenticity of the participant's initial responses.
Setting up the Test with Lyssna (Unmoderated Testing):
Similar to a moderated test, Lyssna provides all the necessary tools to track key analytics for a usability test. However, unlike a moderated session, an unmoderated test does not require scheduling a live call or guiding users through the questions in real time. Instead, the test is created and distributed to anonymous participants on Lyssna to evaluate your product independently.
When setting up an unmoderated usability test, keep these two essential factors in mind:
Remove Identifiable Winsupply Information
Ensure that any references to Winsupply are removed from your images or prototypes. This includes details such as employee names, the Winsupply logo, or any local company branding.
Utilize the “Welcome” Screen Effectively
Since these participants are anonymous and unfamiliar with the product, use the welcome screen to provide context. Clearly explain who they are, their role, and the purpose of the application they are testing. This guidance will help them make more informed decisions throughout the test.
Practice User Testing with your UX Team:
ALWAYS PRACTICE FIRST:
To refine your user testing approach, conduct a practice session with your UX colleagues. Treat them as test participants, guiding them through your prototypes and questions. This hands-on simulation will reveal potential issues in your materials and testing process, allowing for necessary adjustments before engaging with real users.
Field Testing
Short Term Studies:
This involves spending a few hours or half a day with 2-4 users to observe how they interact with your product and any related applications they use alongside it. This approach provides insight into their workflow, preferences, frustrations, potential bugs, and any missing features that could enhance their experience.
Long Term Studies:
The primary objective of a long-term study is to gain in-depth insights over a few days into a single user's role and workflow. By closely observing their tasks over an extended period, you can develop a comprehensive understanding of their actions, responsibilities, and challenges—going beyond surface-level observations to fully grasp the intricacies of their job.
There are two ways these studies can be handled:
1. Engaged Field Study
This method allows for real-time interaction with users as they navigate the product, enabling you to ask questions and gain a deeper understanding of their workflow. When following up, it’s crucial to ask “why?” or “how?” to uncover the reasoning behind their actions. Users often develop subconscious habits that can reveal valuable insights into how your product can be refined to better align with their natural behaviors.
2. Disengaged Field Study
In this approach, you observe users as they work without interrupting them, only asking questions once they have completed their task. This allows users to settle into their natural workflow, revealing subtle interactions and behaviors that might not surface if their process were disrupted.
Surveys
Pros:
Surveys can be a valuable tool in UX design exploration, particularly when gathering feedback from a large audience. Their ability to reach many users quickly and cost-effectively makes them an attractive option for designers looking for broad insights. Additionally, surveys provide structured, quantitative data, making it easier to analyze user sentiment through metrics like satisfaction scores or Net Promoter Scores (NPS). Because all participants answer the same set of questions, responses are standardized, allowing for consistent comparisons across different user groups. Surveys are also convenient for participants since they can complete them remotely and at their own pace, increasing accessibility.
Cons:Surveys have notable limitations that make them less ideal in some UX research scenarios. One major drawback is their lack of depth—while they provide quantitative data, they often fail to capture the nuances of user emotions, motivations, and pain points. The way questions are phrased can also introduce bias, leading to skewed or unreliable results. Additionally, because surveys rely on self-reported data, users may struggle to accurately recall or articulate their experiences, reducing the reliability of the findings. Another issue is low engagement, as participants may rush through surveys or provide minimal feedback, limiting the depth of insights.
You will likely gain the proper understanding of the desired information by conducting use interviews with 5-7 different targeted users.
Outcomes
Insight Reports help make you an expert in your domain
Building your Insights
Our insight report process involves creating a structured document that highlights key findings from user research, usability testing, or data analysis. The goal is to transform raw data into actionable insights that guide design decisions and enhance the user experience.
We typically use Dovetail to create these reports. Before drafting the report, start by uploading your recorded user tests into individual documents. As you review the recordings, tag significant data points or recurring patterns in user behavior that stand out. Once these data points are tagged, you can organize them into relevant groups and incorporate the recording snippets into the corresponding sections of your insight report.
How these reports are typically structured:
• Introduction & Research Context – Explains the purpose of the research, the methods used (e.g., surveys, interviews, usability testing), and the target audience.
• Key Findings & Patterns – Summarizes the most important discoveries, such as usability issues, user pain points, or behavioral trends.
• Supporting Evidence – Includes direct user quotes, screenshots, heatmaps, or data visualizations to back up findings.
• Impact on UX Design – Connects insights to specific design challenges and suggests areas for improvement.
• Recommendations & Next Steps – Provides actionable steps based on the insights, such as design refinements, feature enhancements, or further testing.
Tools for UX
• Google Meets
• Mobbin (This is mainly interface, but they have flows included)
• Lyssna
• Figma
• Calendly (for scheduling interviews)