Recently I had the opportunity to try out a friend’s VR gaming headset. It’s a disorienting experience at first–you’re sensorily experiencing a simulated environment within reality. Meta! It reminded me of trying to hit a piñata at a birthday party while blindfolded:
Look how ridiculous I look trying to ninja that fruit! I want you to picture yourself like me playing a VR game swinging furiously at imaginary fruit every time you start a web design project without doing any research or having any data. Or, if you prefer the old-school version, it’s like swinging at a piñata blindfolded. Taking swings at things you think are there, when the reality is if you’ve done no research and have no data to reference, you have no idea what (or who, in this case) you’re swinging at. You start to theorize who your visitors are along with their wants, needs, and goals based on your gut assumptions. It’s an easy trap to fall into under the constraints of tight deadlines and high expectations, and we’ve all done it. However, the companies with the best results are combining user research data with their experience and knowledge to make web design decisions.
When you create a website that’s not meeting the needs of visitors, you will see the effects in your bounce rate, low conversions, and ultimately revenue. Research tells us that websites that have been optimized with UX strategies experience higher revenue growth. Furthermore, if you’re an agency and you’ve adopted the Growth-Driven Design methodology or you’re dipping your toes into the GDD way of web design, then 2 out of the 3 stages of the methodology center around research and data. It sounds intimidating, and there are many ways to go about collecting the information you need for a Customer-Focused Strategy, as well as for the Learn phase in Stage 3 of the GDD methodology, Continuous Improvement. You could even say that UX research is the basis of agile web design, because at it’s simplest agile web design can be distilled down to a design-test-improve-repeat cycle.
This starter kit is for anyone who is: just getting started with user experience research, overwhelmed by the different kinds of research that you can perform, or confused on when to use which method or tool. Not only is this data necessary for agile web design, it is useful for obtaining buy-in from clients and stakeholders.
At this point you can either continue reading if you'd like to read the explanation of the types of research and information you can gather for UX design, or you can skip straight to the tools.
Types of Research
qualitative VS. quantitative
Qualitative: information gathered directly through observation (answers why should we fix this? How should we fix this?)
Quantitative: information gathered indirectly through a measurement tool (answers how many? how much?)
The chart below from the Nielsen Norman Group is an effective way to view the difference between qualitative and quantitative information as it pertains to UX, because it tells us how to use each type:
Behavioral vs. Attitudinal: What People Say vs. What People Do
Scenario:
Me: “I want to lose 30 pounds.”
The next day…
Also me: “I’d like a dozen donuts, please.”
This holds true for what people say about a website and what people do on a site. Design can have a significant influence on the actions that users take on site over what they came to the site to do. Gathering attitudinal information is useful and can give you a starting point, but experts recommend focusing your efforts on addressing the behavioral feedback you observe instead of the answers given to questions about the site:
The Tools: What They Are Used For & How to Use Them in Agile Web Design
Now that you’ve got a basic idea of what kind of information we’re looking for and why it’s important in agile web design, let’s get to the main event. This is by no means a comprehensive list, but I’ve compiled the tools and methods I’ve found to be most helpful and are easy to get up and running and start gathering information quickly.
The tools are organized by what you’re testing. For each type of tool, I’ve provided a quick start option for those needing immediate actionable information or working with a small budget. There's also and an advanced option for each tool type for those who are able to invest more resources in your UX research initiatives or have some experience with research.
Choose the type of data below to see the tools for each:
- Navigation & Information Architecture
- Usability
- Quantitative Behavioral Feedback
- Qualitative Attitudinal Feedback
Navigation & Information Architecture
Navigation is your visitors' road map. It tells them what your company offers, and where they can find what they’re looking for. Poor navigation and information architecture can frustrate visitors to the point of leaving the site.
Card sorting
Users are asked to sort “cards" of topics into categories that make sense to them. (The cards can be digital or physical pieces of paper. I'll mostly discuss digital tools below.) In open-ended card sorts, participants are asked to create the categories themselves, or in a closed card sort the categories are provided for the participant.
Quick Start Option: Optimal Sort
Free plan: unlimited studies, limited to 30 cards & 10 responses
Advanced Option: Optimal Sort, The Suite Plan
Includes unlimited studies, unlimited cards, unlimited responses (and more)
Optimal Sort is easy to set up and for participants to use. They provide a robust results analysis dashboard, even on the free plan. If you’re an in-house UX Designer or a small agency you can probably use the basic plan. However, if you’re doing a lot of redesigns or have many GDD retainer clients, I’d recommend their Suite or Enterprise plan to get access to other Optimal Workshop tools, like the next tool I’ll show you.
Here’s a small piece of the analysis provided from a card sort study:
Tree Testing
During a tree test study, you can test out a proposed information hierarchy by asking participants to perform specific tasks, and observing and measuring their success with locating items in your navigation structure and their level of difficulty using it.
Quick Start Option: TreeJack or DIY with Index Cards
Start out with the free plan from Optimal Workshop OR you can use index cards with the topics written on them and ask individuals to sort into categories while observing their thought processes.
Advanced Option: TreeJack, the Suite plan (includes Optimal Sort and others)
What I love about TreeJack is their detailed analysis dashboard. Here’s an example of a chart that shows which paths were taken, success and failure, and what was chosen as the answer to the task:
Beginner tip: this graph can be a bit challenging to dissect visually. They have an option in the top right corner to view in a linear style versus the web, which I find to be easier to analyze. You can also download this graph as a .png for your stakeholder presentation deck. Hurray!
Usability
Usability testing can uncover inefficiencies in design, user frustrations, and effectiveness of calls to action.
User Testing
There are a few approaches to user testing: unmoderated, moderated, in person, and online. The basic idea is to define your goals for the test, create tasks or questions for participants, and observe their behavior and thoughts while using the site.
Quick Start Option: DIY prototype testing
If you’re already using a tool like InVision or Adobe XD, the easiest way to start user testing is to create a prototype to observe participants in person, or share a link in an email or survey tool to observe remote participants. If you’re early on in the design process, you can also user test with a paper prototype.
Advanced Option: Usertesting.com
Usertesting.com is a platform for remote usability testing. They provide test templates for: first impressions, competitor comparison, application prototype testing, e-commerce, and more. You set up the demographics of the group you’d like to get feedback from, and screener questions to ensure you’re getting participants that reflect your personas. Results usually can appear in within an hour. You can watch videos of participants interacting with your site, app, or prototype which provides valuable feedback. You can clip parts of the video and share with your team or stakeholders, and input notes alongside the video for referencing. Quotes from User Testing videos can be the hook you need to get buy-in from clients or stakeholders on important web design decisions.
Quantitative Behavioral Feedback
This kind of research is useful for assessing if desired tasks are being completed answering questions such as “what are users looking for?” (or jobs-to-be-done, which is the user’s desired outcome or goal) analyzing “what do we want users to do?” and “do these two answers line up?” Having these answers to go alongside your research findings will provide the paths to design solutions. Use personas to filter your research results through how each user group uses the site, and jobs-to-b- done to validate whether they are achieving their desired outcome at the optimum level of usability (don’t make them click 5 times to get to their outcome). These tools will provide measurements of usability, versus the qualitative information gathered from user testing.
Heatmapping
Heatmaps are used to measure where and how much elements on a site are clicked. Some heatmapping tools also have scroll maps which show the depth users are scrolling on a page. These are useful for assessing user behavior patterns, CTA effectiveness, and navigation.
Quick Start Option: Hotjar
Hotjar has a free plan, but is limited to 2,000 page views per day. This is a great option for low traffic sites or if you want to try it out before buying. Hotjar has the scroll map, as well as visitor recordings. Their Plus plan for $29/month gets you data for 10,000 page views per day and unlimited users and reports.
Other Options: Lucky Orange, Crazy Egg
The advanced options for heatmapping tools would be the higher-priced tiers of the three tools mentioned here. These are all similar in price structure, but have slightly different features.
Complete transparency: I personally have only used Hotjar, but my company just bought Lucky Orange after doing a comparison of several heatmapping tools on the market. Lucky Orange is relatively new on the scene compared to others, but offers a great value and have a few extras that Hotjar doesn’t. Lucky Orange’s dashboard is more comprehensive (and ironically, more user friendly), and they also offer real-time visitor recording so you can ask them to open a chat and help them find what they’re looking for, ask for feedback, etc.
Clickstream Analysis
Knowledge of what paths users are taking through your site, where a high percentage of users are exiting, and what pages are most visited can help you in several ways. You can identify where user goals and design are misaligned, potential information architecture issues, or even where to place high impact content.
Quick Start Option: Google Analytics Behavior Flow Tool
A/B Testing
We’ve all been there–you get into an argument (I mean “discussion”) with a colleague or client over a design decision where opinions are at odds, and forward progress halts. A/B Testing is a great research method for testing two different variants that can be as simple as which header image to use or what color the CTA button should be. When you put design decisions like these in the hands of the users you get two benefits: an effective website and a decisive end to a discussion that’s reached a stalemate.
Quick Start Option: Optimizely 30-Day Free Trial or their Pay As You Go option ($49/1000 unique visitors)
There are a ton of A/B testing tools on the market, if you want to check out more here is a comprehensive list of A/B testing tools reviewed by experts.
Advanced Option: Hubspot Marketing Pro
If you’re looking for an all-in-one solution, HubSpots Marketing Pro plan includes A/B testing.
Qualitative Attitudinal Feedback
What are users' first impressions about the site? What do they say they want to do on the site? What do they say they are looking for?
Surveys & Feedback Polls
Quick Start Options: SurveyMonkey, Typeform & Hotjar
The SurveyMonkey free plan allows up to 10 questions and 100 responses per survey with an unlimited number of surveys. What I like about SurveyMonkey is that even on the free plan they provide a nice survey analysis:
I’ve only used Typeform a little, however if you’re looking for a very simple and beautiful interface, I definitely recommend it. They have a free plan similar to SurveyMonkey, and reasonably priced Pro plans. One huge bonus is they have created their own version of Space Invaders, Form Invaders. You’re welcome.
If you’re like me and wonder, “what should I ask on a UX research survey?” here’s a great article that covers crafting an effective qualitative survey in depth.
The Hotjar free plan includes on-page feedback polls where can ask questions such as visitor intent, what prevented a purchase, and more. This is a neat feature because you can set triggers for when the poll appears.
Advanced Option: Foresee
You'll have to request pricing on Foresee. It's an Enterprise-level solution with robust options for collecting user feedback.
when in doubt...test!
The next time you start thinking you know what users want, stop...and test! If you can employ a couple of these methods and apply the insights gained to your website, it will be more effective than designs created purely on conjecture. I hope this helps you get started with user experience research. In this article, I’ve listed the research types and tools that I believe will give you the most actionable information with minimal setup and pre-work. Of course, the pricing and features for each tool listed on this page is subject to change. I hope you investigate the tools that sound right for you! I’ve barely scratched the surface on this topic. So, if you’d like to go deeper, I’d highly recommend the Growth-Driven Design certification. There's over two hours of video on user research and numerous additional resources.
Did I miss any of your favorite UX research tools? Let me know in the comments!
Topics
