The Importance Of Benchmarking In User Experience Research

by | Nov 3, 2018 | User Experience

User experience is always going to have a place in the world of marketing. Putting the user first and considering their journey and experience of your product or brand is an essential component of business success, but have you ever considered how effective your user experience research, designs or implementations actually are? You might have even been asked if you can present a case study that showcases the effectiveness of your work. If not, consider yourself lucky but don’t discount the fact that your time will surely come.

Enter the importance of benchmark studies. The answer to your user experience outputs and how important they are to the entire user journey. Benchmark studies give you the opportunity to test your website or application and how it performs in certain areas and functions. It also helps you identify where it is falling short and where competitors have the leg up over you. The best part about this process is having benchmarks in place that can be used for comparison time and again.

Here are some tips that will prove useful when defining your own user experience research and design benchmarks.

1. Decide What It Is You Want To Learn More About

This is the foremost part in your benchmark study that will help you identify the direction. Do you want to learn about website performance? Product performance? Application uptake and engagement? Decide this first so that you know what you will be studying. Hone in on particular features or processes to get super detailed information on how they are working for your users.

2. How Often Will You Run Benchmarking Studies?

How often will you run benchmarking comparisons against your competitors or against previous iterations? Decide this upfront and stick to it. You will need to bring neutral participants on board to help you get the answers you require, so factor this into your budget. This might help you decide how often you want to do it. If you have multiple releases coming up, you might want to run a study prior to each release. Aim for at least a quarterly study to ensure you have ample to data for comparisons.

3. Define Your Target Participants And Offer Instructions

You will be getting participants to test out elements of your product, application or website so you want to give them as little direction as possible in order to see how simple and straightforward the experience is. For example, if you own an online store that sells socks, the instruction here could be “But a medium pair of socks”. If you’re selling a service, you could offer the instruction, “Sign up for a free trial”.

You don’t want to give any indication of the actual process because you’ll be leading the participant and your insight will be skewed. Let them forge ahead so that you can get the information you need.

4. Quantify Your Tasks To Measure Success

Each task that you set out for your participants needs to be measured. Certain metrics will help you determine your benchmark. Some of these might include:

  • Task completion
  • Time spent on task
  • Issues encountered on task

This needs to be coupled with feedback that is collected from the participant at each stage of the study. For example:

  • Single Ease Question (SEQ) is a question posed to the participant after each stage that measures the ease of the task. Whether it is a success or failure, the question is asked and answered
  • Subjective Effort Mental Questioning (SEMQ) asks the user to rate the mental difficulty of each task
  • SUM collects the data from SMEQ, SEQ and the time spent on the task and combines it into one figure to represent the usability and experience of said task
  • Confidence allows users to identify how confidently they thought they completed each task

Then you can also combine this with questionnaire metrics, such as:

  • System Usability Scale (SUS) measures perceived user experience as defined by a questionnaire
  • Standardised User Experience Percentile Rank Questionnaire (SUPR-Q)
  • SUPR-Qm – a questionnaire to measure the mobile app user experience
  • Net Promoter Score (NPS)

There are existing statistical reports that you can compare your own data to, but remember you can also compare directly to your competitors or refer back to your previous benchmark studies. It’s one of the most effective ways to measure your user experience research and design success and hopefully one that will help you see where improvements have been made.


Cross-Cultural UX: Designing for Global Markets

Explore the intricacies of cross-cultural UX design in our latest blog, where we delve into balancing localization and globalization, enhancing diversity, and leveraging technology to create inclusive digital experiences that resonate globally. Discover strategies for navigating cultural nuances and driving user engagement worldwide.

Gamification: Enhancing UX through Strategies and Insights

Explore how gamification can transform user experience with our latest blog. We delve into effective strategies, advanced techniques, and real-world case studies to show how gamifying user interactions can boost engagement, satisfaction, and business success. Discover the future of UX with us!