Competitive Testing is a tool that compares a product or service with its competition from the user’s perspective. Used to explore in-depth what already exist in the market to learn about people’s overall experience with a service or product. It is good for an initial research activity, to begin to explore an area of interest by looking at users responses to what other people have designed or - as it is used more traditionally - to evaluate a product or service performance in contrast with its competitors and to help determine design directions.
Comparing the existing offer is commonly done as a Competitive Analysis for strategic or marketing planning to analyze business financials such as market share, profit, price, quality, and keep updated on what’s happening in the market. This approach doesn’t usually consider a user-centered perspective. Competitive Testing method gives the opportunity to understand the user’s point of view. It is a regular practice in web development to discover opportunities and inform decisions as the medium allows easy access to this kind of data.
Competitive testing focuses on the user’s behavior, in contrast to other methods like interviews of focus groups. It provides insight about similarities and differences, patterns, weakness and strengths, users’ preferences, etc.
Advantages: learn and benefit from what other’s are doing right & wrong; get user’s point of view from their experience; understand the market, what already exists, and how your product or service can differentiate.
Disadvantages: access to the competitor’s product or specially a service may be restricted; risk of bias for the product or service you are researching for; difficult to find representatives from your target audience.
- Define the purpose and specific dimensions to evaluate and identify your target audience.
- Select what products or services you are going to compare according to your goals.
- Develop a test plan determining the type of information you want to collect and how, where and who is going to collect it.
- Run the user testing for the products and services selected.
- Analyze and synthesize the information drawing conclusions, recognizing patterns and best practices as well as what does not work.
Review the tool guide, below, for a more detailed process.
Scroll down this web page to “Examples of Competitive UX Benchmarking with UserZoom” to see two examples on the digital industry.
In this article they analyze interaction with banking channels. It is interesting to see how as a method they compare what people declare as a preferred channel versus what statistics show people actually use.
This article shows a Comparison of the Usability of a Laptop, Communicator, and Handheld Computer, using observation and posterior surveys to determine how user-friendly and ergonomically correct these devices are.
Thomas Myer, “How to conduct a Web site competitive analysis, Hints and tips for doing it right”, October 1st 2002.
Amy Schade, “Competitive Usability Evaluations: Learning from Your Competition”, December 15th 2013
Martin, Bella, and Bruce M. Hanington., “Universal Methods of Design 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions”, January 2012. See Competitive Testing description on pg. 36-37