Comments on: Competitor usability testing – is it useful? http://tim-gregory.com/2009/07/competitor-usability-testing-is-it-useful/ personal blog about all things Agile, SEO, Web Development, Scrum, Usability, Photography and whatever else I blurt out Tue, 03 Apr 2012 16:58:49 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Tim http://tim-gregory.com/2009/07/competitor-usability-testing-is-it-useful/comment-page-1/#comment-26 Tim Tue, 14 Jul 2009 12:01:36 +0000 http://tim-gregory.com/?p=83#comment-26 Thanks for the insight Helga... I don't place as much stock in competitor usability testing as you do, but we're not taking opposing views on the subject - my opinion is that more value is gained from multiple tests on own sites rather than tests on competitors. The scenario you are describing (testing particular UI) could have been done on a competitor site or a paper-prototype or a rough version of the site under development. The fact that it was a competitor site doesn't seem to be that relevant? Thanks for the insight Helga… I don’t place as much stock in competitor usability testing as you do, but we’re not taking opposing views on the subject – my opinion is that more value is gained from multiple tests on own sites rather than tests on competitors. The scenario you are describing (testing particular UI) could have been done on a competitor site or a paper-prototype or a rough version of the site under development. The fact that it was a competitor site doesn’t seem to be that relevant?

]]> By: Helga http://tim-gregory.com/2009/07/competitor-usability-testing-is-it-useful/comment-page-1/#comment-25 Helga Tue, 14 Jul 2009 11:17:03 +0000 http://tim-gregory.com/?p=83#comment-25 Hi Tim I disagree with your statement that running usability tests on competitors are not useful. I have done plenty of these kinds of tests and it comes into it own when you are testing a piece of functionality the team want to introduce into the site and a competitor is doing it already. Testing whether customers have adopted the functionality is great and gives a lot of insight into how your own site could be affected by the additional functionality. We constantly faced by marketers that see pieces of functionality on other sites and request the inclusion of these, a competitor usability test will provide the necessary input on whether its valued or not by your customers. A good example of this was the introduction of tabs into the UK market: we were asked to consider the changing of our primary navigation to tabs as per all our other competitors, we went into a competitor test only to see none of the customers clicked or even noticed the tabs on our competitors at all, we ran a series of tests and found consistent results. We did not include this functionality and found 6 months later all our competitors switching their tabs off, when asked they said they found they had no click-throughs on the tab navigation. Considerable amount of money saved not introducing the functionality that we just would have turned off again. Most E-commerce sites internationally test their competitors on a regular basis against their own site to ensure the functionality offered to their customers are still valued and gives measurable comparisons in terms of a market landscape for customer experience and usability. I have only had positive and valued input in doing competitor usability tests, however you always need to include your site into the mix of competitors to measure your own site's effectiveness against your competitors, if you are tight on budget testing your own sites functionality should always take preference. If budget allows you should test your site against competitors at least once a year to get a competitor landscape and be able to see where you sit in relation to your competitors as far as offering a usable site. However test against real competitors and not just an international mix and test with your own customers and not just a random selection to ensure you get the best results that can be used within your business. These tests need to be well scripted to ensure consistent questioning and results. Apple for apple comparison is key to get good solid findings from such a test, but the rewards for running such a test far outweigh not running it. Hi Tim
I disagree with your statement that running usability tests on competitors are not useful. I have done plenty of these kinds of tests and it comes into it own when you are testing a piece of functionality the team want to introduce into the site and a competitor is doing it already. Testing whether customers have adopted the functionality is great and gives a lot of insight into how your own site could be affected by the additional functionality.

We constantly faced by marketers that see pieces of functionality on other sites and request the inclusion of these, a competitor usability test will provide the necessary input on whether its valued or not by your customers.

A good example of this was the introduction of tabs into the UK market: we were asked to consider the changing of our primary navigation to tabs as per all our other competitors, we went into a competitor test only to see none of the customers clicked or even noticed the tabs on our competitors at all, we ran a series of tests and found consistent results. We did not include this functionality and found 6 months later all our competitors switching their tabs off, when asked they said they found they had no click-throughs on the tab navigation. Considerable amount of money saved not introducing the functionality that we just would have turned off again.

Most E-commerce sites internationally test their competitors on a regular basis against their own site to ensure the functionality offered to their customers are still valued and gives measurable comparisons in terms of a market landscape for customer experience and usability.

I have only had positive and valued input in doing competitor usability tests, however you always need to include your site into the mix of competitors to measure your own site’s effectiveness against your competitors, if you are tight on budget testing your own sites functionality should always take preference.

If budget allows you should test your site against competitors at least once a year to get a competitor landscape and be able to see where you sit in relation to your competitors as far as offering a usable site. However test against real competitors and not just an international mix and test with your own customers and not just a random selection to ensure you get the best results that can be used within your business.

These tests need to be well scripted to ensure consistent questioning and results. Apple for apple comparison is key to get good solid findings from such a test, but the rewards for running such a test far outweigh not running it.

]]>