Crutchfield Corp. had a problem with product comparisons.
The web and catalog retailer of car stereos and other electronics was puzzled that online shoppers searching for car parts were selecting vehicle types from the menus and then, for some odd reason, navigating to home theater product pages, says Archie Miller, the retailer’s senior manager of web design and development.
That’s when the testing came in. The company assembled a consumer focus group to test how consumers use the retailer’s e-commerce site, Miller says. Crutchfield soon found that mouse movements could too easily activate the drop-down menus, which led consumers to visit pages they didn’t intend to. After the test, Crutchfield changed its design so it no longer allows cursors to do that.
About a year ago, the retailer wanted to give its online shoppers the ability to compare products side by side. Crutchfield tried out an online chart, complete with bullet points to point out the differences between products, but soon learned that the charts confused consumers. The company decided to add a button that consumers would click to call up product comparisons that offered more clarity than the charts. This time, the company outsourced the testing of the new feature to the vendor UserTesting.com.
The vendor’s tests typically involve a handful of consumers who, guided by questions submitted by the retailer, browse sections of the site as would online shoppers. The company records cursor movements and also the play-by-play descriptions, captured via microphone, of the testers using the site, Miller says. “The testers are very good at out-loud thinking,” he says. The UserTesting teams usually include between five and eight testers.
Miller says it is pointless to have more than five to eight testers on a team because their responses tend to get repetitive and veer from the questions Crutchfield wants answered. “You get diminishing returns with more people,” he says. “They start to identify so many issues you can’t deal with them.”
Miller says outsourced tests have two advantages over tests organized internally: Speed and cost. UserTesting.com results tend to arrive within 24 hours, while in-house tests require that Crutchfield assign at least one employee to monitor the group, which itself must be assembled. As well, an outsourced test usually costs a quarter of what it costs the retailer to do the test in-house. Crutchfield buys blocks of testing time from the testing vendor.
Miller, however, would like to see certain improvements for outsourced tests. He would like to be able to follow up on the answers and findings from testers, and to be able to monitor a live test.
Crutchfield is No. 108 in the Internet Retailer Top 500 Guide.