For the weekly Keynote Mobile Commerce Performance Index, mobile and web performance management firm Keynote Systems Inc. spends a lot of time discussing the importance of mobile commerce site page load time—the amount of time it takes a page to load from a server to a smartphone. But just as important is success rate, or availability—the percentage of the time a page loads completely and successfully. This is illustrated in the results for the week ending July 22.
“While no site is perfect, and may have the occasional outage from hardware, software or network failures, the sites on the index for the week overall show excellent availability numbers,” says Keynote’s web and mobile performance expert Joe Flake. “18 of the 30 showed 99% or better success rate, and Buy.com at the No. 1 spot showed 100% availability for the week. A handful of companies made double-digit moves either up or down ten or more positions. And each of these had significant changes in their success rate percentage.”
One of those retailers is Walmart.com. It dropped from No. 13 to No. 23, its success rate decreasing from a respectable 99.37% to a less reliable 98.34%. Its load time actually improved, from 8.63 seconds to 7.71 seconds, which shows how much the lower success rate damaged Walmart.com’s standings.
“Walmart.com experienced a large number of connection and timeout errors from our New York AT&T connection, while showing near-perfect availability from elsewhere and on other carriers,” Flake says. “This might imply an issue with a regional, carrier-specific content delivery network provider. There wasn’t a hard outage, just more random error points than would be expected.”
The weighted and combined load time and success rate earned Walmart.com an index score of 538 out of 1,000. Walmart.com would only comment that according to its data its m-commerce site was stable last week and that it did not see any problems with the AT&T network. The differences between the two sets of data could stem from where, when and how often testing was being conducted. Keynote’s methodology is spelled out in detail below. Walmart.com did not elaborate.
Buy.com topped the index with a load time of 3.38 seconds and a success rate of 100% for a score of 965. HSN Inc. had a load time of 4.43 seconds and a success rate of 99.48% for a score of 844. And L.L. Bean Inc. registered a load time of 5.31 seconds and a success rate of 99.69% a score of 842.
The average load time for all 30 retailers on the index was 7.06 seconds, the average success rate was 99.06% and the average score was 674.
Walmart.com is No. 4 in the Internet Retailer Mobile Commerce Top 300. Buy.com is No. 19, HSN is No. 58 and L.L. Bean is No. 52.
Click here and then click on Keynote Mobile Commerce Performance Index Part 1 and Part 2 to see this week’s complete results for all 30 retailers on the index.
Keynote Systems measures 30 representative m-commerce sites exclusively for Internet Retailer. The sites include merchants in multiple categories and channels, and of multiple sizes, ranging from such giants as Amazon.com Inc. to midsized retailers like Toolfetch.com LLC. Keynote tests the sites in the index every hour Monday through Sunday from 8 a.m. through midnight Eastern time, emulating three different smartphones on three different wireless networks: Apple Inc.’s iPhone 4 on AT&T, the HTC Evo on Sprint and the Droid X on Verizon. The HTC Evo and the Droid X run Google Inc.’s Android operating system. Keynote runs the tests in New York and San Francisco.
Keynote combines a site’s load time and success rate, equally weighted, into a single score. Given that both performance and availability are important, the score reflects the overall quality of the home page; a higher score indicates better performance. Scores also reflect how close sites are to each other in overall quality. The index average score is the midpoint among all the sites’ scores.
Favorite