High-end menswear seller Bonobos’ e-commerce site went down in 2011 on Cyber Monday, the online shopping holiday that falls on the Monday after Thanksgiving and is one of the retailer’s peak sales days of the year, says chief technology officer Michael Hart. But the next year did not bring a repeat calamity. After spending a few weeks testing Bonobos’ projected site performance with different visitor loads using technology from Soasta Inc., the retailer on Cyber Monday 2012 handled, without a hitch, eight times more site traffic than it expected, launching it into its biggest holiday season yet, Hart says.
Soasta’s Internet-hosted system allows retailers to run various simulations of site traffic and activity, such as 10,000 visitors browsing five product pages and filling out multiple forms on a site at once. Bonobos pays Soasta an hourly rate to run the tests, costing far less that what it would to run similar tests in-house with the retailer’s own staff and extra servers, Hart says. Without revealing the exact price, he says Bonobos spends at least $10,000 on Soasta annually, “but given the revenue it’s protecting for us, it’s money very well spent.”
Retailers can test 100 visitors browsing on a single type of device for free using Soasta’s CloudTest Lite version, the vendor says. They can also opt to pay for larger tests starting at $1,000 per hour, which rises according to the number of servers needed to run the tests. The on-demand service can also be priced by the month, project or annually according to customer needs. Soasta’s product for testing mobile app performance, TouchTest, starts at $2,000 per mobile device, sold in annual subscriptions, the company says.
“Our [general] test simulates customers going through a whole purchase cycle on the site: browsing, signing in, building carts of various sizes and checking out with various forms of payments,” Hart says. Bonobos repeats the test, changing such factors as the number and frequency of site visitors, he adds. The retailer spends about a week putting all its pre-holiday tests together, then starts running them, he says. A peak traffic test takes Bonobos roughly half an hour.
After each test, Soasta provides a report with diagnostics about how fast the site loaded with varying amounts of visitors, details of the errors that occurred, and the computing power and memory used, which can contribute to slowdowns, Hart says. This allows Bonobos to identify places on the site that cause shopper bottlenecks, which can sometimes be fixed by adding extra server capacity, he says.
Sometimes, however the solution isn’t so straightforward. For example, Bonobos learned in one test before the 2012 holiday season that a form near the end of the checkout process was failing to copy the address information a customer had already entered in a previous form, causing an error message, Hart says. Bonobos’ engineers fixed the forms and, with them, the performance issue. Soasta also provides engineers to help design tests and saves the tests from one year to the next, which helps cut down on the time and effort spent setting up the next time, he says.
Cyber Monday alone represents more than 15 times a typical sales day for Bonobos, Hart says. At the height of the busy shopping day, which occurred just before midnight on Nov. 26, traffic at Bonobos.com reached 64 times its normal level, he says. The retailer started using Soasta in 2011, but too close to the beginning of the season to prevent that year’s Cyber Monday crash, he says.
With other holiday sales events each also drawing about 10 times the retailer’s normal sales per day, the impact of smooth site performance on business is significant, Hart says. No. 505 in the Second 500, Bonobos had 2011 web sales of $15 million, according to Internet Retailer estimates. The company does not disclose its revenue.
Bonobos is not alone in simulating peaks in visitor traffic. “Adoption of testing in the cloud by e-retailers is increasing significantly and has been for the past 18-plus months,” says Melinda-Carol Ballou, an analyst at consulting firm IDC Financial Insights. “Demand for adaptive infrastructure—given the peaks and valleys in retail purchase cycles—makes cloud testing an excellent fit for organizations urgently seeking the opportunity to scale tests without having the sunk costs of purchasing and maintaining their own testing and support platforms.”
Ballou adds that more retailers deploying apps on both mobile platforms and the web increases their demand for flexibility in performance testing. Soasta, with its free lite version and rates starting at $1,000 per hour, provides a lower-cost alternative to traditional testing services, she says. That, along with the vendor’s web-based technology and its growing focus on mobile support, helps set it apart from competitors, she says.
Favorite