Internet Retailer has tapped the insights of Digital Performance experts from Dynatrace to gather a monthly view on Digital Competition and a landscape analysis for online retailers. 

We understand that while digital convergence is well under way, many organizations are still in the early days. Retail competition is moving out of the malls and onto our smart phones and tablets. While traditional “big box” and “anchor” retailers have traditionally owned the largest storefronts in the mall, with digital convergence the storefront is 1920×1080 pixels wide for every retailer. With such a level playing field how will retailers differentiate themselves from each other? By the digital experience they deliver to consumers.

Working with Internet Retailer, our team at Dynatrace has produced a ranking system comprised of 10,000’s of tests running from hundreds of locations across the U.S. to track the performance of top online retailers. This ranking is comprised of sub rankings for:

• Mobile: tests pulling up the mobile page across mobile carriers,
• Last Mile: test running from real end-user machines running in homes and businesses across the U.S.,
• Web: test hitting the home pages of each retailer and
• Transactions: tests which search for a product, add it to a cart and checkout.  

Each ranking is created by examining:

1)    Response Time: how fast to execute that transaction.

2)    Availability: how successful were we at executing the tests.

3)    Consistency: how much variability did we see in the results. 

Based on the sub-rankings, we arrived at who are the most competitive retailers in the nation, in terms of online performance.

So why do we focus on performance? Online performance from an end-user perspective is immensely important for every online retailer. Studies have shown that the longer end-users wait for a page to load, the more likely they are to abandon that page and look elsewhere. 

So we are often asked: “How can a few seconds or tenths of a second make any difference?” When was the last time you saw 5 seconds of nothing between commercials on your TV or heard 5 seconds of silence on the radio? How fast can you switch channels? 

Sometimes consumers don’t take conscious note of these things, but we can perceive just seconds as a very long time depending on the context. When was the last time you opened a page on your phone and it took several seconds before you could click on something? That always makes consumers frustrated. Have you ever tried to click on something on your tablet and ended up selecting the wrong link because the page was still loading? Simple issues like this definitely make users more receptive to checking out a competitor’s site next time.

What does slow or inconsistent performance mean for retailers? If the end user is frustrated and abandoning a session before the page loads it directly impacts online revenue. It also impacts retailers when there are peak shopping periods like Black Friday and Cyber Monday. It means retailers have to invest massive amounts of money to support increased traffic. Typical websites are not infinitely scalable and at some point they will collapse if you apply enough load to them. Many of these retailers represent significant brands, and any time the site is slow or fails and causes frustration to an end-user it damages brand equity.

Now that we have shared a little about what we are examining and why, lets have a look at rankings for June 2015.

This ranking shows retailers’ current positions, whether or not they have improved, fallen behind or stayed the same since the previous month. The Top 3 Online Retailers for June 2015 are: Costco, Amway and Apple. Some of the most improved retailers are: Musicians Friend (moving 12 spots), Cabelas (moving 7 spots) and Estee Lauder (moving 7 spots).

The data behind the rankings is comprehensive, as explained above looking at a number of dimensions (Mobile, Last Mile, Web and Transactions). Retailers will often ask: “What can we do to become more competitive?”

A smart approach is to focus on technical details like “over the wire” (what gets pushed and pulled over the internet), Key Delivery Indicators (KDIs) like byte count (how heavy the page or transaction is) or the number of objects and connections requested (how complex the page is) or even the number host servers being called (the number of third parties being used every time the page loads). These KDIs are technical but we can see clear trends in how they impact retailers in terms of their rankings. 

Here is a look at some of the detail behind the Mobile Retail Sub Rankings. We can see a definitive trend that retailers with more complex pages (lots of objects and connections), more third parties (lots of host servers) and more content to serve-up because of heavier pages (number of bytes) all perform slower.

Let’s show a real world example of how this works. Here is the “over the wire.”

Mobile Response Time for Apple’s store: At the beginning of July they reduced their page weight by about 50% and the result appears to be going from 6.5-7 seconds to 5 seconds on average.

This is a big improvement by doing nothing more than just optimizing content being delivered.

While optimizing content delivery can definitely improve a retailer’s competitive position, where it really counts is how the application on the server is performing. Some “over the wire” content can be cached in the browser for returning visitors and some can be pushed closer to the end user through a CDN (Content Delivery Network). Server side processing is where retailers really need to pay attention as seemingly small details in how the application was coded or how it is being hosted can have a huge impact on end users.

The Dynatrace Retail Performance team also looked at some of the most sophisticated tests we run – transactional tests. These tests exercise the entire application by loading a home page, searching for a product, looking at the product details, adding it to a cart and going to the checkout page. We looked at the average W3C request time (sometimes known as First Byte Time). This is the amount of time it takes the server to respond to requests. Another way to look at this is Server Think Time.

The retailers with the fastest server responses are Sears, Barnes & Noble, Victoria’s Secrets (tied) and Apple.

The amount of complexity inside these applications is astounding, but with best practices (like using a DevOps methodology) and the right instrumentation, these applications can be managed and optimized to provide the most competitive digital experiences for end-users.

This is the first of ongoing monthly reports resulting from collaboration between Internet Retailer and Dynatrace. Each month we will look closely at specific sub rankings, retailers, or KDIs. Retailers should take note that now is the time to focus on where they can make improvements, because before we know it, applications will be locked down in anticipation of the holiday season.

Remember; the fight to deliver great digital experiences for end users is a level playing field, where success will not be determined by how much space a retailer has in a mall, but how fast, easy and reliable it is for a user to find the product they want online – with any device – and checkout without a hitch.

Dynatrace provides web performance monitoring services to 123 retailers in the 2015 Internet Retailer Top 500.

Favorite