Online retailers often test whether they can increase sales by changing the color of a Buy button, moving or resizing product images, or adding calls to action like “Shop now.” A new study suggests most of those changes have little impact.

Research by personalization technology provider Qubit examined 2 billion consumer interactions with the vendor’s retail and travel clients and 120 million purchases. A few techniques improved results notably—including indications that a product was in short supply or trending, praise from other consumers and limited-time offers.

But there was practically no impact on average from such common website design modifications as redesigning pages, changing colors, resizing elements or adding calls to action. In fact, only four of 29 common conversion rate optimization techniques studied improved revenue per visit by as much a 1% when averaged across the entire sample.

“People might have a big laundry list of things they’re doing, and a lot of them don’t work for driving revenue,” says Jay McCarthy, Qubit’s vice president of product marketing. While a particular modification that performed poorly in this study may well work for some retailers, he says these results can point retailers to the kinds of changes most likely to increase sales on a typical e-commerce site.

The Qubit study suggests that combining the most effective techniques could increase revenue per visit by as much as 6%. What’s more, when those techniques are personalized—for example, highlighting products trending in a particular area to segments of shoppers most likely to be interested in those products—the impact on revenue is three times greater.

In the report released today, Qubit also disclosed results of two surveys—one of more than 1,000 consumers in the United States and the United Kingdom, and another of nearly 250 senior marketers. The marketers cited as obstacles to personalization a lack of necessary data (42%) and not having the required tools and technology (31%).

The report highlights the disparity in results on mobile versus desktop sites: only 37% of marketers reported that their results were as good when consumers visited their sites on smartphones as when they used desktops. And around 10% said their results were at least 40% worse when shoppers browsed on their phones.

That’s a serious problem for retailers given that a majority of traffic to many retail sites comes from smartphones, McCarthy says. “If we’re saying we can expect to cut all these conversion rates in half because everyone is on mobile, that doesn’t bode well for retailers trying to expand their success,” he says.

There were other signs from the consumer survey that online shoppers are not very loyal, even to their favorite e-retail sites. While three-quarters of the consumers said they do the majority of their online shopping at five or fewer websites, only 16% said they would remain loyal to those web stores if they failed to deliver quality merchandise and good service.

The four techniques that improve revenue per visit the month and the mean uplift in revenue per visit, according to the Qubit survey are:

• Scarcity (highlighting items that are low in stock), 2.9%
• Social proof (leveraging other consumers’ actions to highlight products that are trending or popular), 2.3%
• Urgency (encouraging action before a deadline), 1.5%
• Abandonment (techniques that aim to capture shoppers before they leave a site after indicating intent to purchase), 1.1%.

Here is the complete list of the 29 techniques studied, with the mean increase in revenue per visit: