R.J. Talyor, CEO, Quantifi
A test is not an experiment. While these two words are used interchangeably by retail marketers, the difference is more than semantics. There are many stark differences between the mindset, process and outcome of one versus the other. A test is not an experiment, and an experiment is not a test.
Experimentation is done without a fear of failure or expectation of outcome. Testing is done with with an expectation of a winner. Experimentation leads to new, new things and while testing validates assumptions. Experimentation is dangerous, while testing is safer. Retail marketers want to get credit for experimentation, but they’re held accountable to the funnel and KPIs (impressions, traffic, leads, purchases, revenue), so testing and experimentation become like reading and regular exercise: things we say we do–knowing that we should do more to warrant the claim.
In retail and e-commerce businesses, both testing and experimentation are seen as luxurious, and practices to be implemented when you have extra: extra time, money, resources, energy. But instead, e-commerce and retailers should embrace testing and experimentation as regular practices, like executing campaigns and pulling reports. Experimentation to discover, testing to validate. Rinse and repeat.
It’s critical to understand the differences between tests and experiments, and know where to plug them into your weekly work. Let’s compare:
Both tests and experiments are critical for the growth funnel, with experiments discovering new tactics that tests can validate, then leading to further optimization at scale. But knowing the difference between these two approaches can help you advance your organization’s learning of what works to drive revenue.
Kasey Lobaugh, chief retail innovation officer and omnichannel retail practice leader for Deloitte Consulting LLP, recently said in an interview with Entrepreneur, “The way you win is you find opportunity, you find where there’s an unmet need, where there’s scarcity or where you can build perceived scarcity, and you go after it in an unapologetic, very focused way. But recognize that those windows of opportunity open and close at a faster rate when the competitive fervor is high. So being nimble, to me, means being able to find those, take advantage of them, move out of them and move on to new ones, fast.” This is exactly what a culture of experimentation is all about — it’s a cycle of experimentation, learning, and iterating to find untapped opportunities.
As marketers in retail, we struggle with patience. We want to skip the iterative steps. We want the winner, the silver bullet. So we try to cut corners or mimic competitors to get ahead. But to discover the game-changers, we have to own up to investing in growth. Here are a few ways your can begin to invest in experimentation and build a culture of innovation through experimentation in your organization:
1. Establishing an experimentation quota and budget. This budget can be any amount (start with 5% of the digital budget), but it’s reserved for trying new, new things.
2. Establishing a testing discipline. Tests can be done in-line with normal campaigns and range from small to large—12 hours is typically all your need to test two subject lines, headlines or CTAs [calls to action].
3. Make learning a priority for everyone in marketing, and tie incentives or bonuses to those that are meeting and/or exceeding those goals.
4. Celebrate successes and failures of both experiments and tests. Try awarding a trophy for innovation, success and failure alike.
5. Share your learnings! Blog about your tests and experiments, your discipline, share them internally and externally (as you can) and don’t be afraid to say “sorry” where a mistake is made.
Like any change, the hardest part of testing and experimenting is establishing the habit. But a new habit is just two weeks away! Setting up clear definitions for both tests and experiments enable marketing departments to take advantage of the new world emerging around us. What will VR [virtual reality], chatbots, blockchain, social apps, mobile and other innovations do for your business? Start experimenting and testing!
Quantifi specializes in digital marketing experimentation across social media.
Favorite