What are your best customers doing?
That is the #1 question I hear from customers on a day-to-day basis. How do others companies do optimization and testing? It’s a great question.
Based on thousands of interactions with Optimizely customers and four years of enterprise enablement, I can confidently point to five traits that all best-in-class optimization teams possess:
They’ve established a habit of optimization.
There is a clear “owner” of the optimization program.
The C-suite cares about optimization (and acts on it).
Optimization goals are aligned with key company metrics.
They make it fun.
1. They’ve established a habit of optimization.
“We are what we repeatedly do. Excellence, then, is not an act, but a habit.”
—Aristotle
Today, Aristotle’s adage above still rings true. It also highlights a cornerstones of all successful optimization programs: HABIT.
In Charles Duhigg’s The Power of Habit we learn that habits are a three-step loop: cue, routine, reward. The cue is what triggers the routine. Thankfully, when Google launched Google Calendar in April of 2006 humans obtained an easy way to design their own cues. Enter: the “repeating” meeting for the win.
Sounds trivial, but all of our best customers embrace some form of the recurring meeting format. It is the forcing function that furthers their optimization endeavor.
Do you have a repeating meeting on your calendar to create your company’s optimization habit?
A few other examples of habit-forming meetings:
Weekly optimization standup (Forbes)—technical review of pre-launch experiments
Weekly results review (HomeAway.com)—identify learnings from completed tests
Quarterly KPI evaluation (Crate & Barrel)—goal alignment, deliverables for the quarter
Weekly prioritization meeting (TicketMaster)—stack rank based on effort vs. impact quadrants
2. There is a clear “owner” of the optimization program.
When it comes to execution, a world-class optimization program relies on people. Humans who work to design, manage, and ultimately execute against a plan.
Whether your team is an army-of-one or 50+ people, the linchpin is most certainly the program manager, e.g. the optimization “owner”.
Ask yourself: Who wakes up in the morning and thinks about optimization at my company? If there isn’t an owner, assign one or hire one. Otherwise your optimization program will likely flatline.
Here is what this role typically looks like on LinkedIn:
This critical role takes the time to:
Crowd-source testing ideas from the org
Consolidate them in a testing backlog
Prioritize the backlog based on KPIs and effort vs. impact
Communicate with—and get buy-in from—stakeholders
Green light tests for execution in a centralized project plan (see below)
Track and communicate results and inferred learnings
Iterate. Use what was learned to inform the go-forward strategy.
This is a lot of work for someone who isn’t 100% committed. For this reason, they can’t be a part-time lover (yes, that’s a Stevie Wonder reference on an optimization blog).
If you don’t have the resources internally, its not the end of the world. Look to evaluate Solutions Partners who can help steer the ship for you.
Sidenote: Best-in-class programs also have substantial access to developer/IT resources. If you don’t have this benefit, it might be time to make some new friends in that group. Arming yourself with Red Bull, quirky dev humor, and knowledge of the new Civ will earn you major points. Developer support of your optimization program will add substantial octane to the engine. Rev it up!
3. The C-Suite cares about optimization (and acts on it).
If your leadership team cares about A/B testing and optimization you’re in good place. But talk is cheap, so we look for clues that they actually walk the talk. Does your leadership team:
Allocate strategy & technical resources to optimization?
Review results regularly?
Suggest ideas for testing?
Say, “I don’t know, let’s test it”? or “We should test that.”
Provide guidance and direction on quarterly optimization goals?
Prevent certain stakeholders from blocking the deployment of winning tests?
Without executive sponsorship, building a best-in-class optimization program can be a scratch & claw uphill battle. The Roadmap to Building a Testing Culture eBook contains a number of ideas to get their buy-in.
4. Optimization goals are aligned with key company metrics.
In our 6 Best Practices article we highlight “defining quantifiable success metrics” as the #1 driver of success. But the industry leaders take it a step further: their testing goals are not only well-defined, but also aligned with their key company metrics. For example, a retail website like The Honest Company would align their goals as such:
This alignment helps them deprioritize less-relevant tests by keeping their eye of the prize, i.e. improve Customer Lifetime Value, and ensures your testing program doesn’t go off the rails into random-behavior land.
(Disclaimer: I don’t agree with the premise of this cartoon at all, but I do think its hilarious.)
5. They make it fun.
The best-in-class companies make optimization fun.
Three weeks ago I attended the Zappos Culture Camp: a 3-day deep dive into their special sauce that’s fueled their ridiculous growth to $1B+ in revenue and compelled Amazon to acquire them in 2009 for 40x EBITDA. It’s also worth mentioning that Zappos AOV is ~$130 vs. Amazon’s ~$50. Boom goes the dynamite.
Impressive numbers aside, Zappos is unique for another reason: they’ve built a company culture that intentionally values fun, e.g. Zappos Family Core Value #3: Create Fun and a Little Weirdness.
Here are a few ways we’ve seen optimization made fun:
Submit an idea competition! (IGN)
Test of the week/month (HomeAway.com)
Quarterly A/B Headline Hackathon (CNN)
Company-wide recognition for the person that suggested a winning experiment (A&E)
Host a quarterly off-site and invite optimization experts to speak (Mozilla)
I hope you’ve enjoyed reading this! If something resonated with you—or I completely missed something—please post a comment below.
Don’t forget the technical side…
The 5 traits above are mostly organizational & non-technical in nature. I’d be remiss to not mention the technical side of the optimization yin-yang. Here are the 3 Technical Best Practices we recommend based on best-in-class optimization programs:
1. They make their product and visitor data available client-side.
This is a game-changer.
This could be in the form of cookies, Javascript variables, custom tags, etc.
This is really important because you—as a technical person—have now enabled non-technical folks to leverage the data that’s available.
2. They really understand all the nooks & crannies of their site.
What cookies does your company already use? What’s in those cookies?
How do you leverage the cookie data for optimization and personalization?
Knowledge of their website(s) page hierarchy and URL structure
Strong grasp of the moving parts of your site: dynamic content, AJAX, etc.
3. They ensure that existing processes doesn’t stand in the way of velocity from a development perspective.
Streamlined QA and development process due to the reduction of red tape
Is it really necessary to create a fully functional design and requirements doc for a CTA or image change?
Don’t let perfect be the enemy of good.