Blog
Viewing entries in
Customer Success
Story of leveraging machine learning to predict—and reduce—churn (SaaS terminology for when a customer leaves). Our approach pitted two ML models against each other: XGBoost vs. Random Forest. The latter emerged victorious, and the model output (csv file from a Python notebook in Mode) was integrated both technically (Salesforce) and operationally (via weekly Red Account meeting) for the Clearbit Customer Success team.
I recently had the pleasure of catching up with one of my old teammates from Optimizely. He's doing well: director-level position, managing a growing team, and meanwhile hyper-focusing on how he can take it to the next level as a leader.
So I asked him a question:
"If I were to meet with your team over drinks, and I asked each of them to describe you in three words, what would you want them to say?"
Simple question. Long, thoughtful answer.
His final answer:
"Caring, consistent, and challenging." [Three C's FTW]
While the answer was revealing, the true value was in talking through the various tradeoffs one makes as a leader:
Friend vs. boss
Lenient vs. stern
Strategic vs. tactical
Inspirational vs. directive
Spontaneous vs. predictable
And while none of the above are mutually exclusive, a leader has limited time. Very limited. So you have to explore: what is your management philosophy? How do you want to be perceived? Even. . . remembered?
What are the 3 words you would want your team to describe you?
If you're still staring instead of thinking, below is a insightful list of questions your team is likely asking themselves right now.
Credit due to Fred Kofman’s book, Conscious Business, in which he crafted these questions about communication clarity, mission, shared values, respect and teamwork.
Do I know what is expected of me at work?
Do I have the materials and equipment I need to do my work right?
At work, do I have the opportunity to do what I do best every day?
In the last seven days, have I received recognition or praise for doing good work?
Does my supervisor, or someone at work, seem to care about me as a person?
Is there someone at work who encourages my development?
At work, do my opinions seem to count?
Does the mission/purpose of my company make me feel my job is important?
Are my co-workers committed to doing high-quality work?
Do I have a best friend at work?
In the last six months, has someone at work talked to me about my progress?
This last year, have I had opportunities at work to learn and grow?
The DBT Ventures team hopes this article has been helpful in urging you to proactively control what 3 words describe you as an executive.
The gravity of this challenge can not be overstated: if you aren't able to improve your renewal rates to a sustainable level, your business will bleed out and die.
Okay, maybe that was a bit dramatic but you get the point: the stakes are high. Failing to improve renewal rates can result it:
Decelerating growth
Poor unit economics
A battalion of former customers that never found value in your product and talk about it
Low employee morale
Severe fundraising problems
Bankruptcy
Over the last 10 years, the proliferation of software solutions, business models, and pricing strategies has resulted in a plethora of new revenue retention metrics, so let's first align on our terminology. I am defining renewal rate as:
[ACV: Annual Contract Value]
Let's do an example where we have $5M ACV up for renewal in Q1-2017 and your team is able to successfully renew $4.6M of it, but alas $400k is lost to the churn monster:
Renewal Rate = 100% - Lost ACV/(Lost ACV + Total Renewed ACV)
Renewal Rate = 100% - $400k/($400k + $4.6M)
Renewal Rate = 100% - $400k/$5M
Renewal Rate = 100% - 8%
Renewal Rate = 92%Easy enough, right? Good. Related to this, if you're interested in seeing all the different ways that SaaS companies report these metrics to the Street, I highly recommend bookmarking Pacific Crest's report: Public SaaS Company Disclosure Metrics for Retention and Renewal Rates.
It's become fashionable to report on "net" revenue metrics. "Net" metrics are created by simply combining two numbers, typically expansion and churn. The risk here is that expansion from existing accounts can mask a churn problem. Or as I like to say, "Nets can cover things up."
Therefore, "net" metrics are out of scope for this article as they increase the complexity of something simple: keeping the revenue you already have.
"Screen the team." This strategy has to do with the rigor your company applies to screening customer teams during the sales process. Given the hundreds of software options in the crowded cloud, the real battles aren't being fought over technology, but rather program resources. Similar to organizing an effective sports team, your Sales Engineers and Account Executives must evaluate:
Does this potential customer have the right players on the field?
If not, how do we make a business case for additional resources whether that be internal or external agency help?
Do they have access to developer resources? If so, how many hours per week? How many sprints per release? How many stories per epic? Specificity is key here.
Who is going to own the day-to-day adoption and evolution of the program to use your software, i.e. who is the program manager?
Does the potential customer have strong executive sponsorship and a desire to make this work?
"ROC your renewals." If you aren't familiar with ROC curves, now is a good time to start. The goal of this strategy is to perform data science analysis to identify the 1-2 customer attributes and/or behaviors that are highly correlated with retention success, e.g. renewal rates, and then mobilizing your entire company—marketing, sales, customer success, design, engineering, everyone—to prioritize and improve these metrics:
For Facebook it was 7 friends in 10 days (video, if preferred)
For Twitter its was follow 30 people
For Slack it was sending 2,000 messages
For Zynga {cough} it was 1 day retention (user returns day after signing up for game)
For Dropbox it was adding 1 file to 1 folder
"Pay the retention piper." Incentives matter. No matter how inspirational your company or product vision is, the behavior of your employees are primarily driven by how they are compensated. If your Account Executives are comp'd 100% on growth, they will spend 100% of their time closing new business (and 0% on retention). If your Customer Success Managers are comp'd 100% on usage metrics, they will spend the vast majority of their time on improving usage (and little time on lead generation, customer references, etc).
A lot.
The Twilio IPO has two powerful lessons for all of us, especially Customer Success folks:
Net expansion is king. Boasting 155% dollar-based net expansion, Twilio was able to readily sell shares to willing buyers at a handsome premium. Investors really care about this.
Customer stories are compelling. Twilio lets their happy customers do the talking which creates an undeniable Go-To-Market competitive advantage.
But before we get ahead of ourselves, let's first acknowledge the recent health of the SaaS IPO market:
"Scared and shitty," says one institutional investor when describing the SaaS IPO market. If this trend were to continue, we're looking at the worst SaaS IPO year in six years. Yikes.
But then came Twilio. . .
Bright-eyed and bushy-tailed, Twilio filed its S-1 on May 26th, 2016. But this wasn't just any S-1. This was one of the most unique S-1s we've ever seen (check it out here).
Why was it so unique?
Literally. Before you even get to the f*&%$ Table of Contents, Twilio hits you with this image:
Source: Twilio S-1. DBT commentary: "strong to quite strong"
Revenue growth, sure. Active customer growth, join the club. But 155% net expansion?!
OOHH YEAAHHHH [picture Randy Savage saying this, naturally]
155% dollar-based net expansion means Twilio's customers are expanding 55% annually net of churn. These are best-in-class metrics by any benchmark. One might suspect that large customers could skew this number heavily (WhatsApp is 15% of Twilio's revenue), but even that isn't necessarily a bad thing because their technology is considered essential (WhatsApp isn't even in a long-term contract).
Who does that?! Before you even get to the S-1 summary, you see Travis Kalanick—the f'ing CEO of Uber—raving about Twilio. Your welcome, Goldman Sachs. I-bankers dream of an investor roadshow like this.
And just in case you hadn't heard of Uber, Affirm, Nordstrom, or ServiceNow. . . Twilio includes another full page of customer testimonials to hammer home the point.
Diversity FTW.
What was the financial impact of all this goodness? In short, they crushed it. Twilio is currently valued at $3.12 billion (as of 7/6/16). The 10 million shares Twilio offered at $15 per share ($150MM raise) is now trading at $37.93 per share, or up 153% in 9 trading days.
Source: Google Finance
For the sake of comparison, what has the NASDAQ index done this year? Answer: -2.96%
Source: Google Finance
See that V-shaped low-point in late June? That was the Twilio IPO. You can see the (partial) impact it had on sentiment and investor appetite for tech/SaaS stocks. So while the NASDAQ was taking a cat nap for the first half of 2016, Twilio is a beacon of warmth in a frigid IPO market.
Net expansion is king. If you are a Customer Success executive, or individual contributor, take pride in the fact that your impact on churn reduction and expansion revenue is a HUGE ARBITER of your company's future success.
Customer stories are compelling. When your company goes public, what customer's would you put on your S-1? Do you have executive relationships that would go to bat for you?
"To truly adopt new software, customers have to change their habits. Users must convert their aspiration into new routines." —Tomasz Tunguz
As a partner at Redpoint Ventures, Tomasz is no stranger to the challenges software companies face. As an experienced venture capital investor, Tomasz and Redpoint have studied a variety of factors to determine What is the most correlated business metric with Series A valuations?
The answer: negative churn.
For the uninitiated, negative churn means that your existing customers are paying you more (expansion) than they're paying you less (churn), over a given time period. Confusingly, there are many names for this metric, all referring to the same thing: account expansion, negative net churn, net retention, net expansion.
But the takeaway here is: whatever you call it, net retention is the #1 arbiter of Series A valuations.
In order for expansion to outpace churn, your customers have to adopt your software. And that's where habits come in. As Tomasz shares:
"I learned this selling billing and invoicing software to law firms. It was one thing to convince the managing director of a law firm to pay for the software over a 90 day sales cycle. But it was an entirely different matter to educate, convince and convert individual attorneys to use the software. That took far longer." Source: TomaszTunguz.com
But how?
That's the billion-dollar question that every software company is trying to figure out.
As we've cataloged in past DBT blog posts, achieving successful adoption usually starts with empowering a team to build new habits. Two key words here: 1) team, 2) habits.
It's no coincidence that Atlassian's ticker is TEAM. As I write the stock is trading at $22.82 with a market cap of $4.78 billion. Revenue and cash flow look like this:
Source: Google Finance, 5/24/16
Source: Google Finance, 5/24/16
Atlassian—a software company designed for teams—is growing substantially and kicking off plenty of cash because they've figured out something important: how to change customer habits.
But how does a company change customer habits? Based on five years of research, I've found the 3 best ways to change your customer habits are:
Keep the executive sponsor engaged post-sale
Make your product's value so undeniable they can't ignore it
Triggers
Before we unpack these strategies, let's revisit the formula for how habits are created in the first place:
Source:The Power of Habit, Charles Duhigg, 2014
The formula, therefore. . . HABIT = cue + routine + reward
It's a virtuous cycle of a prompted action, the action itself, and then a Pavlovian shot of dopamine to keep the cycle going. Boom. Habit created. New routine established. But back to the how. . .
1. Keep the executive sponsor and developers engaged post-sale
You'd be surprise how frequently the executive who made a software purchase vanishes after the product is bought. Based our DBT research, this occurs in 40% of the time. It's not his or her fault—they're busy. Very busy. Therefore, the challenge is on the account team to keep them engaged post-sale. This strategy is both effective and within your control.
This is doubly true for Product-Led Growth (PLG) companies where the developer is a critical stakeholder and primary user. To keep them engaged you must provide: 1) regular feature & bug updates, 2) clear technical documentation, and 3) accessible feedback loop to help PROD/ENG accelerate product development.
Backstory: The dirty little secret of SaaS companies is that software is incredibly easy to start/buy, but much harder to fully adopt (and therefore harder to reap the benefit of why said software was bought in the first place). Without the executive sponsor's influence, the actual users of the software are faced with a choice: 1) do what they usually do, 2) change.
As you might have guessed, they often chose option one :|
If you focus on a tight Sales >> Customer Success handoff where the executive sponsor and developer stay involved, you will have greatly increased you chances to successfully deploy and change behavior. To do so, we recommend creating a workflow where the executive's goals are clearly documented in Salesforce so the Customer Success team can carry the torch:
2. Make the product's value so undeniable they can't ignore it
This really should be #1 but for the sake of practicality is is #2 because your product's value is (primarily) determined by your product and engineering team. If your product isn't valuable (accordingly to your customers), than this article doesn't really matter because your customers will eventually churn out.
But let's assume your product does create value. Then what?
A few things:
Think about the "Reward"—what makes your customers want to keep coming back and using the product? How does the product reward them for the routine the user has completed?*
Automate it: make that seminal value moment a thing. Celebrate it. Send them an email. A notification. Highlight the demonstrable show of value.
Schedule a Business Review. Pull the usage and results data from your software and showcase the value to the business user AND the Executive Sponsor.
*This is the "Reward" moment for MyFitnessPal (acquired by Under Armour for $475MM in Feb-2015).
3. Triggers
This is the step required to start the new routine. What brings your users back to the product? WHY do they come back? What prompts them to ditch their Gmail inbox and log into your software and DO STUFF?
It's kind of like running. Personally, if I don't set our my running shoes the night before, I likely won't run. It needs to be easy. I need a trigger to brave the foggy cold air at 6am and MAKE IT HAPPEN. The trigger is often simple, but nevertheless very effective is catalyzing action.
In closing:
“We are what we repeatedly do. Excellence, then, is not an act, but a habit.” —Aristotle
Hopefully you can influence your customers habits to help drive adoption and ultimately achieve negative net churn and thereby earn a lucrative valuation so that YOU = HAPPY.
TL;DR
On February 5th the entire tech/SaaS industry got a wake up call: public cloud companies collectively lost $28B in market value on a single day.
Valuation models have changed: markets are rewarding software companies with sustainable growth + a path to profitability
How tech companies navigate this tectonic shift will determine the success (or failure) of their businesses
By now most people have heard the news: On February 5th the entire tech/SaaS industry got a wake up call: public cloud companies collectively lost $28B in market value on a single day.
Even Linkedin (LNKD), a company that did $3 billion in sales last year, got hammered from $205 to $100 per share. So for Linkedin, the week looked like this:
Feb 1 valuation: $24 billion
Feb 9 valuation: $11.8 billion
Market cap incinerated: $12.2 billion
To put that another way, Linkedin lost the equivalent of Nicaragua's GDP in a six trading days.
What the hell happened?
Answer: the way your company is valued, changed—particularly for high-growth, SaaS, cloud-based tech companies. Markets are rewarding software companies with sustainable growth + a path to profitability
How did the overall market perform?
the S&P 500 was down 5.9% in the three months ending 3/1/16
if you look at the light blue and red lines below, you'll see that certain software and SaaS companies were down 26.2% and 24.1%, respectively
Okay, so that happened. But what does that mean going forward?
for starters, cash is king. For emphasis, cash is king.
these companies are moving (very rapidly) to reduce costs
these companies are moving (very rapidly) to improve cash flow
Why are they doing this?
to avoid imminent failure
to earn a better valuation (see: below chart)
the market is clearly awarding higher EV/Sales valuations to companies that expect to be free cash flow positive within one year
And finally we get to the meat of the discussion. What is one to do in navigating this unique—but not unprecedented—market environment? At DBT Ventures we don't try to boil the ocean, but rather focus on four pillars of business:
Ideas
Given the market challenges, a CEO would be wise to prioritize ideas that enable your business to 1) reduce costs, or 2) increase output.
Focus should be placed on "getting back to basics" and doing the simple things to perfection
Company first, team second, individuals third
Data Science
Increase data science effort on unit economics, specifically Customer Lifetime Value and empowering your account teams to understand the economic tradeoffs between how they spend their time (company money) and revenue (customer money).
It's a great time to kill exploratory projects and double-down on DS initiatives that are closely linked with tangible business value, e.g. prospect's likelihood to buy, customer's likelihood to churn/expand
Perform outlier analysis on all company-wide expenses to identify areas of bloated budget with lackluster ROI
Customer Success
Scale, scale, scale—alleviate the dependency on 1:1 support engagements
Get a tattoo: scaling ≠ hiring
Prioritize roles with higher output metrics with an emphasis on automation and 1:many deliverables, e.g. virtual trainings, online resources, personalized email drip campaigns
Attack churn proactively to reduce the headwind on top-line growth
Set a goal to reduce CRC to 10-15% of revenue, or less
Leadership
Adversity is an excellent opportunity to exude strength, resilience, and adherence to your cultural values and ethics
Give merit based comp adjustments to top performers to reduce the risk of attrition
Take the extra time to celebrate wins as a function, team, department, and company—this investment will help improve dour morale
Reward AEs for contracts with payment upfront which will help cash flow
Overcommunicate. Do not turtle shell. Your company needs you now more than ever
Recognize that "constraint yields creativity" and "necessity is the mother of invention" such that you green-light projects that improve scalable output
Lead by example: now is not the time to be lavish or spendy
If your goal is to go public, recognize that the median revenue at IPO is $83 million—set goals to achieve this number while imparting a sense of urgency at the executive level. Mindset: "the last round of funding was our last" given VC money has become more stringent.
Go forth and prosper.
Story Highlights:
There are MANY SaaS metrics to consider these days. Today we will review "C4"—which includes CLV, CAC, CRC, and Churn—and explain why they matter to a SaaS subscription revenue model.
Any one of the C4 elements can blow up a business.
Therefore, we must have a deep understanding of C4, how each element is calculated, and what we can do to manage and improve them.
Want to confuse your board along with all your employees? Start by showing them the below table of 59 different SaaS metrics.
These days, SaaS metrics are abundant if not overgrown. Similar to the investment industry, you can go down the metric rabbit-hole pretty quickly before you realize, "Wait a minute, what are we actually trying to accomplish here?"
But there are four key metrics that rule them all. You guessed it: C4. What is C4?
This explosive composite combines four vital SaaS diagnostics:
CLV: Customer Lifetime Value
CAC: Customer Acquisition Cost
CRC: Customer Retention Cost
Churn
Churn—which typically garners most of the limelight—is the most cancerous, yet easiest to calculate. Churn is typically expressed as either a dollar figure or a percentage of revenue over a certain time period.
For example, let's say a SaaS startup has $1MM in monthly recurring revenue (MRR). Last month, a customer paying $10k/mo churned. Therefore, churn could be expressed as:
$10,000 MRR
Gross churn = 1%
Simple enough, right?
Churn is insidious, even maddening at times. But once properly understood and effectively managed, churn can materially improve how your run your business. The key is to treat every churned customers as an archeologist might approach a dig. The good stuff is down below, and you'll have to institute a process to have these conversations, diagnose root cause, and ultimately arrive at a LEARNING that will improve how you do business.
For example, you will likely lose a customer for product reasons. Perhaps you lacked the feature, functionality or performance they seek. This information MUST reach the ears of the Product and Engineering teams so that they can prioritize such items in their release planning.
Is there anything worst than a churned customer? Yes: when you fail to learn something as a business. That, in other words, is the greatest disservice of all.
Simply put, building a Net Promoter System (NPS) will give your business a sustainable, competitive advantage. This insightful management tool can be used to gauge the loyalty of a your customer relationships. It serves as a modern alternative to traditional customer satisfaction research.
To calculate your NPS score, you begin by asking your customers a simple question: How likely are you to recommend [Your Company] to a friend or colleague? (Scale 0-10). You then bucket your responses based on their score:
Promoters: 9-10
Passives: 7-8
Detractors: 0-6
Next, take the % of promoters (# promoters / total responses) and subtract the % of detractors (# detractors / total responses). The best NPS is 100%. The worst NPS is -100%.
But why does this matter? A few reasons:
Retention: detractors are 2-3 times more likely to churn. Therefore, you should treat detractors like ticking churn grenades.
Growth: promoters account for 80-90% of positive word of mouth. Therefore, you should treat promoters like referral machines which buoy your company's reputation in the marketplace.
Feedback: this is arguably the most important, and the most actionable. Detractors will provide you with feedback on WHY they don't recommend your product, service or company. You will then have a decision: take action, or don't take action. This feedback loop is critical in the evolution and improvement of your functional teams.
Alignment: the NPS framework is a great way to align EVERYONE in your company—from the intern to the CEO—around a common dialect that reinforces a "customer first" mentality.
On a tactical level, DBT recommends the following vendors to build out NPS:
Get Feedback: survey tool (integrates with Salesforce; sample survey)
Salesforce: CRM
Marketo: automated distribution of NPS email with link
Qualaroo: use for obtaining in-product NPS scores
But what does a good NPS look like? Well, it depends on your industry. Since DBT primarily works with high-growth technology companies, we can share that most of our clients are targeting a NPS score in the 50-65 range.
NPS by industry
Some companies, particularly in the mobile space—like Uber—use a modified NPS system. A ten-point scale doesn't design nicely within the limited real estate of a smartphone, so a lot of companies use a five-point scale and require a reason if you are a 1-3. For example, Uber's mandatory NPS feedback loop looks like this:
Lastly, here are several of the companies considered (by some) to be NPS leaders in their space. B2C companies tend to have significantly higher NPS than B2B customers.
[Chuck Berry's birthday was yesterday. The legendary rock and roll guitarist turned 89. Happy birthday Chuck!]
Assuming your Customer Success Manager team has avoided the pitfall of being subsumed by technical support (there should unequivocally be a separate function this), you typically have three schools of thought on how to comp CSMs:
Revenue (broadly)
Product usage & adoption
Discretionary
Revenue: this is arguably the best way because 1) revenue matters, 2) CSMs can directly impact it and achieve upside, and 3) it creates a sense of ownership over their accounts.
Product usage & adoption: this is the second best option. Your customers won't derive value from your software/product without usage. Therefore usage is key to obtaining ROI, earning organizational adoption, and securing renewals and growth.
Discretionary: this option is easy in the sense it requires little overhead for the manager; however, it is highly subjective and doesn't clearly align CSMs with any tangible business impact. Therefore, it is the worst option (although it is quite common).
Side note: In terms of base/variable split, most CSMs we've encountered are on an 80/20 split, e.g. $80k base, $20k variable, $100k OTE. The $20k variable is often paid quarterly ($5k per quarter). Salaries range from $75k (entry level) to $180k (very senior).
Let's unpack the revenue method: CSMs typically own a "portfolio" of accounts. CSMs work with customers to understand their goals, and connect those goals with capabilities of your product the success of which will be increased usage, increased value and ultimately retained/expanded revenue.
Since Chuck Berry's birthday was yesterday, let's say a hypothetical CSM—named Chuck—starts Q4 with 80 accounts paying $5k MRR for a total of $400k MRR or $4.8MM ARR. In each month customers can churn, expand, or renew with no change (flat renewal).
You can therefore calculate a net retention metric for Chuck:
Chuck Berry's hypothetical Q4 performance.
The formula for net retention = 1 + ((Expansion + Churn)/MRR managed). So in Oct-2015, we'd have: 1 + (($3,000 + (-$4,000))/$400,000)), or 99.75%. Sum it all up, take the average, and Chuck delivered an average monthly net retention of 100.04%, or 100.48% annualized (100.04^12).
Now the question becomes: Is that good or bad based on your business model?
Generally speaking, net retention below 100% is bad. That means you have a churn problem and customers are net leaving you. That is a separate conversation. Best in class net retention is 101-102% monthly, or 112.7%-126.8% annualized. Companies that DBT is advising are targeting 101.6% monthly net retention which equals the nice round number of 120% annualized.
A fair CSM comp model might look something like this: if Chuck achieves between 99-100% net retention, he gets his OTE quarterly bonus of $5k. BUT, if Chuck gets 100%, or 101% or 102% he can hit accelerators and earn $7k, $10k, or $15k respectively.
Some might gawk at paying a CSM $15k in a single quarter, but think about what Chuck has done for your business: he has net expanded his portfolio by 2% each month (102%) which is $8k MRR per month or $24k MRR for the quarter. $24k MRR is $288k in additional annual revenue for your business! $15k represents 5.2% of the gain, a modest price to pay for enviable net retention metrics.
Think about THAT next time you're listening to Johnny B. Goode.
Go Johnny go.
What are your best customers doing?
That is the #1 question I hear from customers on a day-to-day basis. How do others companies do optimization and testing? It’s a great question.
Based on thousands of interactions with Optimizely customers and four years of enterprise enablement, I can confidently point to five traits that all best-in-class optimization teams possess:
They’ve established a habit of optimization.
There is a clear “owner” of the optimization program.
The C-suite cares about optimization (and acts on it).
Optimization goals are aligned with key company metrics.
They make it fun.
“We are what we repeatedly do. Excellence, then, is not an act, but a habit.”
—Aristotle
Today, Aristotle’s adage above still rings true. It also highlights a cornerstones of all successful optimization programs: HABIT.
In Charles Duhigg’s The Power of Habit we learn that habits are a three-step loop: cue, routine, reward. The cue is what triggers the routine. Thankfully, when Google launched Google Calendar in April of 2006 humans obtained an easy way to design their own cues. Enter: the “repeating” meeting for the win.
Sounds trivial, but all of our best customers embrace some form of the recurring meeting format. It is the forcing function that furthers their optimization endeavor.
Do you have a repeating meeting on your calendar to create your company’s optimization habit?
A few other examples of habit-forming meetings:
Weekly optimization standup (Forbes)—technical review of pre-launch experiments
Weekly results review (HomeAway.com)—identify learnings from completed tests
Quarterly KPI evaluation (Crate & Barrel)—goal alignment, deliverables for the quarter
Weekly prioritization meeting (TicketMaster)—stack rank based on effort vs. impact quadrants
When it comes to execution, a world-class optimization program relies on people. Humans who work to design, manage, and ultimately execute against a plan.
Whether your team is an army-of-one or 50+ people, the linchpin is most certainly the program manager, e.g. the optimization “owner”.
Ask yourself: Who wakes up in the morning and thinks about optimization at my company? If there isn’t an owner, assign one or hire one. Otherwise your optimization program will likely flatline.
Here is what this role typically looks like on LinkedIn:
This critical role takes the time to:
Crowd-source testing ideas from the org
Consolidate them in a testing backlog
Prioritize the backlog based on KPIs and effort vs. impact
Communicate with—and get buy-in from—stakeholders
Green light tests for execution in a centralized project plan (see below)
Track and communicate results and inferred learnings
Iterate. Use what was learned to inform the go-forward strategy.
This is a lot of work for someone who isn’t 100% committed. For this reason, they can’t be a part-time lover (yes, that’s a Stevie Wonder reference on an optimization blog).
If you don’t have the resources internally, its not the end of the world. Look to evaluate Solutions Partners who can help steer the ship for you.
Sidenote: Best-in-class programs also have substantial access to developer/IT resources. If you don’t have this benefit, it might be time to make some new friends in that group. Arming yourself with Red Bull, quirky dev humor, and knowledge of the new Civ will earn you major points. Developer support of your optimization program will add substantial octane to the engine. Rev it up!
If your leadership team cares about A/B testing and optimization you’re in good place. But talk is cheap, so we look for clues that they actually walk the talk. Does your leadership team:
Allocate strategy & technical resources to optimization?
Review results regularly?
Suggest ideas for testing?
Say, “I don’t know, let’s test it”? or “We should test that.”
Provide guidance and direction on quarterly optimization goals?
Prevent certain stakeholders from blocking the deployment of winning tests?
Without executive sponsorship, building a best-in-class optimization program can be a scratch & claw uphill battle. The Roadmap to Building a Testing Culture eBook contains a number of ideas to get their buy-in.
In our 6 Best Practices article we highlight “defining quantifiable success metrics” as the #1 driver of success. But the industry leaders take it a step further: their testing goals are not only well-defined, but also aligned with their key company metrics. For example, a retail website like The Honest Company would align their goals as such:
This alignment helps them deprioritize less-relevant tests by keeping their eye of the prize, i.e. improve Customer Lifetime Value, and ensures your testing program doesn’t go off the rails into random-behavior land.
(Disclaimer: I don’t agree with the premise of this cartoon at all, but I do think its hilarious.)
The best-in-class companies make optimization fun.
Three weeks ago I attended the Zappos Culture Camp: a 3-day deep dive into their special sauce that’s fueled their ridiculous growth to $1B+ in revenue and compelled Amazon to acquire them in 2009 for 40x EBITDA. It’s also worth mentioning that Zappos AOV is ~$130 vs. Amazon’s ~$50. Boom goes the dynamite.
Impressive numbers aside, Zappos is unique for another reason: they’ve built a company culture that intentionally values fun, e.g. Zappos Family Core Value #3: Create Fun and a Little Weirdness.
Here are a few ways we’ve seen optimization made fun:
Submit an idea competition! (IGN)
Test of the week/month (HomeAway.com)
Quarterly A/B Headline Hackathon (CNN)
Company-wide recognition for the person that suggested a winning experiment (A&E)
Host a quarterly off-site and invite optimization experts to speak (Mozilla)
I hope you’ve enjoyed reading this! If something resonated with you—or I completely missed something—please post a comment below.
The 5 traits above are mostly organizational & non-technical in nature. I’d be remiss to not mention the technical side of the optimization yin-yang. Here are the 3 Technical Best Practices we recommend based on best-in-class optimization programs:
1. They make their product and visitor data available client-side.
This is a game-changer.
This could be in the form of cookies, Javascript variables, custom tags, etc.
This is really important because you—as a technical person—have now enabled non-technical folks to leverage the data that’s available.
2. They really understand all the nooks & crannies of their site.
What cookies does your company already use? What’s in those cookies?
How do you leverage the cookie data for optimization and personalization?
Knowledge of their website(s) page hierarchy and URL structure
Strong grasp of the moving parts of your site: dynamic content, AJAX, etc.
3. They ensure that existing processes doesn’t stand in the way of velocity from a development perspective.
Streamlined QA and development process due to the reduction of red tape
Is it really necessary to create a fully functional design and requirements doc for a CTA or image change?
Don’t let perfect be the enemy of good.
Reposted from an interview with Aircall.io on August 8, 2015.
Optimizely is a San Francisco-based startup and the leader in A/B testing and experience optimization. I recently sat down with Luke Diaz, a manager on the Customer Success team—and founder of DBT Ventures—to share his experience and advice for organizing and measuring a customer success team.
Luke currently leads a 15-person Customer Success Manager (CSM) team in charge of over 80% of Optimizely’s revenue. The CSM team manages launch (onboarding), success management (adoption & value), renewals (retention) and expansion (account growth; in tandem with the Sales team).
The CSM team is one team within the larger 70-person Customer Success team at Optimizely which includes Technical Support, Strategy, Solutions Architects, and Education.
From my (personal) standpoint, Optimizely is very advanced on the topic. Yet every startup—whatever their stage of development or price point—can learn from Luke’s very actionable tips:
Measure the value customers extract from your product
Start customer success with the sales team
Transform your customer’s organization to achieve success
In order to effectively lead the CSM team, Optimizely focuses on 3 metrics:
Customer value derived from the product
Customer satisfaction
Revenue generation
Luke articulates below on how these objectives are translated into processes and culture.
Optimizely tracks the activity of each Enterprise customer: usage logs, number of A/B tests run, etc. Seems obvious – all serious SaaS businesses (should) do that. To do so, Optimizely uses a blend of Totango (a Customer Success Intelligence software) and proprietary regression models built by their data science team, e.g. churn score, upsell score, account potential score, etc.
What’s more, they follow the number of successful experiments (in their case, delivering a clear A/B winner), and, as much as they can they can, actuallycompute the value in $ generated by successful experiments.
As Luke says:
“When I plan a customer business review, I hope to have a very clear, factual view of the $, or millions of $, we’ve helped them generate using Optimizely. If it doesn’t make dollars, it doesn’t make sense.”
ROI is most explicit when displayed in revenue for, say, a Retail or E-commerce site. But this information is not always easy to gather, especially when dealing with SaaS or media businesses. How do you measure the actual value of improving lead conversion on an optimized sign-up form, or additional clicks on a media module?
Optimizely’s team is considering adding such functionality and reporting into their product, or integrating with technologies like Moat (ad viewability) to convey ROI better.
Beyond this #1 metric, Optimizely measures customer satisfaction (metric #2) using NPS (Net Promoter Score) surveys. They run regular NPS surveys at brand level (4x a year), at the end of the onboarding phase (8-10 weeks), and after each interaction with the support team. The NPS results give an idea of the service quality, the perceived value, and a proxy for customer loyalty.
Finally, the #3 metric is revenue. Luke and his team are incentivized on the net retention rate of their portfolio. They’ve actually manage to have a negative net churn rate (reminder: net churn formula = gross churn – expansion + contraction), around (.5%), in recent months. One simple metric used for almost the entire CSM team.
Luke is currently experimenting with two additional functions on the CSM team:
Launch Manager: a new role dedicated exclusively to the enterprise onboarding process (measured by NPS and volume)
Mid-Market CSM: higher volume, lower touch account management approach (measured by renewal rates & net retention)
As Luke shared with me:
“I feel lucky to work with the Sales team we have at Optimizely. They are some of the most empathetic and intelligent folks I’ve ever worked with, and they put the customer’s needs and goals first and foremost to ensure a proper fit. Sales and Customer Success have crafted a strong partnership which is imperative to achieve best-in-class net retention.”
Beyond the performance of the Customer Success team, the complementary process to ensure a negative churn rate is a sales validation process implemented by Optimizely’s VP of Sales, Travis Bryant: whenever a sales rep identifies a new account, he or she is required to populate a 45-question validation form before closing the deal. Simple, straightforward, objective, Yes/No questions designed to determine whether the new prospect is actually a good fit for Optimizely (Steli Efti, from Close.io, shares a similar philosophy although he uses different method).
According to Luke, the validation form by itself isn’t what guarantees the quality of new customers, but it sets the baseline for having a culture of customer success and retention inside the Sales team. It implies a shared agreement between the Sales and Customer Success teams that each customer is a good fit for Optimizely.
In addition, Luke personally screens every new customer and gives a special importance to validating their needs. According to Luke, with this process, truly “bad” enterprise customers are extremely rare.
It’s common practice in businesses to build retention metrics into Sales people’s incentives to avoid chasing customers without a longer term view. Optimizely’s approach is interesting in the sense that it clearly incentivizes Sales people on revenue generation, but ensures coordination with the Success team—which owns net retention—using a validation process, along with tight collaboration.
In the spirit of transparency, they entire Optimizely organization receives an email alert whenever an enterprise customer decides to churn.
“This simple workflow ensures that all employees—from the intern to the C-level—are in the loop when we fall short for a customer, and it often sparks internal dialogue about priorities and opportunities to change, iterate and refine.”
Believe it or not, a major part of the customer’s ultimate success does not rely on your customer success team, your sales team, or your product, but rather: in your customer’s organization. We came to this conclusion as Luke was sharing his best and worst customer success experiences.
Worst? The biggest challenge is when the customer lacks the skills or people to execute: generate ideas, setup experiments, QA, measure, rinse & repeat. According to Luke, the main reason for “customer failure” with Optimizely is a disconnect between a buying decision made by a senior executive and the actual resources available in the team to use it and get value out of Optimizely’s software. The help close this gap, Optimizely has curated a network of 80Solutions Partners to help their customer build—or accelerate— their optimization program.
Another challenge: earning executive mindshare at the VP and C-suite level. In a recent survey of 500 CMOs, optimization ranked #12 out of 17 various marketing priorities.
“We are crafting our sales and account management strategy to uplevel the conversation and earn executive sponsorship. This strategy, along with the coming product releases (e.g. personalization) will ultimately make Optimizely unturnoffable.”
Best? Luke’s most memorable customer success happened when one of the Customer Success team members managed to convince a client to make a hire in order to lead and improve optimization initiatives, after demonstrating that the first tests generated a 15% increase in revenue. Optimizely provided the data to help the customer’s Marketing Director make the case for net new headcount, thereby creating transformational change in the company. “In my opinion, it was one of our proudest moments,” Luke said.
Worst scenario, best scenario: both are related to the customers’ resource allocation, and that’s one key lesson for all SaaS out there: the ROI your customers derive from your software is correlated to the resources devoted to it. Convince your customers to organize for success!
http://blog.aircall.io/customer-success-optimizely/
