7/17/13

My Very First Experiment: How Changing One Sentence Led to a 42% Increase in Revenue


It seems like forever ago, but I remember it like it was yesterday.  It was a sunny February day in Miami, Florida about four years ago.  I had just settled into to the Intercontinental hotel getting ready for the MarketingSherpa Email Optimization Summit.  My reasons for attending was to brush up on some skills, and maybe walk away with a few new email tactics that I could apply to my nonprofit clients.  After all, I was the expert, right?  I was sure that everything I was going to see I had seen before, or read about online, so my expectation was that this would be a good refresher course.  Boy, was I wrong.

Within the first five minutes, I had my marketing intuition challenged when Dr. Flint McGlaughlin put up a slide on the screen with the following three emails.  Each one was a different version of the same content.  Each was designed by a top-tier NYC agency.  The audience was asked to look at each one and vote for the email that was the most optimized, that is, the email that would produce the highest response. 
  



It might be fun for you to do the same thing.  Look at each one and pick out the email that you think will perform best.

Now consider the following:

Why did you pick the version you picked?
What made you believe that this one would do better than the others?
Why didn’t you pick the other ones?
What is the criterion that a marketer can look at and know that it is optimized?

Now, here is the results of a test of each of the three treatments.  



As you can see, each one performed worse than the control.  Now, the control was never showed to the audience-- but it didn’t matter-- the point was that when it comes optimizing emails, marketing intuition is not enough.  Testing is what trumps marketing intuition.  And when you test, you need to use a rigorous methodology.

After that, I was sold.  At the first break, I called my team and told them that “starting Monday, we will change the way we do everything for our clients…actually, no-- starting now.”

At the time, I was consulting with the George W. Bush Presidential Center, and my team and I were doing all of the online donor acquisition work.  The very next day, we were sending an email to a few rented lists in an effort to acquire new donors.  This seemed like the perfect opportunity to test some of this new “rigorous testing methodology” that I was learning, so right there--from the floor of the conference--I phoned-in our very first test.  Below are the screenshots of both the treatment and the control:



Can you see the difference?  It is very subtle.  In fact, the only difference appears in only one sentence-- the very last sentence of the email.

Now, best practices would tell me that only 18% of people that actually open and read emails will read all the way to the bottom, so how could making a change like this make a meaningful difference?

Even my client raised this concern.  In fact, he was very reluctant to do the test, but eventually did acquiesce reminding me that I was the one that would be held accountable for results.  Great.  I love pressure.  But I really believed in the test and was willing to bet the farm on this one.

So, we ran the test.  And check out the results-- the new treatment, the one with revised sentence generated a 139% increase in the Clickthrough rate and a 42% increase in revenue.

Here’s what you need to understand

The last sentence contained the call-to-action of the email.  You can see an enlarged version of both the control and the optimized treatment below.




  • Key Principle #1 - You must make them click.  The call-to-action is perhaps the most critical element of any email, and interestingly enough, the action that you want the recipient to take is always the same-- you want them to CLICK!  One mistake that many email marketers make is that they often give the email recipient too much information.  And the recipient responds by saying no, and not clicking through to the landing page.  But the landing page is where the conversion takes place-- not the email.  So the goal of the email must first and foremost, inspire the recipient to click, not give, or buy, or register-- or anything else-- just take the next step by clicking.



  • Key Principle #2 - Your offer must be desirable to the recipient.  Look again at the control copy and the optimized copy.  What conveys more value to you-- “…making a tax-deductible online contribution now” or “…[becoming] a Charter Member of the George W. Bush Presidential Center.”?  Consider the exclusivity of the two offers: can I make a tax-deductible online contribution anywhere else?  How about joining-- as a Charter Member-- a Presidential Center that bears George W. Bush’s name?  Can I do that anywhere else?  When we look at the offers in this light, it becomes very clear what is the most desirable offer.



  • Key Principle #3 - Increasing traffic to your landing page usually means more conversions.  Here is what is interesting about this experiment.  We had 139% increase in Clickthrough from the optimized treatment.  That means we had 139% more people visiting the landing page where they actually could make a gift.  But, when we looked at our landing page metrics, the conversion rate (the percentage at which visitors donated) was much lower, and yet we still increased donations by 42%.  So how can this be?  What I learned is that by focusing my email on selling a click-- not a donation-- I can get a ton more people to at least go that next step with me to learn more on the landing page.  Because I get so many more people to the landing page, some of the people that would have never even considered giving a gift have the opportunity to change their mind (with a strong value proposition presented on the landing page) and donate.  These are people that if had received the control version with the call-to-action to donate, would have probably said no and not clicked.


Now, it’s your turn

Now it’s time to apply what you just learned.  Look at one of your emails.  Examine the call-to-action.  Ask yourself, “Is there an offer?  Is it desirable?  Am I giving people too much information in the email that can be off-loaded to the landing page thus creating a compelling reason to click?”  Most email service providers can do simple A/B split tests, so why not create a little experiment.  Break up your email file into two segments and send one segment version A (the Control) and send the other version b (the Treatment).  Then watch and see what your subscribers tell you by the way they respond.  I’d love to hear about it, so shoot me a note and let me know what you learned.