My Very First Experiment: How Changing One Sentence Led to a 42% Increase in Revenue

It seems like forever ago, but I remember it like it was yesterday.  It was a sunny February day in Miami, Florida about four years ago.  I had just settled into to the Intercontinental hotel getting ready for the MarketingSherpa Email Optimization Summit.  My reasons for attending was to brush up on some skills, and maybe walk away with a few new email tactics that I could apply to my nonprofit clients.  After all, I was the expert, right?  I was sure that everything I was going to see I had seen before, or read about online, so my expectation was that this would be a good refresher course.  Boy, was I wrong.

Within the first five minutes, I had my marketing intuition challenged when Dr. Flint McGlaughlin put up a slide on the screen with the following three emails.  Each one was a different version of the same content.  Each was designed by a top-tier NYC agency.  The audience was asked to look at each one and vote for the email that was the most optimized, that is, the email that would produce the highest response. 

It might be fun for you to do the same thing.  Look at each one and pick out the email that you think will perform best.

Now consider the following:

Why did you pick the version you picked?
What made you believe that this one would do better than the others?
Why didn’t you pick the other ones?
What is the criterion that a marketer can look at and know that it is optimized?

Now, here is the results of a test of each of the three treatments.  

As you can see, each one performed worse than the control.  Now, the control was never showed to the audience-- but it didn’t matter-- the point was that when it comes optimizing emails, marketing intuition is not enough.  Testing is what trumps marketing intuition.  And when you test, you need to use a rigorous methodology.

After that, I was sold.  At the first break, I called my team and told them that “starting Monday, we will change the way we do everything for our clients…actually, no-- starting now.”

At the time, I was consulting with the George W. Bush Presidential Center, and my team and I were doing all of the online donor acquisition work.  The very next day, we were sending an email to a few rented lists in an effort to acquire new donors.  This seemed like the perfect opportunity to test some of this new “rigorous testing methodology” that I was learning, so right there--from the floor of the conference--I phoned-in our very first test.  Below are the screenshots of both the treatment and the control:

Can you see the difference?  It is very subtle.  In fact, the only difference appears in only one sentence-- the very last sentence of the email.

Now, best practices would tell me that only 18% of people that actually open and read emails will read all the way to the bottom, so how could making a change like this make a meaningful difference?

Even my client raised this concern.  In fact, he was very reluctant to do the test, but eventually did acquiesce reminding me that I was the one that would be held accountable for results.  Great.  I love pressure.  But I really believed in the test and was willing to bet the farm on this one.

So, we ran the test.  And check out the results-- the new treatment, the one with revised sentence generated a 139% increase in the Clickthrough rate and a 42% increase in revenue.

Here’s what you need to understand

The last sentence contained the call-to-action of the email.  You can see an enlarged version of both the control and the optimized treatment below.

  • Key Principle #1 - You must make them click.  The call-to-action is perhaps the most critical element of any email, and interestingly enough, the action that you want the recipient to take is always the same-- you want them to CLICK!  One mistake that many email marketers make is that they often give the email recipient too much information.  And the recipient responds by saying no, and not clicking through to the landing page.  But the landing page is where the conversion takes place-- not the email.  So the goal of the email must first and foremost, inspire the recipient to click, not give, or buy, or register-- or anything else-- just take the next step by clicking.

  • Key Principle #2 - Your offer must be desirable to the recipient.  Look again at the control copy and the optimized copy.  What conveys more value to you-- “…making a tax-deductible online contribution now” or “…[becoming] a Charter Member of the George W. Bush Presidential Center.”?  Consider the exclusivity of the two offers: can I make a tax-deductible online contribution anywhere else?  How about joining-- as a Charter Member-- a Presidential Center that bears George W. Bush’s name?  Can I do that anywhere else?  When we look at the offers in this light, it becomes very clear what is the most desirable offer.

  • Key Principle #3 - Increasing traffic to your landing page usually means more conversions.  Here is what is interesting about this experiment.  We had 139% increase in Clickthrough from the optimized treatment.  That means we had 139% more people visiting the landing page where they actually could make a gift.  But, when we looked at our landing page metrics, the conversion rate (the percentage at which visitors donated) was much lower, and yet we still increased donations by 42%.  So how can this be?  What I learned is that by focusing my email on selling a click-- not a donation-- I can get a ton more people to at least go that next step with me to learn more on the landing page.  Because I get so many more people to the landing page, some of the people that would have never even considered giving a gift have the opportunity to change their mind (with a strong value proposition presented on the landing page) and donate.  These are people that if had received the control version with the call-to-action to donate, would have probably said no and not clicked.

Now, it’s your turn

Now it’s time to apply what you just learned.  Look at one of your emails.  Examine the call-to-action.  Ask yourself, “Is there an offer?  Is it desirable?  Am I giving people too much information in the email that can be off-loaded to the landing page thus creating a compelling reason to click?”  Most email service providers can do simple A/B split tests, so why not create a little experiment.  Break up your email file into two segments and send one segment version A (the Control) and send the other version b (the Treatment).  Then watch and see what your subscribers tell you by the way they respond.  I’d love to hear about it, so shoot me a note and let me know what you learned.


"Testing Trumps Best Practices"

That's a lesson I keep learning over and over again.

This week, I'm getting geared up for the MECLABS Landing Page Optimization Summit in Boston.  Last year, I had the opportunity to speak on the big stage with my friend and colleague, Nat Ward.  In our presentation, we shared some of the learnings from our experiments.  This is a short clip from our presentation where we show how we increased our donation conversion rate by 25% by putting multiple calls-to-action on our landing page.


I Live Left of LOCO

Those of you that know me know that this statement has nothing to do with my political orientation.  LOCO stands for the Law Of Channel Orthodoxy.  Well, it's a theory actually.  A theory I made up.  But I call it a law because it makes a better acronym and because people take laws much more seriously.

Anyway, the Law of Channel Orthodoxy goes like this:

The time and expense necessary to delivery a fundraising message (Resources) is positively correlated to the potential success of that message (Response Rate).

For example, let's consider face-to-face fundraising.  It is most definitely the most time-intensive, and expensive fundraising method.  But it is often the most successful.  If I were to fly around the country and ask each of my donors to give a gift, then I could expect a response rate of 40 to 60 percent or more (note: this is based on 18,000 face-to-face visits that was conducted over a 10 year period which maintained a 45% response rate).

Then, let's hop down to direct mail or telefunding.  Is it as expensive to call or mail all of our donors?  Absolutely not!  But will I get a 45% response rate-- probably not.  Typical response rates for a successful direct mail and telefunding campaign would be between 6% and upwards of 20%.

Now, consider email.  Is it as expensive to send an email as it is a direct mail package?  No way!  But are you going to get a 20% response rate on an email appeal?  Not likely.  According to the 2012 Convio Nonprofit Online Marketing Benchmark Report, the average response rate for a house file email appeal continues to hover around 0.15%.

So, in each of these examples, it seems that the Law of Channel Orthodoxy seems to hold up.  Still with me?

Okay, so the question we must ask then is, "how do we beat the Law of Channel Orthodoxy?"

Answer?  Well, there are actually two ways:

Innovation - When we develop a way to achieve the same results while at the same time reducing the time and expense investment, we call this an innovation.  Believe me, there are many ways that we can innovate in the nonprofit space.  But the problem is that too often nonprofits feel they need to play it safe.  We go for the guarantee, instead of taking a risk on something that could either go big, or be a big bust.  But what I think many organizations fail to realize is that there is a risk associated with the status quo.  If we fail to test and try new things, we are pretty much guaranteed to become obsolete. 


But there is another way that we can beat LOCO.

Optimization - When we increase our response rates or improve results without proportionally increasing the amount of required resources, we call this optimization.  Optimization begins with the belief, as Peter Drucker once said, that "adequacy is the enemy of excellence."  It forces us to never be satisfied.  It teaches us to interpret a 10% response rate as a 90% non-response rate, and inspires us to develop better ways to communicate so that we can get those non-responders to join us.  Optimization can transform the culture of an organization by changing the conversation from "if only..." to "what if..."  Optimization is synonymous with perpetual improvement and is essential to success.

So, in order to beat the Law of Channel Orthodoxy, we need to find a way to live left of LOCO.  I'm telling you, once you get used to it, you wouldn't want to go any other way.  Here's a few questions that you can ask yourself to help break free from going completely LOCO.  It might be a good idea to ask your staff these same questions and compare your answers with theirs.


1.  My fundraising calendar is really solid because we have been doing things the same way for the past few years and have received good results.  (T/F)

2. I don't typically try new fundraising techniques because they can be risky and we owe it to our donors to only invest in sure winners. (T/F)

3. I'm very happy with our fundraising results.  I don't see how we could possibly do any better.  (T/F)

4.  Im convinced that if we just implement best practices, then we will receive optimal results. (T/F)

5. Ive been doing this a long time, and my experience and intuition is what I lean on when I evaluate a new fundraising program. (T/F)

So, How Did You Do?

If you answered true to one or more of the above statements, then you may be going LOCO!  But don’t worry, it is curable…


Game Changing: How Ministries Are Using Social Gaming to Engage New Supporters

Is it possible to use social gaming for social good?  This presentation explores the fast-growing social gaming market and presents two case studies of ministries that are using social gaming to build brand, relationships, and donors online.  The presentation concludes with a tactical 6-step planning exercise for anyone considering integrating social gaming into their online marketing and fundraising strategy.

 Download the complete 105-slide presentation here.

This presentation was delivered on Wednesday, May 1, 2013 at the National Christian Leadership Alliance Conference in Anaheim, CA

Other Related Resources

Social Game Viral Simulator - Use this Excel spreadsheet to develop an estimate of the number of participants you can expect to acquire through an interactive social game.

Game Changing Innovation Article - This article, featured in the fall 2012 edition of Outcomes Magazine, walks through the Moody Virtual Marathon case study.

Oracle Whitepaper: Using Social Gaming to Drive Engagement: Insights and Best Practices for Brand Managers - This whitepaper features some best practices and tips for integrating social gaming.  At the end are two case studies that feature major brands that use embedded philanthropy in their social games.

Social Network Games 2012 Report - This report is a good primer on social games, market trends, and terminology that you need to know if you are going to consider developing a social gaming strategy.


The Conversation is More Important than the Brand

Brand marketers suggest that brand is everything.  In order for your brand to be effective you must invest in building strong brand equity and make sure that you consistently deliver on your brand promise.  To do this, your brand must be consistently portrayed and always visible.

But what if your brand creates friction in the communication process?  Often when we think we are being marketed to we tend to put our guard up.  Even if this doesn't happen on a completely conscious-level, it happens to some degree in our subconscious. 

This was the premise of a recent marketing experiment that I conducted in partnership with The Heritage Foundation.  The main question was this: “If the essence of marketing is messaging, and the essence of messaging is a conversation, how can we remove psychological barriers to that conversation?”  Do "brands" induce feelings of anxiety?  If so, by removing the brand and focusing on facilitating a more effective conversation, can we produce a stronger response?

The Client

The Heritage Foundation is a conservative think tank in Washington, DC.  Their work focuses on formulation and promotion of conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.  They are the most broadly supported organization of their kind with nearly 700,000 financially participating members. 

The Experiment

Heritage was in the third year of a talk radio campaign with Rush Limbaugh and Sean Hannity, and was beginning to experience the law of diminishing returns.  The massive audiences for these two nationally syndicated talk show hosts were not as receptive to our Heritage-branded messages as they were initially and we considered doing something drastic and perhaps even unintuitive-- creating an unbranded campaign.  The idea was to have the radio hosts create excitement around a controversial issue and then send their audience to an unbranded web site where they could learn more.  The visitor would be taken through a guided conversation online that provided an educational experience, and then at the end, provided with an opportunity to respond by signing an open letter to Congress calling for the end of wasteful government spending.  After signing the open letter, the visitor was then taken to another page and provided an appeal to join The Heritage Foundation.

The Web Site

DirtySpendingSecrets.com is very simple web site that was designed for a very specific purpose-- to acquire new names and to inspire new donations.  The message is simple and clear-- Washington is spending too much of your money, and here are some examples of how wasteful some of that spending is.

The site then stepped the visitor through a guided conversation about spending by asking them to answer simple questions.  For example, how much money the US Government spent to train Chinese prostitutes to drink more responsibly on the job?

When the visitor provided their answer, a message appeared with the correct answer and provided additional details-- this provided the educational experience.

When the visitor got to the end of the quiz-- when they were all good and worked up-- the web site prompted them to do something about it by signing the Heritage Foundation's Open Letter to Congress to tell them no more wasteful spending.  This is where the brand is finally introduced.

The visitor is then taken to a Heritage-branded page where they can sign the open letter.  After they sign it, the visitor is prompted to make a donation.


The Results

In the first three weeks of the campaign, the web site generated 115,000 new names for the Heritage Foundation and a significant amount of revenue from new donors.  The web site also went viral with over 40,000 facebook shares, 4,100 tweets, and over 32,000 email shares.

The Key Takeaways

This experiment illuminated a couple of interesting principles:

  • The conversation is more important than the brand.  When we lead with a compelling concept, and initially divorce that concept from being connected with any sort of brand, we can inspire a great many more people to engage in that conversation.  When Rush and Sean talk about this new web site they discovered that exposes all of these government spending secrets, people take interest and seek out the web site.

  • The Call-to-Action should not be introduced until the value proposition has been experienced by the visitor.  Too often, we think too little of our potential donors.  We think if we just stick a donate button on our page, that people will just donate.  And the problem is that some actually do.  But the bigger problem is that so many more actually would if they have been properly guided through a conversation that enables them to discover the value proposition on their own.  Our goal in communication should be to lead the constituent to an "aha! moment." This is no easy task and requires tremendous creativity and strategic thinking.

  • Good ideas go viral.  A lot of people think there is an easy button for social media.  But there isn't.  If you think about things that are share-worthy, they are always things that move people.  They must shock, surprise, make you laugh, make you cry, inspire you in some way-- the point is that a good social media strategy starts by developing a good and share-worthy idea.

Now, It's Your Turn

The model and concept that is outlined here is very repeatable.  I would encourage you to perform your own experiment with the unbranded concept and share your results and learnings.  Remember a good idea that isn't implemented is a worthless idea.



Case Study: Optimizing Name Acquisition

This is an abbreviated case study that illustrates how increasing the relevance and perceived value of your offer can produce stronger response on your landing page.  Flint McGlaughlin, CEO of MECLABS pulled me up onto the stage to share this at the MECLABS Landing Page Optimization Summit in Colorado last month.

You can read the more detailed post on the Marketing Experiments Blog.


"Best Practices Are Nothing But Polled Ignorance"

I wish I could say that I was the first one to say that, but I wasn't.  Last week, I attended the MECLABS Landing Page Optimization Summit in Denver, CO and one of the first things I wrote down was that quote from my mentor Dr. Flint McGlaughlin.  Flint went on to say that the marketer must be the philosopher of an organization.  Philosophers are constantly consumed with the question of "why."  This proves to be a difficult challenge when we face the perpetual barrage of questions that begin with "how."

How can I get this done in time?
How can I make my subject line more effective?
How can I increase my revenue?

Most marketers are too busy asking the questions that begin with "how" that they fail to ever ask the more important questions that begin with "why."

Why did this perform better than that?
Why do my customers/donors prefer this type of content over that type of content?
Why doesn't my landing page convert more people into customers/donors?

But it is precisely the "why" questions that lead to wisdom.  I recently performed an experiment that challenged a longstanding direct response best practice and it reminded me that asking "why" is okay.  The best practice goes like this:

Having more than one conflicting call-to-action on a landing page leads to non-decision and hurts conversion rate.

Now, I think that this actually makes a lot of sense.  What I wanted to discover through this experiment is whether or not there could be a scenario where having multiple calls-to-action could actually lead to a greater conversion rate.  The results were quite convincing.


The Experiment

In this experiment we tested two different landing pages that each had differing conversion paths.  The control had a two column layout with a donation form in the left column and copy in the right column.  The treatment was actually exactly the same with one exception: at the bottom of the right column below the copy we added a secondary call-to-action to register your email address to receive a free PDF download.  Now, because the PDF offer required a confirmation page, we decided to add a secondary opportunity to donate with a page layout similar to the control, but with different copy.

Once the control and treatment pages were developed we performed a simple A/B split test and diverted an even 50% of traffic to both versions over the next 30 days.  Once the results were compiled and validated, the control had a donation conversion rate of 2.52%.

The interesting thing is that the treatment, the version of the landing page with two calls-to-action actually had a statistically identical donation conversion rate of 2.51%.  However, the treatment also received a 4.98% name acquisition conversion rate for people that responded to the secondary call-to-action for the free PDF offer.

What was even more interesting is that the people that responded to the PDF offer then converted at a 14.16% donation conversion rate when provided a second opportunity to give.  This means that by adding the secondary call-to-action for the PDF, and adding a secondary opportunity to give, enabled us to create a 25% lift in donation conversion.

What I Learned From This Experiment

There were actually a few different takeaways from this experiment:

  1. Donation conversion is greatly affected by visitor motivation.  This is something that Dr. Flint McGlaughlin and his team at MECLABS talks about with their Conversion Heuristic:

    C = 4m + 3v + 2(i - f) - 2a 

    This expression in plain english goes like this: Conversion equals four times visitor motivation, plus three times clarity of value proposition, plus two times incentive minus friction, minus two times anxiety.  The variable with the highest coefficient is Visitor Motivation.  What this means is that if visitor motivation is high enough, then it can often overcome inhibitors like friction and anxiety.  Clearly when we add a secondary call-to-action to a web site we are adding friction into the process. But because the traffic that was coming to the web site was motivated to give, then it really didn't matter.
  2. Reciprocity is alive in well in the donation conversion process.  Reciprocity is a psychological principle that suggests that when I give you something, in some way you feel indebted to me, and because our natural inclination is to not be in someone's debit we tend to want to settle the score by returning the favor.  When we offered a gift (free pdf) to the visitors to the site, they were more predisposed to return the favor (donating to the organization) after receiving the gift.
  3. Best practices are not enough.  If you really want to maximize your revenue by optimizing your web pages, then relying on long-held best practices is not enough-- you need to be willing to challenge best practices and even conventional wisdom  by constantly testing within a rigorous methodology.  Science trumps the marketer's intuition every day of the week.  Remember what Dr. Flint McGlaughlin says, "Best practices are nothing but polled ignorance."

Happy Experimenting!