Most Sites Make This Costly Conversion Testing Mistake

You’ve done your research.

You’ve surveyed your customers, poured over your web statistics, searched for optimization opportunities on your website and generated a hypothesis.

You have a theory: if you change the angle of your pricing page to be more customer focused you can increase the conversion rate of that page.

After all of the research, coding and designing involved to get you here, it’s finally time to launch your test.

And away it goes….

A few days later it’s time to take a look at the results. There MUST be a decent increase in conversions, right?

Wrong.

You have a measly 3% increase. And the trouble with small bumps in conversion rates is that they take a really LOOOOONG time to reach statistical significance.

You might even witness your test go from 3% to -3% and back again in the same week.

So what’s the deal?

You could be diluting your test results.

I’ve been working with a site that has 3 different landing pages that promote the same offer: Getting a pricing quote.

In Google analytics the conversions are set up as goals. When someone hits the thank you page after filling out the form, they are tracked as a conversion. That thank you page is used for all three landing pages that collect leads.

That’s the problem.

If I were to run a test to optimize one of the quote landing pages, Google Analytics Experiments will dilute the test results because it’s tracking the conversions from all 3 sources instead of just one.

conversion-rate-3-landing-pages

When I run a test on Landing Page #1, I will see a change in conversions, but it will be diluted.

conversion-rate-3-landing-pages-2

This may look like a great test (it boosted conversion rates by 21.4% after all), but it isn’t the whole story.

In reality, this is how the test should be tracked:

conversion-rate-3-landing-pages-3

See the difference? Your test actually yielded an increase of 150%!

This is a mistake that most websites make when running multiple landing pages to the same offer. And if you track it properly you’re able to:

  • Find a winner faster
  • Save the extra time and money it costs to run a longer test
  • Record a much bigger win – which will help you to convince your boss that this whole CRO thing is worth while

Here’s a real world test

To show you the difference, I recently ran a test on a sidebar form of that website. The test wasn’t extensive, it was just testing the word “Quote” vs. “Pricing”.

If you track the test the way most sites do, the test was inconclusive.

But if I track only the conversions from the sidebar form, I see a 12.7% lift in conversions.

sidebar-test

If I had set up the test incorrectly, like most sites do, I wouldn’t have seen a winner and this test most likely would have hit the wood pile.

But in this case, I’ve found a winner and I get to enjoy almost a 13% boost in conversions.

But how do you fix this problem?

Let’s talk about how you can track conversions correctly using Google Analytics goals.

Without making a single change to the code of your thank you page, you can track your conversions correctly by landing page.

The easiest way to accomplish this is to change the URL of the thank you page for each lead form that you have on your website.

You could do it this way:

www.yourwebsite.com/thankyou
www.yourwebsite.com/thankyou2
www.yourwebsite.com/thankyou3

But then you have to manage multiple pages with the same content on them. If you make changes to your thank you page, you have to make the same change on the other two thank you pages.

Instead, you could just add a URL variable:

www.yourwebsite.com/thankyou?id=1
www.yourwebsite.com/thankyou?id=2
www.yourwebsite.com/thankyou?id=3

You get the same result (multiple URLS for your thank you pages) but you still have a single page to manage.

The beautiful part of this strategy is that you don’t even need to add any code to your thank you page. It’s enough that the URLs look different for Google Analytics.

What does this really mean?

If you’re running tests and you don’t see the results that you thought you would get, it might not be the fault of your test. It may be just that your tracking isn’t setup properly.

Always make sure that your tests are technologically sound before launching them so that you get your results as quickly as possible. Diluting test results could delay your test results by weeks or even months which could cost your company thousands of missed conversions.

Let me know if you have any questions in the comments below.

~E

Share on FacebookTweet about this on TwitterShare on Google+

5 Responses to “Most Sites Make This Costly Conversion Testing Mistake”

  1. guru

    Eric,
    your figures make this complicated subject so lucid
    but ?id=1 is not considered duplicated content. Do you lbel the landing pages similarly?

    Reply

Leave a Reply to guru

Click here to cancel reply.

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>