Yesterday I told you about Jim Yaghi.<\/p>\n
The computer scientist and lead generation master.<\/p>\n
Remember that?<\/p>\n
Well, today, with his permission, I’m reprinting an email he wrote about why split testing is (GASP!) a waste of time.<\/p>\n
Yes, I know this goes against the grain.<\/p>\n
But, most truth does.<\/p>\n
In fact, I’ve been doing what he says (below) and my opt-in conversions are overall higher.<\/p>\n
Definitely worth seeing if you get the same results.<\/p>\n
So, without further ado…<\/p>\n
Take it away, Jim:<\/p>\n
Last night, I went to bed with the SPLITTING-EST of headaches.<\/p>\n
Was gonna pass out; even after 3 pills of the strongest kind, and\u00a0Voltarin cream on neck and head…the pain got the better of me.<\/p>\n
Lack of sleep is the culprit – but like I always say – plenty of\u00a0time to rest when I’m dead.<\/p>\n
Speaking of SPLITTING…<\/p>\n
How about what I said a while ago?<\/p>\n
“Ben and I both agree that split-testing is a waste of time”<\/p>\n
I knew someone would challenge me on it…and it was an Engineer\u00a0client of ours who ended up asking, “What do you mean it’s a waste\u00a0of time?”<\/p>\n
Oddly enough, I was on a call with Ben Settle at the exact moment\u00a0the email came in, discussing that exact topic.<\/p>\n
When I told Ben YEARS ago that split-testing is stupid…he was\u00a0shocked!<\/p>\n
“Jim,” he said. “You are a SCIENTIST! Don’t you even have a Math\u00a0degree or something? I thought you’d be the ONE guy who’d\u00a0RELIGIOUSLY split-test everything.”<\/p>\n
Yes I do have a math degree somewhere. But Nope, I split-test\u00a0NOTHING.<\/p>\n
Skeptical as Ben was, when I told him split-testing was stupid, he\u00a0figured I knew what I was talking about and gave MY method a try.<\/p>\n
Today, he and I laugh (all the way to the bank) at people who\u00a0split-test.<\/p>\n
Split-testing is an attempt to make SCIENCE out of\u00a0marketing…which, let’s be honest, marketing is more a TRADE than\u00a0a science. Or if you’re me – Handsome and Awesome – then it’s more\u00a0of an ART than science.<\/p>\n
Engineers and people from science backgrounds, they tend to enjoy\u00a0the idea that marketing could somehow become scientific and\u00a0predictable…although it NEVER really is.<\/p>\n
It’s virtually random. Without order. It’s all grounded in PEOPLE’s\u00a0EMOTIONS – no logical thing can describe or model it.<\/p>\n
– Does this red button work better than this green button?<\/p>\n
– Does this headline perform better than this other?<\/p>\n
– Does this landing page convert higher than the old one?<\/p>\n
All good questions.<\/p>\n
But running the split-test itself could be DANGEROUS.<\/p>\n
Not only do you risk killing a money-making version of your ad…<\/p>\n
You also risk keeping the LOSER version instead of the WINNER\u00a0versions over time, which will progressively DROP your sales rate.<\/p>\n
It’s true.<\/p>\n
Look, the whole concept of split-testing relies on statistics,\u00a0which is the science of APPROXIMATING the FUTURE from OLD DATA.<\/p>\n
All statistical tests start on the premise that a sample should on\u00a0some level represent the population.<\/p>\n
Meaning, if you run a statistical experiment in marketing, you’re\u00a0doing it with the belief that the 1,000 or 10,000 people who see\u00a0your sales message, behave like, and are a “good-enough”\u00a0representation, of your ENTIRE market.<\/p>\n
You certainly don’t show your ads to the ENTIRE market during a\u00a0split-test.<\/p>\n
And the ENTIRE market does not behave the exact same way as the\u00a01,000 people you test.<\/p>\n
And 10,000 people don’t behave exactly the same as 1,000 people in\u00a0a test.<\/p>\n
We know this much.<\/p>\n
Sure there could be patterns in each group’s behavior, but it\u00a0differs from one group to the next.<\/p>\n
And theoretically, the bigger the group, the closer you get to\u00a0describing the entire population of your market.<\/p>\n
So split-tests may be MORE meaningful when you’re dealing with a\u00a0sample size in the order of 10,000 – 100,000 hits\/day…but that’s\u00a0rarely when marketers are using split-tests.<\/p>\n
One thing we can be certain of, is that SOME of your market will\u00a0respond to Version 1 of your ad while others will respond to\u00a0Version 2 of your ad.<\/p>\n
And those who respond to Version 1 may not ever respond to Version\u00a02 and vice versa.<\/p>\n
This in itself is a good enough reason to NOT split-test.<\/p>\n
Add don’t forget, that when REAL scientists run statistical\u00a0experiments, they attempt to create a COMPLETELY controlled\u00a0environment with as close to ZERO bias one way or the other.<\/p>\n
Something you can ONLY pull off in a lab, with lab-rats, which we\u00a0don’t have the luxury of online.<\/p>\n
And they use tests like Chi-Square, Kendall’s, Pearson’s, or\u00a0Spearman’s correlation to understand if the results of one\u00a0variation against another might somehow be due to PURE chance.<\/p>\n
(Meaning, they ALSO test HOW WRONG THEY MIGHT BE in reaching their\u00a0conclusion!)<\/p>\n
It’s so complicated; it’s enough to give YOU and ME both a\u00a0splitting headache.<\/p>\n
Let’s put it this way:<\/p>\n
Split-testing is a dangerous tool in untrained hands.<\/p>\n
Plus there’s a much simpler alternative that works better.<\/p>\n
Whether your ad is Version 1 or Version 2, or V.3 or V.10, all you\u00a0REALLY care about is that your ad is making money (or generating\u00a0leads).<\/p>\n
So if you run two versions and BOTH succeed in doing that (eg v1\u00a0gets 10% optin and v2 gets 30% optin), DON’T kill v1!<\/p>\n
If you kill v1 because it scored worse than v2, you’re assuming\u00a0that ONE ad is going to be responded to by 100% of your market.<\/p>\n
That’s not true is it?<\/p>\n
V1 got 10% of your market to respond. So keep it. Unless you want\u00a0to turn away 10% of the population.<\/p>\n
Instead, create a v3 based on v1’s success and run all 3 versions\u00a0of your ad.<\/p>\n
This way, you INCREASE your odds of getting an even LARGER share of\u00a0your market to respond to your ads…WITHOUT turning away anyone\u00a0else who behaves like those who responded to v1 or v2!<\/p>\n
Hey, now you don’t suppose that’s the ONLY marketing trick I know,\u00a0do you?<\/p>\n
See, the point here is that I know GANGS of marketing tricks that\u00a0make life easier, less complicated, and make much more money.<\/p>\n
You can read about them here:<\/p>\n