{"id":5963,"date":"2013-06-21T06:00:00","date_gmt":"2013-06-21T13:00:00","guid":{"rendered":"http:\/\/bensettle.com\/blog\/?p=5963"},"modified":"2013-06-21T06:00:00","modified_gmt":"2013-06-21T13:00:00","slug":"the-scientific-case-against-testing-emails","status":"publish","type":"post","link":"https:\/\/bensettle.com\/blog\/the-scientific-case-against-testing-emails\/","title":{"rendered":"The (Scientific!) Case Against Testing Emails"},"content":{"rendered":"

One of my subscribers recently said she thought my daily emails are like “a giant machete slashing and hacking a safe path for her through the jungle of lies, deceit and outright bullshit<\/em> surrounding the Internet.”<\/p>\n

True dat.<\/p>\n

I take great pleasure in tipping sacred cows.<\/p>\n

One biggie is this whole idea of testing emails.<\/p>\n

Whenever someone says they “scientifically test” emails, I call BS on it right away.<\/p>\n

They may THINK they’re testing emails.<\/p>\n

They may be using software designed for testing emails.<\/p>\n

They may even think they’re getting sooper important intel from these so-called “tests.”<\/p>\n

Sorry, Chuckles.<\/p>\n

It just don’t work that way.<\/p>\n

Ain’t nobody really testing anything.<\/p>\n

At least, not scientifically.<\/p>\n

My friend Jim Yaghi (an actual scientist — he doesn’t just play one on goo-roo TV) explained it to me like this:<\/p>\n

Most people have no clue how testing works.<\/strong><\/p>\n

Or of the rigorous discipline it takes to pull a real test off.<\/p>\n

A real (scientific) test is done multiple times, in a controlled environment, where there are NO changing variables. In other words… you’d have to mail the EXACT same emails, delivered to the EXACT same list, at the EXACT same day\/time, to the EXACT same device they check (i.e. phone, computer, tablet, etc) and make sure it was delivered to the EXACT same people during each test. And, you’d have to get the EXACT same results each time. Plus… everything else that could possibly affect the open rate aside from the variable of its subject has to be controlled and kept the same over all tests, too.<\/p>\n

(That is, if you want your test results to be genuinely reliable.)<\/p>\n

That’s a scientific test.<\/p>\n

And it’s impossible with email.<\/p>\n

After all, there will always be subscribers unsubscribing during the test.<\/p>\n

New subscribers joining.<\/p>\n

ISP’s blacklisting your email service.<\/p>\n

Servers down.<\/p>\n

And, people who simply didn’t check their email more than once during the test, or checked their email on a different device as they did the first time. (Not to mention if you do email right, many people will make the decision to buy 1, 2 even 3 weeks earlier, but just happened to have the $$ or be ready today — so was it today’s email or that shnazzy email you wrote 3 weeks ago… or the 10-15 accumulating emails leading up to today’s email that really made the sale? You just don’t know…)<\/p>\n

More:<\/p>\n

You think you’re REALLY tracking open rates?<\/strong><\/p>\n

Not if your readers are checking their emails on phones, you’re not.<\/p>\n

Tracking on phones is UNRELIABLE.<\/p>\n

Especially when the email client only receives text emails.<\/p>\n

Why?<\/p>\n

Because tracking opens requires HTML and that the email client automatically loads images, which most (even pc ones) don’t for security sake.<\/p>\n

And what about clickthrus?<\/p>\n

What about them?<\/p>\n

They’re more useful than opens.<\/p>\n

But you’re a durned fool if you think high clickthrus equal high sales.<\/p>\n

Even spammers get high clickthrus, but little sales.<\/p>\n

Plus, if you’re tracking sales (the only thing that matters at the end of the day) and you’re using a cookie-based tracking mechanism, you’re missing a lot of data. We saw this last year several times by running solo email ads. Sales were crazy high (over 50+ more sales than usual over a week), but according to the stats we only had 11 or so sales.<\/p>\n

Why?<\/p>\n

People clicking with one device and buying with another.<\/p>\n

Or, maybe opting in with one email address, and buying with another.<\/p>\n

(Or both…)<\/p>\n

Anyway, the point?<\/p>\n

Email is not “scientific.”<\/p>\n

Never has been.<\/p>\n

Never will be.<\/p>\n

The real power is in repetition.<\/p>\n

In consistently sending them every day.<\/p>\n

And in having the right kind of conversation with your market.<\/p>\n

(All of which I teach in “Email Players” each month.)<\/p>\n

Don’t let the goo-roos intimidate you.<\/p>\n

Don’t let them lie to you about their “testing stats” which, even if true, are 100% irrelevant to you, your market and your product.<\/p>\n

And, don’t let them paralyze you with a testing complex.<\/p>\n

Focus on writing emails regularly.<\/p>\n

On doing them right (so people look forward to getting them).<\/p>\n

And on tracking sales trends over time — not having a cow over opens or clickthrus on any given day (which, btw, will often be LOWER if you have a smaller list and do email right anyway, but that’s another discussion for another time…)<\/p>\n

The rest will take care of itself.<\/p>\n

Alright.<\/p>\n

I’m gonna sheathe my blood-stained machete.<\/p>\n

(For now…)<\/p>\n

To get started with email, go here:<\/p>\n

www.EmailPlayers.com<\/strong><\/a><\/p>\n

Ben Settle<\/p>\n","protected":false},"excerpt":{"rendered":"

One of my subscribers recently said she thought my daily emails are like “a giant machete slashing and hacking a safe path for her through the jungle of lies, deceit and outright bullshit surrounding the Internet.” True dat. I take great pleasure in tipping sacred cows. One biggie is this whole idea of testing emails. […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[9],"tags":[],"class_list":{"0":"post-5963","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-email-marketing"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/posts\/5963","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/comments?post=5963"}],"version-history":[{"count":0,"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/posts\/5963\/revisions"}],"wp:attachment":[{"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/media?parent=5963"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/categories?post=5963"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bensettle.com\/blog\/wp-json\/wp\/v2\/tags?post=5963"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}