Back in June, I wrote this ditty about email open rates:

“I care less and less about open rates the more I realize and learn – from computer scientist types over the years – how inaccurate they are, unless something has changed I am unaware of.”

To which “Email Players” subscriber Fabien Delorme chimed in:

“As a computer scientist I can confirm this won’t ever change…It’s the same with websites and google analytics btw. There’s no way around it. That’s the way the internet is built.”

More:

My pal Jim Yaghi (another computer scientist) says Android phones have HTML turned of by default, meaning they aren’t tracking opens anyway. And he used to joke about how online marketers claiming to “scientifically” test emails have clearly never been introduced to the scientific method, with no clue about the rigorous discipline it takes to pull a real test off.

Engineer Sanjay Pande once broke it down like this:

As you know I’m the geek who has designed and built many of these so called tests, I can tell you your scientist friend is 100% correct.

There are way too many variables in e-mails.

1. E-mail volume is relatively small. The larger the volume the margin for error goes up anyway.

2. Split testing subject lines is useless because you may have different delivery or open rates.

3. It’s hard (if not impossible) to tie an e-mail to sales unless the offer is in the e-mail. Even then you don’t know what “really” caused the sale. Sometimes it’s the sequence. Sometimes it’s people’s mood. There are a number of causal effects before the sale. The prior e-mail could have been more of an influencer of the sale.

4. A price change (even an increase) can put your sales up or down. The e-mail in this case was not the cause of the sale and all the split tests in the world wouldn’t work (Ted Nicholas book flopped at 20 bucks and was a best-seller at 70 bucks to the same lists).

5. Even the average marketer knows that 80% of sales are made after the 5th contact. So, what’s the real point of your split test on an e-mail with an offer? This is a problem that even plagues direct mail marketers.

6. Most tests are done in smaller numbers with the premise that rolling up will replicate the results. This is flawed from a scientific perspective again. The sample size changed dramatically which will affect the results and you’ll never know why you had such a big hit or a flop – even though it’s helpful to have indicators to go on.

All these folks who spout their expertise on testing should really talk to a few scientific people (and perhaps geeks) on how tests are done and how they still mean squat.

People who think they’re marketers are the worst offenders followed by the folks who call themselves “real” business people – very few who even understand how these things even work.

Tests will only give you “indicators” and as you said, you really do not know when the same e-mail will work or bomb when re-used (with your evidence).

Thanks for covering this topic. People really should wake up and get it.

Another kicker:

Was when my pal Jon McCulloch showed me how Gmail now does something to grab images once and isolate them from the server hosting them — making open rate measuring completely out of whack.

I don’t know if this is still the case or not with Gmail.

But, it would not be surprising at all if it was.

Anyway, if you want to worship the open rate fairy and call it science that is your business.

I won’t say tracking opens is 100% useless.

But, I will say it is 99% overrated. What matters is ROI, not some undependable stat with as much relevance to sales as your last Space Invaders score.

All of which is why I care about open rate data about as far as I can throw a piano.

Bottom line:

I focus on tracking sales trends over time, expanding my world of offers, building my list with higher quality names, keeping my fingers on the beating pulse of my list, and continually making myself better today than I was yesterday at the things I can control (writing better emails and subject lines, curating my list, thinking up attractive offers, etc) vs focusing on things I cannot control (soft metrics like opens).

All of which requires no HTML.

No tracking software or analytics.

And, no having to go blind staring at percentages.

Whatever the case, “Email Players” newsletter subscription info is here:

www.EmailPlayers.com

Ben Settle

P.S. My all-time favorite open rate fairy story:

Was when one of my “Email Players” subscribers gave me a testimonial about how using my non-caring about opens ways nabbed his client the most sales in a particular month than ever before…but the client was still worried why his open rate was only 9%.

The irony wrote itself…