Lyris’ Spam False Positive Statistics are Misleading

We note that Lyris, a service provider to direct marketers, claims that Gmail's spam filters cause 3% false positives and they used to cause 44% earlier this year. Hard to believe, but of course when one reads such worrying statistics, one should always consider methodology and motives.

From our research, it's highly unlikely that a real Gmail user is seeing that kind of false positive percentage. There are, of course, several ways to measure false positive proportions -- depending on whether one prefers to publish a tiny number or a big, scary number.

Our estimate of Gmail's false positive performance is about 0.01% to 0.02%. That's based on roughly one per week, and measured as a proportion of total inbound email, including spam.

Reading between the lines of Lyris' report, it's measuring false positives only as a proportion of inbound marketing email, which might explain why the headline figures are so high. It seems Mark Twain was right about statistics (although it may not have been Twain who first coined the phrase).

These inflated numbers cast doubt on the rest of the statistics presented in this report. Lyris clearly has an agenda here -- to concentrate the minds of direct marketers so that they'll sign up for Lyris' services.

... Richi Jennings

One Comment

  1. Carrie
    Posted April 29, 2009 at 9:51 PM | Permalink

    as soon as I unsubscribed from lyris.Philly.com, I started recieving more spam in my inbox. (gmail). prior to that, spam in my inbox was nonexistent.

    what do you say about that?

  2. Posted April 29, 2009 at 10:29 PM | Permalink

    Carrie, without more information about the spam you’re receiving, it’s impossible to say.

    However, you should note that Gmail has recently suffered an unusual spike in false negatives (i.e, spam in the inbox), so I’d suspect that the two events are unrelated.

    I find the best way to spot spammers misusing the email addresses of legitimate signups is to use a unique alias for each sender. Gmail users can do this using the ‘+’ sign (e.g., richij+frist@gmail.com )

One Trackback

  1. By IT Blogwatch on October 9, 2006 at 4:22 AM

    VCs: bereft of life, they RIP? (and handheld inkje

    I wish to register a complaint! It’s IT Blogwatch, in which we find out if venture capital is dead. Not to mention the handheld inkjet that prints onto anything…

Post a comment

You must be logged in to post a comment. To comment, first join our community.