Back in January of 2001, some of my buddies and I did some serious thinking about spam, the obnoxious unsolicited email, not the canned luncheon meat (email spam is sometimes referred to as unsolicited commercial email or UCE). For several days we sat around a table in a room paneled with whiteboard in the basement of a house in a suburb of Philadelphia. Collectively we came up with some useful and enduring insights. With spam now accounting for up to 90 percent of all Internet email traffic and new, more malevolent variations appearing weekly, I thought it might be useful to revisit some of those insights in this post......starting with this observation:
1. Some people like to receive unsolicited commercial email.
In 2001, among anti-UCE activists, this statement was controversial to say the least. After all, the anti-UCE folks could see, quite clearly and correctly, that spam was about to overwhelm Internet service, at least for some people. Furthermore, we had all seen how spammers had ruined Usenet. The feeling in some quarters was, understandably, that email would be next, and so it was better to ban all unsolicited commercial email, and define UCE very tightly, in order to stem the rising tide (sorry, rising tide of spam is not a good image, but I think you get what I mean).
Quite logically, the people who tended toward this strong anti-UCE opinion on spam were those who had suffered most, like early adopters of email. Statistically, the longer you have an email address, the more spam you will get. By 1999 my old email address at 2cobbs.com was getting scores of spam messages a day at a per-day rate that seemed to increase every week. I got that address in about 1995. Furthermore, like many early domain owners, I was accustomed to enjoying the benefits of a "catch-all" account, i.e. you could email to any_name@2cobbs.com and I would get it. This was great for lots of things, not least of which was people who mistyped or nisheard my address name (you could call me scobb@2cobbs.com or scodd@2cobbs.com or dacobb@2cobbs.com--I would still get the message). A catch-all was also great for segregating email; if I registered at CNN's web site I would use cnn@2cobbs.com and use that to direct my CNN mail to the CNN folder.
But spam put an end to the convenient catch-all account because spammers started emailing to random names at target domains, first to see if they got a reply and thus a validated address, second to get past the basic barriers that Internet Service Providers were starting to erect. As spammers obtained more bandwidth we got dictionary blasts, i.e. sending the same message to aa@target.com, ab@target.com, ac@target.com and so on. You didn't want a catch-all account if one of these "attacks" came your way.
And "attack" became the operative word when early email adopters talked of spam, even as more recent converts to email, some still fascinated by the apparent benefit of getting a great deal on sunglasses out of the blue, dismissed such talk as extreme, referring to spam as a nuisance but nothing more. Believe it or not, nuisance was the term some politicians in Washington, and even some industry bigshots, used to describe spam as late as 2002. By that time I had my own well-documented spam stats that proved spam would, barring some successful industry initiative, become 90% of all email within the decade.
Many of my stats came from a very interesting experience that I had with a domain name: cobb.com. I acquired this domain name in 1999 after Ziff-Davis had used it for several years and then dropped it. I soon found that email kept arriving for addresses that were no longer valid, even if I turned the mail away. My spam bucket, the contents of which were hand-picked back then, was soon registering hundreds of UCEs per day.
When my buddies and I started to figure out ways of stopping spam, my collection of old spam proved very useful. I was able to prove that the business model for spam was very time-sensitive. Checking the URLs in stale spam showed that they expired very quickly. This practical limitation--that spammers had to abandon their sites very quickly in order to avoid censure, blocking, service cancellation or prosecution--provided the theoretical basis for an anti-spam model that slowed down spam delivery rather than filtered it. That model enabled the development of the first anti-spam router technology, a very effective approach to turning away spam at the perimeter of the enterprise network (as a product this was originally launched as SpamSquelcher and then re-launched as TurnTide and it is now used in many of Symantec's enterprise security products).
One reason not to filter spam is the problem of false positives, legitimate messages identifed as spam and thus prevented from getting to their destination. This proved to be a big problem for early efforts at instituting anti-spam measures in the enterprise. Sysadmins discovered that some C-level executives got very annoyed if the electronic newsletter they signed up for got discarded or quarantined (and some would sign up to pretty weird mailing lists). I even had sysadmins tell me that there was a hard core of 20%-25% of users who liked to get UCE. But in the years 2000-2004 some major changes occured in the world of email, some of which were driven by market forces, and some of which were good.
For a start, some spam turned ugly--for example touting porn and getting graphic--which tipped the scales for some folks who hadn't previously objected to it (and newcomers who had little idea of just how wide the WWW was, culturaly speaking). Then spam volumes started to ramp up and hit more people as their email addresses started to age. And spammers started faking sender addresses, which really confused newbies and upset a lot of politicians (I mean, if you can't trust that a message is from the person it says it's from). Eventually, the hue and cry against spam really took hold and market forces effected some positive changes, which I will deal with in my next post, along with some more observations from those early prognostications.
No comments:
Post a Comment