Attacks on the Dataset - Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification [Electronic resources] نسخه متنی

Attacks on the Dataset

There have been a few attacks on the actual data in the dataset. The ultimate goal is to either poison the filter with guilty data it thinks is legitimate or neutralize existing data that the filter thinks is guilty. Dataset attacks aren’t as widespread as other types of attacks, but they can be effective for sending out one specific distribution targeting a small group of users. Since these types of attacks require dedicated work and attention, spammers are unlikely to use them, and only a limited number of attempts has been seen.

Mailing List Attacks

Mailing list attacks have recently become popular as a last-ditch effort to get spam through. The purpose of abusing a mailing list is that many statistical filters learn over time to trust all messages originating from such lists, putting confidence in messages with the list’s headers, subject markers, and embedded taglines.

How It Works

The spammer subscribes to a mailing list and begins harvesting addresses from the list. Usually the spammer will spend a month or two just listening in to acquire a list of valid email addresses and will possibly even use a Bayesian algorithm to determine the most likely candidates for legitimate tokens among mailing list users. The attack begins in one of two ways. First, the spammer could easily just send a spam to the list. If the list is unmoderated, the spam will go out to all mailing list members with the headers, markers, and taglines specific to the list. Any users on the mailing list whose statistical filters have learned the particular characteristics of the list will be at risk of having the message misclassified as a legitimate message. Since this approach generally works only once per mailing list, the spammer will attempt to forge as many of the headers as possible and send out several spams at once from their own mail server. Even though some of the list’s headers won’t be present, such as the IP address of the list’s mail server, enough information can usually be copied from the original list email to spoof a legitimate message from the list.

Why It Doesn’t Work

Fortunately, this technique typically works only once or twice, as the statistical filter will then correct itself and learn not to trust the list based on its identifying marks alone. Not all filters train on every message either, and these are less likely to be tricked by this type of attack. Nevertheless, it can work once or twice. Since this approach requires work on the spammer’s part, you don’t see it very often. Also, spammers are able to target only individuals who are actually on the list (it won’t work at all to send a spoofed message to a nonmember). Since they are used to sending mass messages to millions of individuals per day, putting them in a box where they may reach only a thousand certainly cuts back on their ability to profit. In addition, most individuals on a mailing list are likely to be savvy users; they are not generally interested in spam, and therefore the response rate should be even lower for spamming a list.

The Importance

The lesson in mailing list spam is that there are many sources of email out there that our filters have trusted implicitly. Many people debate the wisdom of trusting individual sources, but everyone agrees that a mailing list should never be considered a trusted source, even if it’s moderated. Statistical filters do a good job of distrusting lists that spam has originated from, and so this error is generally dismissed as a process that is worked out by the user’s training loop. Filters that wish to implement some type of specific counteraction for this type of attack can refuse to assign extreme values to header tokens, although this generally isn’t necessary.

Bayesian Poisoning

Bayesian poisoning can be employed in many different ways, although the most common is through a mailing list or series of blank probes (email with an empty message body, usually sent for the purpose of detecting the existence of your email account). Bayesian poisoning is designed to trick the dataset into thinking that several different pseudo-tokens are legitimate. These pseudo-tokens are usually random strings made up by the spammer. Random strings of text on their own are meaningless and have no chance of evading a statistical filter, because unknown words are generally given a very neutral probability. In order for these random words to be successful in evading a filter, they have to be programmed into the filter as legitimate words. On top of this, they have to be interesting enough that they’re likely to appear in the decision matrix, and they must appear in enough abundance to flood the decision matrix with so much legitimate data that there is no room for the spam.

How It Works

Bayesian poisoning is a covert operation. The spammer will usually subscribe to a mailing list or find other ways to send innocuous email to a large group of users—email that has a very low chance of being reported into the system as spam. Apart from mailing list messages, this could potentially include blank messages or bogus mailer-daemon notifications. These emails will appear legitimate but will have several hidden tokens embedded in the message—usually in the headers. Over a period of time, these tokens will be learned by some statistical filters. It is beneficial for the spammer to send messages that are likely not even to be read but just deleted. What’s most important to this operation is that the target user remain unaware of what’s going on. One of the simplest ways to do this on a mailing list is to reply to an existing thread without saying anything of much importance.

To: <full-disclosure@lists.netsys.com> 
From: ross9917@Flashmail.com
Subject: Re: Cross-Site Scripting Vulnerability 
Date: Wed, 10 Jul 2002 04:09:18 -0700
MIME-Version: 1.0
Content-Type: text/plain
X-Wajdf0ief: wlekfjlwefk lkjfewln l fwekl ewfkj l1eoi1e02 21e 0e1j 0j
Wow, thanks for this information!

Over a period of time, several different subtle tokens are created in a user’s dictionary as the spammer continues to insert hidden headers. The random tokens would need to be repeated several times in order to adequately train the user’s dictionary. In addition to this, the spammer would most likely choose alternating sets of tokens so as to build a larger corpus of data in the user’s dataset.

To: <full-disclosure@lists.netsys.com> 
From: ross9917@Flashmail.com
Subject: Re: Vulnerability in Linux Kernel v2.4 
Date: Wed, 10 Jul 2002 04:09:18 -0700
MIME-Version: 1.0
Content-Type: text/plain
X-q0djq0dw9j: lkej lwk23 01 0fwj0 w0j 9 09jr320 j09jr32lnfdlkn lkf wef
Wow, thanks for this information!

Over a short period of time, these random tokens have trained TOE filters of all of the users on the mailing list. Ideally, the spammer would train 30 to 45 tokens minimum, to attempt to flood the decision matrix with data that now has a very low probability in the dataset. When the spammer decides to send the spam, they’ll usually send a microspam with as little guilty information as possible. What ultimately leads to the spammer’s success, however, is these once-unknown tokens, which have been trained as innocent. The spam gets sent, either through the mailing list or with forged headers.

To: <full-disclosure@lists.netsys.com> 
From: ross9917@Flashmail.com
Subject: Enjoy!
Date: Wed, 10 Jul 2002 04:09:18 -0700 MIME-Version: 1.0
Content-Type: text/plain
X-q0djq0dw9j: lkej lwk23 01 0fwj0 w0j 9 09jr320 j09jr32lnfdlkn lkf wef 
X-Wajdf0ief: wlekfjlwefk lkjfewln l fwekl ewfkj l1eoi1e02 21e 0e1j 0j 
<10 other poisoned sets of header data>
<html> 
<head> 
<title>Dietary Supplement</title> 
</head>
<body link=3D"#FFFFFF" alink=3D"#FFFFFF" vlink=3D"#FFFFFF"> 
<table width=3D525 cellpadding=3D5 border=3D0><tr><td><p>
<img src=3D"http://www.hebalist.com/m1.jpg"></p>
</td></tr></table>
<table width=3D525 cellpadding=3D5 border=3D0><tr>
<td valign=3Dmiddle><p>
<img src=3D"http://www.hebalist.com/m3.jpg"><br>
<a href=3D"http://www.hebalist.com?id=3D610">
<img src=3D"http://www.hebalist.com/m2.jpg"></a></p></td>
</tr></table>
<table width=3D525 cellpadding=3D5 border=3D0><tr><td><p>
<a href=3D"http://www.hebalist.com/servicel"> <img 
src=3D"http://www.hebalist.com/m4.jpg"></a></p> </td></tr></table>
</body>
</html>

Another way to poison users’ databases is to do just the opposite—to feed some would-be guilty tokens into innocent messages to deprogram the filter. Just a few words at a time is all the spammer can get away with, but if they’re able to dumb down words in the filter that they plan on using in future spam, they’ll have a much better chance of making their message through the filter.

Why It Doesn’t Work

Fortunately, once users receive the spam and train their filters with it, the tokens become useless to the spammer. The tokens also require more than a single message to train on—filters originating from Graham’s research generally require a minimum of three innocent occurrences before the tokens are considered “real.” The forged mailing list headers are also useless, as the data no longer favors the mailing list. The data from the Received: headers, which is not easily forged, has also been trained, marking the origin of the spam distribution.

All in all, this approach is not very beneficial to the spammers as it takes time and resources, and usually results in a very low success rate. If the process is spread out over too long a period of time, the spammer risks having the tokens purged from users’ datasets. The spammer has truly worked for their one spam in time and computing power. The approach is effective but is not lucrative for the spammer. Nevertheless, as statistical filtering continues to become mainstream, spammers may use desperate attempts like this more often.

Statistical filters that want to protect themselves against these types of attacks can take several precautions to avoid allowing the dataset to be poisoned. First, the filter could be designed to detect unknown headers and make sure they’re brought to the attention of the user (or administrator). Injecting unknown headers into a collaborative tracking system could easily identify these types of attacks before they became useful for the spammer. Even invisible ink is somewhat detectable in the message body, more so than headers that nobody looks at. Invisible ink is the process of hiding tokens by making them the same font color as the background. They alone are a good indicator of spam.

<FONT COLOR=WHITE>lwefkj lkjfe lkwfjlwekf</FONT>

Some filter authors prefer to ignore any text in invisible ink, while others don’t see it as enough of a statistical problem to worry about.

The second way of protecting against this type of attack is to change the training modes so as not to allow extreme data that would permit mailing lists and other tokens to become implicitly trusted. If your filter isn’t training on every new word, then it’s not likely to be poisoned by this kind of data. In order for a spammer to poison a dataset protected by TOE-mode training, for example, they would have to convince the spam filter to quarantine the message as spam and then convince the user that the message was a false positive. This is quite an ambitious task, however, and is improbable in most cases as even dumb users aren’t about to mark a suspicious message as a false positive—and the message would have had to been made suspicious in order to be quarantined in the first place.

Finally, some filters have experimented with giving preference to guilty tokens over legitimate tokens when the probabilities are tied. This will ensure that decision matrices don’t get flooded with bogus data but that the guilty data will always be considered first. Other implementations attempt to inter- leave the dispositions of tokens with similar probabilities.

The initial philosophy when statistical filters were designed was to give bias toward legitimate mail. As filters have become more and more accurate, this bias has started to tip in the other direction, and ideals that were once rejected for fear of false positives are now feasible solutions that actually improve the accuracy of filters. This makes perfect sense, as the average user’s email is now upward of 85 percent spam. Some unfortunate souls out there receive 90 to 95 percent spam.

The Importance

Bayesian poisoning warrants some concern, but because it requires a lot of work on the spammer’s part, it isn’t something filter authors should be overly paranoid about. It’s important to find ways to counter Bayesian poisoning to avoid the possibility for this approach to become widespread in the future. Decommissioning this type of attack by implementing training modes other than TEFT or by finding other defenses will help prevent Bayesian poisoning from becoming mainstream. Mail client authors should also become aware of better ways to alert the user to unorthodox headers to expose innocent looking poisoning attempts. Ultimately, Bayesian poisoning could be used successfully to circumvent some filters, and therefore it’s important to consider training, collaborative filtering, and other approaches for fighting it before it’s an issue.

Empty but Not Empty Probes

Lately, a flood of emails with empty message bodies have been circulating around the Internet. Many of these messages have been simple probes by spammers to figure out what email addresses are valid—some implementing a form of web bug hidden in HTML code and others using exclusion lists based on the bounce messages returned. A web bug is an HTML object embedded in an email, such as an image or other resource, that loads automatically when the email is previewed or opened. When the user’s email client makes a connection to the spammer’s server to acquire the resource, a special piece of data embedded in the URL “phones home” telling the spammer which recipient is requesting the object. This confirms the validity of the email address and also confirms that the person received the probe. Most newer email clients have a feature to prevent the loading of external images (and other objects), but this feature is usually disabled by default.

How It Works

The philosophy behind empty message probes hinges on the presumption that the user will simply delete the message rather than train their filter on it. Since many filters are still cumbersome in training, users generally prefer to train spams only when they receive one. Because most users don’t realize that many of these empty messages are spam, they’re more likely to just delete them rather than insert them into whatever type of training cycle is available for the filter.

The empty messages can contain invisible headers, just as they do in Bayesian poisoning attempts. As the user continues to delete these messages, the tokens will eventually train in the user’s dataset. Finally, a single spam is sent, using these tokens as “keys” to fool the statistical filters. Spammers know they can use these tokens only once before they have to be reprogrammed, and so it is more beneficial to try to perform this type of training on a mass crowd of millions of users. Empty messages are one of the only vehicles for doing this.

Why It Doesn’t Work

There are some obvious problems with this approach. We’ve already discussed the problem that tokens become useless once the first spam is sent, after which they become only a marker for additional identification of the spams. Another problem with this approach is that a series of blank messages is far less likely to make it through an individual’s inbox without raising some suspicion—after a few, someone will undoubtedly begin to examine the messages and either discover what’s going on or automatically punt them into the spam bucket. The third problem is that many spam filters (among other tools) will dump any messages containing an empty body. Finally, as we discussed previously, empty messages don’t necessarily train every filter out there, only TEFT-based filters. If the tokens that the spammer is attempting to deprogram have already matured, this approach won’t work in TUM- based filters any more than it will in TOE-based filters.

In conclusion, all attempts to poison a user’s dataset are potentially effective only with filters that perform TEFT-mode training. Although filters that perform this training are susceptible to these types of attacks, the attacks themselves are very tricky and require a lot of resources that most spammers just won’t find profitable. Too many people make the mistake of thinking that spammers are motivated by sending spams; they’re not—they’re motivated by how much money they can make. While in many cases, sending email does equate to making money, most spammers do realize over long periods of time that attempts like this are mostly futile and not nearly as lucrative as blindly sending out ten million messages.

The Importance

Probes should be trained into the spam filter for all of the reasons just discussed. Since they could be used to train new tokens and exploit some training modes, empty messages could become more widespread if users aren’t educated in the proper handling of these potential probes.