Introduction
In order to struggle spam, companies exploit spam traps for finding out and blocking spammers. A spam trap is a destination, which is used to lure spammer attacks. One example of spam traps is an email address, which is not used, but incoming traffic to this address is monitored. A lot of spam traps exist in the Internet expanses; they are generally managed by big anti-spam organizations (SURBL, Spamhaud), security companies (McAfee, TrendMicro), Internet Service Providers and corporate email server. The possible types of spam traps are typos (mistyped email addresses), expired addresses (not used for long time), purchased (the lists looked as stolen data sets of employee information) (Van, 2013, p. n. d.).
The results of sending email to a spam trap can be following:
- damage of sender reputation, which causes in decrease of delivered messages percentage
- IP address of sender can be put into blacklist, that also means decrease of delivered messages percentage
- if spam trap of an ISP (AOL, Yahoo) is hit, the consequence can be dreadful: whole domain of sender can be blacklisted
- if spam trap of an anti-spam organization (SpamCop, Spamhaus, Abusix) is hit, all ISPs and companies who negotiate with this organization will be influenced (Campaign Monitor, 2015, p. n. d.).
Spam traps results are widely used for IP based blacklisting. RBL (Real-time Black Lists) and DNSBL (Name Server Black Lists) are IP addresses lists, which are looked up in real-time. More widely used DNSBL/RBL are:
- RNBL (Return Path Reputation Network Blacklist): uses spam trap and compliant sources for its predictive model
- SBL (sbl.spamhaus.org): volunteer editors manually supplements lists, by looking to senders, who hit their spam trap networks
- XBL (xbl.spamhaus.org): most IPs are located here as a result of mailing viruses and spam to spam traps of Spamhaus
- PSBL (psbl.surriel.com): this list contains the list of IPs that sent messages to their spam traps and IP is not a notorious mail server (Mitchell T., 2014, p. n. d.).
In this report the techniques, used for corporate and private domains will be analyzed, which will help to protect websites, blogs and company’s mail.
In this section the review of techniques for securing email domains will be presented. The configuration of Bat and MDaemon message transfer agents will be considered.
In order to secure POP3 or IMAX4 mailbox such as Outlook, Outlook Express, Eudora or Bat, SpamPal software can be applied. SpamPal is free software for mail classification, which facilitates removing spam from useful mail messages. This program is applicable for Windows Operating System, so in case of Linux usage, SpamAssasin software is recommended. SpamPal cannot work with proprietary mail systems, such as Hotmail, Juno, Gmail, Microsoft Exchange, MSN and etc. The principle of SpamPal functioning is the following: it locates between email program and mailbox, and checks email messages. Email messages which are considered spam by the SpamPal are “tagged” with special mark; it is just needed to configure email client software to put all messages with that header to special folder. As a result, spam would not be mixed with normal email messages. In order to define whether retrieved message is a spam or not, this program uses DNSBL lists. It is possible to use any of the free DNSBL lists with SpamPal. SpamPal will check every message on the way to mailbox and if its source is from DNSBL lists then this message will be considered as spam. For preventing false positives (not all from DNSBL lists will be spammers) SpamPal provides convenient white listing feature, which permits to ignore DBSBL listings for definite senders or for Internet parts.
For installing SpamPal double click spampal.exe (program for SpamPal Setup) and follow displayed instructions. SpamPal installation will run and after that icon, which looks like pink umbrella, will be shown in system tray. First of all, it will be required to change POP3 settings. In order to accomplish it, load the Bat and then choose Properties from the menu Account. Click Transport option in the left list.
The changes will be needed in fields Mail Server and User. Copy to notepad name of MailServer and replace it with localhost or 127.0.0.1. Add @ sigh and POP3 server name from notepad to User field.
Figure 1: POP3 settings for Bat
For saving changes click OK button. If no error messages were sent to mail, then continue to the next step. Perhaps it will be needed to enter again POP3 passwords. In case of error messages check that incoming POP3 server is set to localhost and port has been configured correctly. In case of SpamPal error, check the server name was correctly attached to the username and Internet connection is available.
Secondly, create filter and message rules. In case of using IMAP4 server, it won’t be needed to setup this rule, because all spam tagged messages would be automatically sent to inbox.spamtrap folder. In case of using POP3, choose Sorting Office/Filter from the menu Account. Select option Incoming Mail and click New button. On the shown up window, open Rules tab, set filter sensible name and in the 1st Filtered Strings box write ^X-SpamPal: SPAM, and mount Location to Anywhere. After that, checkmarks Delete the message and Create a copy of message in another folder (SpamPal Website, 2015, p. n. d.).
For access spam filter setting of MDaemon press Ctrl+P. By default MDaemon detects a spam email, and scores message before giving it opportunity to be delivered to the end user. Generally, it is more preferable to put spam tagged messages to ‘spam trap’ folder, which will be checked by an administrator for false positives. In order to do it, choose second option in the section ‘What to do with spam’.
Figure 2: Spam Filter options for MDaemon (Zen Software Ltd, 2015, p. n. d.).
In MDaemon it is possible to set up spam traps. In MDaemon all messages, which hit spam traps are feed into Bayes filter as spam samples for algorithm learning. It that case, it is required to use addresses, to which only spam letters come. For provision of the best results of spam luring, that addresses should be addresses, which are most intensively used by spammers.
First of all it is needed to find out – to which addresses spam comes more frequently? In order to accomplish it – remove prohibition to receiving letters from non-existent addresses.
Secondly, there is a need of collection of all messages to non-existent receivers in one place, in order to feed them all to Bayes algorithm. MDaemon can resend such letter to own postmaster, so use this option.
Figure 3: Removal of prohibition to send email to non-existent receiver mails
Figure 4: Checking option of sending messages to postmaster user
As a result there will be notifications about messages about non-existent address.
Figure 5: Notification about letter to non-existent address (eLearning for MDaemon Messaging Server, 2016)
Now it is needed to load all attachments from notifications and analyze them. In order to do it VBA Macros can be used with MS Outlook. The logic of attachment loading is the following:
- user selects in folder all messages for analysis and turns on Macros
- VBA Macros looks over all selected messages and saves attachments of every message to special folder.
'Save attachments from selected messages
'to folder DestFolder
'To do this:
'1) Select messages at Outlook
'2) Turn on this macros
Sub SaveAttachs()
Dim myOlApp As New Outlook.Application
Dim myOlExp As Outlook.Explorer
Dim myOlSel As Outlook.Selection
Dim MsgTxt As String
Dim x As Integer
Dim y As Integer
'Folder to save attachments
DestFolder = "C:\_tmp\33\"
'
Set myOlExp = myOlApp.ActiveExplorer
'Get object with all selected messages.
Set myOlSel = myOlExp.Selection
'Look up all messages
For x = 1 To myOlSel.Count
'Get object with attachments of current message
Set MyOlItemAttachments = myOlSel.Item(x).Attachments
'Look up all attachments
For y = 1 To MyOlItemAttachments.Count
'save them to folder DestFolder
MyOlItemAttachments.Item(y).SaveAsFile DestFolder & MyOlItemAttachments.Item(y).FileName
Next y
'
Next x
MsgBox "Ready"
End Sub
Listing 1: VBA Macros for loading trapped messages (Bakanov D., 2010, p. n. d.)
In that way the addresses, to which spam is sent frequently are found and they can be used for creation of spam traps. During this experiment the following addresses to which spam was frequently sent were discovered:
-
-
-
-
-
-
After creation of this spam trap fake addresses, all letters which come to these email, will be considered as spam.
Blog spam
In order to defend from spam bot attacks the trap for spam comments can be used. This trap is additional field for text input, which is not visible to usual blog visitors. When spam bot comes to website or blog it fills all input fields, so it will fill this invisible field, which usual users won’t even notice. While sending spam message with filled invisible field, the bot will be redirected to error page and comment won’t be sent. The advantage of this solution is easiness of installation and no use to apply any additional plug-ins, which create more workload to website.
<textarea id="comment" class="textarea" name="comment"></textarea>
Listing 2: Initial webpage source code (Kolosov D., 2014, p. n. d.)
It is needed to hide the main field and show additional for user input.
<div class="smm">
<textarea id="comment" class="textarea" name="comment"></textarea>
</div>
<textarea id="real-comment" class="textarea" name="real-comment"></textarea>
Listing 3: Webpage with hidden input field (Kolosov D., 2014, p. n. d.)
We assigned class to main field and now add styles to this class and hide field from usual users:
.smm {display: none;}
Listing 4: CSS for hiding field (Kolosov D., 2014, p. n. d.)
Add function to template file functions.php to loop.
add_filter('pre_comment_on_post', 'verify_spam');
function verify_spam($commentdata) {
$spam_test_field = trim($_POST['comment']);
if(!empty($spam_test_field)) wp_die('Spam will die!');
$comment_content = trim($_POST['real-comment']);
$_POST['comment'] = $comment_content;
return $commentdata;
}
Listing 5: Function to deal with spam comments (Kolosov D., 2014, p. n. d.)
Website defend from spam bots
In order to capture spam spider bots, which are may be finding mails from your website use invisible link (Webmaster World, 2006, p. n. d.).
Use special file directory /bot-trap:
1. Create subdirectory /bot-trap.
2. Make the following writing to robots.txt file
user-agent: *
disallow: /bot-trap/
Listing 6: robots.txt record (Internet Technologies, 2006, p. n. d.)
3. In the initial page of website at the beginning put invisible (size 1*1 pixels) hover.
/bot-trap/
Listing 7: Invisible hover (Internet Technologies, 2006, p. n. d.)
4. Put index.php to /bot-trap subdirectory. With this script IP addresses will be automatically saved to blacklist.dat file (this file should be created in advance and be available for RW).
<?php
if(phpversion() >= "4.2.0") {
extract($_server);
}
?>
<html>
<head><title> </title></head>
<body>
<p>There is nothing to do. What are you doing here?</p>
<p>http://your.domain.ru/</p>
<?php
$badbot = 0;
/* look up all records from blacklist.dat to prevent double recording */
$filename = "../blacklist.dat";
$fp = fopen($filename, "r") or die ("Open file error <br>\n");
while ($line = fgets($fp,255)) {
$u = explode(" ",$line);
if (ereg($u[0],$remote_addr)) {$badbot++;}
}
fclose($fp);
if ($badbot == 0) {
/* Send information to webmaster */
$tmestamp = time();
$datum = date("y-m-d (d) h:i:s",$tmestamp);
$from = "";
$to = "";
$subject = "Attention! Spam bot was at website!";
$msg = "Бот пришёл с $request_uri $datum \n";
$msg .= "адрес $remote_addr, агент $http_user_agent\n";
mail($to, $subject, $msg, "from: $from");
/* add record to file blacklist.dat */
$fp = fopen($filename,'a+');
fwrite($fp,"$remote_addr - - [$datum] "$request_method $request_uri $server_protocol" $http_referer $http_user_agent\n");
fclose($fp);
}
?>
</body>
</html>
Listing 8: /bot-trap/index.php (Internet Technologies, 2006, p. n. d.)
5. In order to prevent repetitive coming of spam bot to website pages, check file blacklist.dat before loading all website pages.
<?php include($_server['document_root'] . "/blacklist.php"); ?>
Listing 9: Include script to check blacklist.dat before page loading (Internet Technologies, 2006, p. n. d.)
<?php
if(phpversion() >= "4.2.0") {
extract($_server);
}
$badbot = 0;
/* look up blacklist.dat records */
$filename = "../blacklist.dat";
$fp = fopen($filename, "r") or die ("File open error <br>\n");
while ($line = fgets($fp,255)) {
$u = explode(" ",$line);
if (ereg($u[0],$remote_addr)) {$badbot++;}
}
fclose($fp);
if ($badbot > 0) { /* this is spam bot */
sleep(12);
print ("<html><head>\n");
print ("<title>Website is not available. <title>\n");
print ("</head><body>\n");
print ("<center><h1>Welcome!</h1></center>\n");
print ("<p><center>This website currently does not work </center></p>\n");
print ("<p><center>If you are sure that this is error, write to webmaster.</center></p>\n");
print ("</body></html>\n");
exit;
}
?>
Listing 10: Script of preventing access to spam bots (Internet Technologies, 2006, p. n. d.)
Keeping mailing list free of spam traps
In case of using mailing lists, it is good idea to maintain it clean. Every 6 months remove old subscribers. Do not use mailing lists from different websites and forums, and even do not purchase them. Use double opt-in.
Conclusion
Spam is any undesirable message sent to user. Methods for spam filtering were considered in this research (including Deliverable 1): use of URL regular expressions, text signatures and evaluating sender reputation in real time, blacklists (DNS, ISP, corporate website Black Lists). This research is important, as the spam is wide problem, which results in frauds, user annoyance and message overwhelming.
Configuration of the corporate mail servers (Bat, Daemon) was presented. Generally, the following steps should be performed for blacklisting spam messages: 1) finding out non-existent addresses to which messages are tried to be sent (also good choice is to use expired mail address) 2) collect messages sent to these emails (spam traps) 3) analyze these messages URLs, contents and sender IP addresses by different algorithms, such as Bayes networks and put required information to black lists 4) use learnt out algorithms and black lists in order to filter spam emails.
Moreover, methods of spam trapping for blogs and website bots were reviewed. Basically, feature, which is noticeable to bots, but not visible to usual users, was used in these scenarios.
In order to prevent false positives, white lists are widely used. This means creating lists of IPs, senders and etc., who will be considered legitimate, even if they are present in black lists. The future research may be connected with research of spam trap results analysis algorithms, with the problem of how to learn algorithm by spam trap data in the most effective ways: algorithm inputs and methods of analysis.
References
Van, 2013. Where Spam Traps Come From and How They Work. Retrieved from: http://blog.mailchimp.com/where-spam-traps-come-from-and-how-they-work/
Campaign Monitor, 2015. Spam trap email addresses and how to deal with them. Retrieved from: http://help.campaignmonitor.com/topic.aspx?t=802
Mitchell T., 2014. Blacklist Basics: The Top Email Blacklists You Need To Know. Retrieved from: https://blog.returnpath.com/blacklist-basics-the-top-email-blacklists-you-need-to-know/
SpamPal Website, 2015. SpamPal for Windows. Retrived from: http://spampal.sourceforge.net/index2.html
Zen Software Ltd , 2015. MDaemon configuration recommendations for dealing with spam related issues. Retrieved from: https://www.zensoftware.co.uk/mdaemon/resources/Configuring-MDaemon-to-deal-with-spam.pdf
eLearning for MDaemon Messaging Server, 2016. How To Configure MDaemon Messaging Server. Retrieved from: http://www.mdaemon.co.nz/Products/MDaemon/eLearning
Bakanov D., 2010. Defining email addresses for using them as spam traps in MDaemon. Retrieved from: http://wringen.ucoz.net/news/opredelenie_email_adresov_kandidatov_dlja_ispolzov/2014-03-08-140
Kolosov D., 2014. Trap for spam bots for WordPress comments. Retrieved from: http://creatorinfo.net/lovushka-spam-botov/
Webmaster World, 2006. Anti spam bots. Retrieved from: https://www.webmasterworld.com/forum88/12987.htm
Internet Technologies, 2006. Spam bot traps. Retrieved from: http://www.internet-technologies.ru/articles/article_441.html
