Using our spam blacklists

• Introduction
  
• Register as a blacklist user
  
• Donate with PayPal
  
• URLs for download
  
• Blacklisting policy
  

Despite the high volume of additions (about 2000 domains and 2000 email addresses per week) we have maintained an extremely low error rate, a fact we take pride in. This low error rate is due to a conservative blacklisting policy (see here for more details) and manual inspection. We also handle email inquiries about them. This takes time. We can maintain this service with your support:

• We don't currently restrict access to our domain blacklist or 419 sender address lists for filtering email to paying customers because we want to give you a chance to try the data before you pay anything. However, we encourage you to make a voluntary donation or monthly payment (subscription) once you use our data regularly. This applies especially if you are a commercial user (corporate IT department or ISP).
• In chosing your donation, please consider the amount of time saved by your user base and support department for every spam email that doesn't get through. If someone is paid as little as $15/hour, then every minute this person does not have to waste on wading though spam will save you 25 cents. Whether you donate 25 cents or $1.00 per user per month ($3-$12 per user per year), it is money well spent. It will give us the resources to provide more anti-spam data more quickly.
• You are welcome to download plain text files of both lists, but as a courtesy we ask you to leave your name, organization and contact address. If possible, also specify the IP address or network range from which you will download. This allows us to notify you if there are any changes to our service, such as a change in download URL. This is strictly for communications related to use of these data sources.

Register as a blacklist user
If you're going to download our list on a regular basis, please contact us with the following information:

• Name and email address of contact
• Name of organization
• Number of email users
• If possible: Static IP address or address range used

Donate with PayPal
We welcome donations of any amount. Alternatively you can also order copies of our spamfilter software. Here are some examples of payment schedules:

Personal use: $10 or more, at least once

Small Business: US$20 per month 20 users x $1/month

Large Corporate: US$125 per month 500 users x $.25/month

Other amounts: talk to us!

joewein.de LLC is a Limited Liability company based in Yokohama, Japan. We can provide you with a commercial invoice.

Download URLs of plain text files
Here are plaintext versions of our blacklists. The domain blacklist consists of two files, the 419 blacklist of one file:

• Domain blacklist base (spamvertised domains):
  This file is updated every few weeks and need not be downloaded more than once a week. It contains the bulk of the spam domains: http://www.joewein.net/dl/bl/dom-bl-base.txt
  
  
• Domain blacklist new (recently added spamvertised domains):
  This file contains additions made during the last few weeks only. You can download once per day or more, but we currently don't recommend intervals more frequent than hourly. http://www.joewein.net/dl/bl/dom-bl.txt
  
  
• Email sender blacklist (419 scam and other spam senders):
  http://www.joewein.net/dl/bl/from-bl.txt
  

MD5 checksums
The following very small files contain hash codes computed from the above files. You can download the following files every hour or even every 15 minutes (make sure your script works properly before you try this rate!) and then run "md5sum -c filename" on each one. If the checksum fails it means the corresponding data file has changed and it's time to download it as well. That way you will never download copies of the actual data files unless they have changed.

http://www.joewein.net/dl/bl/dom-bl-base.txt.md5
http://www.joewein.net/dl/bl/dom-bl.txt.md5
http://www.joewein.net/dl/bl/from-bl.txt.md5

Blacklisting policy
These days we are aiming primarily at blacklisting domains that have no legitimate uses. There are a number of domains that have questionable privacy policies or no confirmed opt-in (closed loop) subscription process and are often reported as spam that we don't list, because some people do indeed subscribe to their sites.

The current blacklisting procedure has been in place since December 2003. All entries added to the list before that have been purged. Our false positive rate is less than one per month, which means an error rate below 0.01%. None of these have been widely used domains. Here are the main points about our process:

• We are trying to be conservative in our blacklisting. We recognize that false positives are far more painful and costly than false negatives. That means: If in doubt, don't blacklist. Use built-in checks and double check whatever you can.

• We don't blacklist on hearsay. Every entry is backed by at least one evidence email originally sent to our mailboxes or to customers of an ISP we're cooperating with. We recognize that there are Joe jobs, fake sender addresses and innocent bystanders mentioned in spam. We make efforts to detect these cases.

• In order to minimize false positives, we start out with a pre-selected set of messages. Many of the mails we receive at our domains go to largely or completely unused accounts that we don't sign up for anything. Furthermore, unless these mails meet certain criteria, our spamfilter won't even look at the embedded domain names. At our partner ISP every mail has to reach a certain SpamAssassin score before our filter gets to take a look at it.

• Every mail then goes through our in-house spam filter, which extracts domains names, makes WHOIS queries and together with other data about the original mails, stores the information in a database. It sorts domains by perceived spamminess, taking into account factors such as domain age, registrar, supporting name servers, Spamhaus SBL records for related servers, etc.

• Domains registered by a fixed small set of hardcore spammers such as for many of the "OEM software" and pharmaceutical spams are automatically detected and blacklisted.

• Other domains get sorted into several bins for manual inspection. This is where it gets labour-intensive. We generally discard the least suspicious domains because there's too much of the more interesting stuff.

• For the more suspicious ones we look at the reasons the filter didn't like the mail, the sender, the subject, we check WHOIS info, the actual message itself, we perform Google web searches, Google NANAS lookups, etc. We look for signs mail for third parties might have been legitimate and subscribed to or - the opposite - for signs of obfuscation to defeat filters, in order to determine if it may be a legitimate newsletter or not. This is not always easy if the recipient is a third party, but there are certain patterns that can be detected.

• The older a domain, the more evidence we require to list it. SBL listings are a strong indicator but not sole determinator of spamminess. We always judge several factors in combination.

• We don't currently have a process for purging discarded spam domains, but are working on that.

• If a listing is challenged, we provide information about the email that triggered the listing, but without identifying the mailbox it was sent to. If the listing appears to be because of a mistake or if we think it is unlikely the domain will appear in spam again we remove the listing.

Are you sick of spam too? Do you want it stopped now? Try jwSpamSpy, the spamfilter we use to track the spammers! Free 30-day trial version available now!