How Sorbs can fail

[kas]

I depend highly on Blacklists to sort out who are known spammers or email harvesters and who are normal human beings who just want to register and partake in the conversations here.

However I have seen blacklists receive a lot of flack. The more I read people's responses on many site regarding this the more I realize why.

Let's take Sorbs for example. Sorbs stands for "SPAM and Open Relay Blocking System" but many people have ended up on the SORBS list without even sending spam due to inclusion through ranges submitted by their ISP who use the SORBS system to detect spammers without any consideration to hijacked computers via trojans and IP's becoming clean later on when the computer is cleaned. Spammers tend to not use the same IP for very long because they get banned from where they are spamming. They tend to like to used what are called "Zombie Computers" so that they don't get caught but innocent computer users take the fall. Sometimes even spam fighters get falsely caught (false positives) and usually there is a way to white list your IP or contact someone to prove you are human. It is normal that if your IP is flagged again within a certain amount of time that you won't be able to get whitelisted again but Sorbs does not offer any way to white list your IP even though they say you can delist.

Take this example of this run around of someone trying to delist as they are listed for only ONE activity while their computer was being used by a "net-noob" family member and then got redone and cleaned of any trojans / virile files.

They found it to be such a run around and then found out that they didn't send out any spam at all (they thought that they might have and assumed the family member had infected their computer even though they never found any trojan but redid the computer anyway) but that their ISP had added their whole IP range to SORBS!



Now as a spam fighter who has tried to educate people whose computers have sent spam -- or easier to prove, email accounts have sent spam -- and I can understand how people could just say they haven't sent out spam and just not realize that it doesn't mean that their computer never has. However I have seen numerous complaints by spam fighters and well educated techs complaining about Sorbs and how there is no info on what the spam was or where it came from or a non-error way to delist. Sorbs seems set up more as just a way to annoy spammers but what about real people? Even honeypots still have room for if you are a real person to leave before just tagging you as a spammer. You have to actually spam. But for Sorbs to have IP ranges listed that never spammed (just because 100 IP's out of 255 might spam, does not mean that all 255 belong to spammers) and even if your computer used to send spam but was eventually cleaned, there is no way to get off the list.

ISP's and webhosts just look at every anti-spam site as a good way to go but don't actually look closely at it. I have compared many and have my favorites for many various reasons and for different things. I also am aware of the pros and cons of each and know there is no one perfect system. However Sorbs seems to think they are perfect.

Read more here on a thread where people talk about the issues with Sorbs and even the owner of Sorbs (named Sorbs on there) posts!:

forums.whirlpool.net.au/forum-replies.cfm?t=1227139

Also it seems that sorbs has also blocked some sites that there are no problems with. au.sorbs.net went down so I checked it with:

www.websitedown.info/sorbs.net

but it says:

Website is down for everyone. This site is currently unavailable. It's not just you!

12/06/2010 23:17:58
sorbs.net is down

Is sorbs.net down? Yes!
Our site checker did not receive any HTTP response status code for your query.
This site is unavailable for everybody not just you or you have misstyped the domain name!

Possibly the sorbs.net web server is down, overloaded, unreachable (some network related problem) or a website maintenance is in progress.

Also this could be a DNS (Domain Name System) lookup problem like incorrect settings and / or configuration of the DNS servers.

However when I typed in www.sorbs.net instead of au.sorbs.net in the address bar .. it worked and I could even log in!!

I had tried to see if it was down because I got an error and tried to contact Sorbs about it at the email address on the error page. However even the email bounced. Turns out they are still up but I guess their subdomain over at au no longer works... They now moved to the us subdomain it seems??

Also this site said it is down as well, showing an page not found error page:

www.domaincrawler.com/domains/view/sorbs.net

[kas]

Oh I also wanted to include errors that were part of the problems on Sorbs:

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, [e-mail] and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

Software error:

Open DB Handle needed at /usr/dnsbl/htdocs/cgi-bin/db line 148

For help, please send mail to the webmaster ([e-mail]), giving this error message and the time and date of the error.

However when I sent an email the email bounced back to my gmail account (I do not use a mail client due to feeling that gmail is more secure) and the bounced email said:

from Mail Delivery Subsystem
to *private*@gmail.com
date Mon, Dec 6, 2010 at 7:01 AM
subject Delivery Status Notification (Delay)

This is an automatically generated Delivery Status Notification

THIS IS A WARNING MESSAGE ONLY.

YOU DO NOT NEED TO RESEND YOUR MESSAGE.

Delivery to the following recipient has been delayed:

[e-mail]

Message will be retried for 2 more day(s)

Technical details of temporary failure:
The recipient server did not accept our requests to connect. Learn more at mail.google.com/support/bin/answer.py?answer=7720

[anaconda.sorbs.net . (5): No route to host]

[catapilla.sorbs.net . (5): No route to host]

[desperado.sorbs.net . (10): Connection timed out]

[scorpion.sorbs.net . (15): Connection timed out]

[kas]

Also more people complaining of not being able to delist when they have not had any bad activity for years:

osdir.com/ml/users-spamassassin/2010-10/msg00079.html

osdir.com/ml/users-spamassassin/2010-10/msg00080.html

osdir.com/ml/users-spamassassin/2010-10/msg00082.html

osdir.com/ml/users-spamassassin/2010-10/msg00084.html

[kas]

Seems the people just realizing there are problems don't check it monthly and just now are finding themselves listed .. so when I found this I thought this was interesting:

www.theregister.co.uk/2010/10/07/sorbs_cockup/

Spam blacklist snafu prompts global gnashing of teeth

Legit IPs blocked in SORBS cockup

By Dan Goodin in San Francisco • Get more from this author

Posted in Spam, 7th October 2010 21:03 GMT

Many email users around the world have been unable to send messages because of ongoing technical problems with a popular service designed to prevent spam from reaching its intended destination.

The problems at SORBS — short for the Spam and Open Relay Blocking System — began on Wednesday and continued into much of Thursday, said Michelle Sullivan, who founded the real-time blacklisting service in 2002 and sold it to GFI Software last year. As a result, messages sent from a huge number of legitimate mail servers were labeled as junkmail and returned to sender.

The widespread outage caused migraines for sales and support staff around the world, as this ongoing Twitter feed demonstrates.

The snafu was the result of a transition from one SORBS system to another that corrupted a database containing potentially millions of IP addresses, Sullivan told The Register. SORBS admins have responded by temporarily clearing out the entire table of faulty listings under the theory that it's better to let through spam than to block real email. They are in the process of rebuilding the database and populating it to user servers around the world, a process that could take up to 24 hours.

Sullivan denied reports such as those here and here that the problem caused all inbound email from Google, Yahoo, and Rackspace, to name a few, to be blocked from SORBS subscribers. The portion of the database that was corrupted stored entries for its DUHL, or dynamic user host list, which mainly contains dynamic IP addresses offered by ISPs, Sullivan explained.

The outage was nonetheless large. There were more than 79,000 corrupted entries, each of which could contain millions of individual IP numbers, said Sullivan, who was unable to provide an estimate of the total number of addresses that were incorrectly being filtered.

The DUHL portion of the database contains a huge amount of historical data, mainly in the form of IP addresses that were listed as spam sources in the past but are no longer considered as such. During a data migration, the flags that were used to indicate that a listing was historical were deleted, causing the addresses to be considered current. SORBS is in the process of restoring the flags. The historical entries are used to help programs rate the likelihood that a given IP address is malicious.

To make matters worse, the database problem coincided with a denial-of-service attack that made it hard for people to reach the SORBS website. The DoS assault has been ongoing for several weeks, Sullivan said, but in the last 24 hours, it turned “smart,” meaning it was able to evade the defensive measures admins had put in place to filter out the garbage traffic.

SORBS has been able to repel the latest attacks, although the site still appears to be sluggish at times. The attacks in no way caused the database problems. ®

[kas]

More here:

blog.utest.com/interview-michelle-sullivan-founder-of-sorbs-part-i/2010/10/

Chris Smith said:

Maybe you should have a part 3 and ask Michelle what her thoughts on blacklisting entire /13 /12 subnets via duhl in the last 3 days are? Why their delisting procedure claims it does dns lookups when in fact it is caching (incorrectly I might add) ttl data and counting down?
Posted on December 1st, 2010 at 1:12 pm

blog.utest.com/interview-michelle-sullivan-founder-of-sorbs-part-ii/2010/10/

Michelle Sullivan said:

The SORBS fine does not exist any more.

With the deployment of SORBS v2.0 we have discontinued the fine; instead we are taking a time-based penalty approach. I have in the past been a supporter and a hater of the SORBS fine, which often confused people. I hated having it, but saw no other option. I believe that people running the servers for spammers, whether knowingly or unknowingly (e.g., by having a robot on their machine) or ISPs and hosters running the DNS and webservers for profit are responsible for the spam in their own way. I feel that those people need to be encouraged to take a more responsible and active role in the prevention of spam by proactively ensuring their machines do not participate in botnets, that their sales staff do not sell ‘pink contracts’, etc. The fine outraged many people, until they realised that:

* SORBS never received a penny of the fine
* They wouldn’t likely have actually done anything if they could ‘just delist’
* Policies for some very large ISPs of the world have changed because of it

The thing I find interesting is that I know for a fact that many, if they knew what they COULD do about it, would but most are not as tech savvy as s/he and need guidance. Payment is a pain in the ass and does NOT get anywhere toward fixing the spam problem, despite what Michelle Sullivan thinks.

uTest: People seem to have a love/hate relationship with blacklists. On the one hand, they cut down on a lot of spam. On the other hand, nobody wants to be a false positive. The consequences of having a mail server accidentally be blocked can be catastrophic for a business or individual. What kinds of processes does SORBS use to make sure each listing is legit?

MS: This goes back to my previous comments on transparency all the submission systems for SORBS v2.0 are based on specific events, and they are automated, this means that there is virtually no possibility of error. Of course this doesn’t mean a thing if your error-free data is mangled on transfer when performing a migration, but within normal SORBS operation every piece of data is signed by the host in a way that is easily detectable that the host sent the data and it is un-tampered with. Should the tamper check fail, we discard the information, even if it is legitimate. This helps us ensure that the data in the database is always correct at time of insertion.

BS. They accept lists from ISP's that submit whole ranges even though not every IP in the range has done any spamming what so ever. I know this for a fact as one ISP listed only has static IP's yet the whole range has been banned which does not make sense unless it was a dynamic IP.. which oddly enough Sorbs lists the IP as being...

Randolf Richardson said:

mike: You are missing the point of DNSBLs like SORBS.net. Your problem is that your ISP is listed because they meet SORBS’ criteria, and you need to either get your ISP to resolve the listing (usually by terminating accounts that are operated by spammers), or switch to an ISP that refuses to provide services to spammers.
Posted on November 2nd, 2010 at 3:32 am

Hmm... now I wonder .. is the aforementioned ISP known for providing services to spammers or did they submit the the IP range to SORBS after all??

Antu Mathiyas said:

Randolf: You are wrong. We are the ISP, and we deal with this useless guys past 4 years. We gave all our subnets both currently assigned, and unassigned. Recently we assigned 4 number of /24 subnets for our customers statically from Aug 1st to till date (those are unassigned previously and listed on DHUL by default), We send the request to them 2 in August and 2 in November. Until now, i didn’t receive any update from these useless fellow, and we are unable to answer our customers those are running servers behind their IP.
Posted on November 2nd, 2010 at 10:34 am

Ah never mind. It is what I originally thought...

[kas]

Tried to comment on the article at Utest but seems they have possibly implemented Sorbs screwed up tactics as they tried to say that my short and simple post of me agreeing with the guy above me saying about the IP with only 1 previous occurrence but not for over a year now and can't delist as the site has errors .. was "spammy" What? I know what spam looks like and no way it was spammy! grrr....

So many people upset that they even have a website bumper sticker going around lately:



With tweets like this I can see they have valid points because I know that static IP's have been showing up on Sorbs as Dynamic (mine has which the IP has remained the same for over a year ... not sure how long it has been like this but yeah a long time and is not considered dynamic by any means!) and also single IP's are being added to Sorbs simply because a whole range is added to Sorbs at once. Seems what Sorbs says in interviews contradicts what is fact ...

Mistaken a static IP Range for Dynamic = FAIL. Due to this inconvenience, my server cannot connect to other MTA's that use SORBS Blacklist.
2 Jun 2010 by Bullmoose_Vx on Amplicate

my server ip fall into some range. So its not my server ip listed, im just caught into there ip range. what BS. Give them $50 and file a paypal dispute. only way to get delisted. I hate spam, but I also hate crooks and SORBS are 100% thieves.
24 Aug 2010 by User-107nvf on Amplicate

Wow seems Sorbs has been having people complain about it for awhile now. Guess I should count myself lucky that I had no problems til just the last month..

Attachments: