Spam filtering (again)

RE: Mailwasher

In case its of any use to any of you, i`ve put a slightly annotated set
of my filters.txt online that you can copy and paste into your own if
you so desire...

http://www.phoenixbbs.dsl.pipex.com/filterlist.html

I`ve now had to manually mark 5 spam mails for deletion over the last 4
days that weren`t picked up between these filters and a short(ish) list
of blocked country codes in the blacklist.txt (in fact, i`ll paste them
below) - I used to use an 80k+ list of blocked domains, but this seems to
perform almost as well - just doesn`t mark them as blacklisted.

I check 8 accounts and at least 4 get spammed heavily, so I don`t think
this is a bad result overall :-p - and for the very observant, you will
probably be able to guess at least 6 of my real email addresses :-}

---
[Blacklisted emails]

*@*.adelphia.net 0
*@*.ar 37999
*@*.as 37898
*@*.at 38006
*@*.be 38007
*@*.bg 38004
*@*.biz 38007
*@*.bm 37876
*@*.bn 37817
*@*.bo 37924
*@*.br 38004
*@*.bw 0
*@*.by 37980
*@*.bz 37822
*@*.ca 38008
*@*.cc 38006
*@*.ch 38007
*@*.cl 38003
*@*.cn 38006
*@*.co 37988
*@*.co.jp 37778
*@*.co.za 37817
*@*.com.tw 37992
*@*.cu 37802
*@*.cy 37895
*@*.cz 38006
*@*.de 38008
*@*.dk 38007
*@*.do 37863
*@*.ec 38006
*@*.edu.tw 37957
*@*.ee 37989
*@*.eg 37984
*@*.es 38005
*@*.ew01.* 37913
*@*.fi 38007
*@*.fm 37999
*@*.fr 38007
*@*.gl 37976
*@*.gr 38003
*@*.gt 37939
*@*.hm 37949
*@*.hn 37982
*@*.hr 37984
*@*.hu 38003
*@*.id 38007
*@*.ie 38007
*@*.il 38002
*@*.in 38000
*@*.info 38007
*@*.int 37993
*@*.is 38002
*@*.it 38005
*@*.jp 38007
*@*.kr 38005
*@*.kw 37901
*@*.kz 37881
*@*.lb 37961
*@*.lk 37898
*@*.lt 37975
*@*.lu 37995
*@*.lv 37984
*@*.md 37961
*@*.mil 37819
*@*.mk 37913
*@*.mn 38006
*@*.ms.com 37923
*@*.ms.net 37923
*@*.msn.* 37927
*@*.mt 37971
*@*.mx 38004
*@*.my 37963
*@*.ne 37905
*@*.ne.jp 38002
*@*.nl 38007
*@*.no 38004
*@*.nu 38008
*@*.nz 38001
*@*.om 38006
*@*.open-wholesale.com 37987
*@*.pe 38001
*@*.pf 37956
*@*.ph 37891
*@*.pk 38005
*@*.pl 38004
*@*.pt 38007
*@*.ro 38007
*@*.rr.com 38003
*@*.ru 38006
*@*.sc 37807
*@*.se 38006
*@*.sg 38005
*@*.si 38003
*@*.sk 37995
*@*.sp 37881
*@*.ss01.* 37928
*@*.su 38002
*@*.sv 37954
*@*.tekmailer.com 38001
*@*.th 37977
*@*.tk 0
*@*.tn 38006
*@*.to 38006
*@*.tr 38005
*@*.trbrgns.* 0
*@*.turtlemailz* 37985
*@*.ua 38007
*@*.us 37988
*@*.uy 37980
*@*.uz 37966
*@*.ve 37899
*@*.vg 37948
*@*.vn 37961
*@*.vu 37883
*@*.web1000.com 37864
*@*.ws 37970
*@*.yu 37937
*@*.za 38003
*@*.zw 37895
*@*.zzn.* 37987
*@*_ms.com 37922
*@*_ms.net 37923
*@*_msn.* 37922
*@*americalenders.net 37849
*@*americanunitedlenders.net 0
*@*bestusrx* 37889
*@*bluerocketonline* 37924
*@*bluerocketonline.com 38008
*@*bobalou2.com 37956
*@*brightermail.com 37960
*@*budgetgaming.net 37966
*@*d59072* 0
*@*deals-and-steals* 37938
*@*easymeds* 37960
*@*email-info.net 37833
*@*equityloanhelp.com 37897
*@*homelender* 37892
*@*malibumailz* 37882
*@*microsoft* 37990
*@*msdn* 37968
*@*msnbc* 37902
*@*ozemail.com.au 37898
*@*savings1friend* 37893
*@*savingsfriend* 37929
*@*seed.net.tw 37785
*@*seeder.net.tw 0
*@*specialbuys* 37930
*@*trbrgns* 38007
*@*true-bargains* 37948
*@*wbbrgns.com 38008
---

--
Please add "[newsgroup]" in the subject of any personal replies via email
* old email address "btiruseless" abandoned due to worm-generated spam *
--- My new email address has "ngspamtrap" & @btinternet.com in it ;-) ---

Colin Wilson wrote:

> RE: Mailwasher
>

Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
it still cost me time constantly updating the lists. I moved my a
Baysian based filter, and now spam is gone for good!! I have well over
99% accuracy. I highly recommend POPFile (popfile.sf.net)

> Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
> it still cost me time constantly updating the lists. I moved my a
> Baysian based filter, and now spam is gone for good!! I have well over
> 99% accuracy. I highly recommend POPFile (popfile.sf.net)

Yeah, it looks ok, but you have to download the spam for it to filter -
and with the proliferation of viruses, I prefer to get shut of the crap
at server level. How newer AV programs don`t go apeshit at downloaded
viral attachments in spam prior to the bayesian filtering and subsequent
deletion is beyond me. My old version of AVP clamps the machine
immediately one appears even as a temp file.

I was spending a lot of time updating the lists too, until I tried
mailwasher on a friends old clanking machine. It was impossibly
horribly slow.

I deleted almost all the domains in my blacklist (80k+ of them) and now
just block country codes in the blacklist (.uk .ie .ru etc) and have the
filters as shown on the webpage I pointed at.

So far in 4 days i`ve had to manually mark 5 spams, which when
considering I use 8 accounts, of which 4 accounts are industrial strength
spam magnets, I don`t think that`s a bad track record :-}

--
Please add "[newsgroup]" in the subject of any personal replies via email
* old email address "btiruseless" abandoned due to worm-generated spam *
--- My new email address has "ngspamtrap" & @btinternet.com in it ;-) ---

On Thu, 22 Jan 2004 22:05:20 -0000, Colin Wilson <(E-Mail Removed)> wrote:

>> Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
>> it still cost me time constantly updating the lists. I moved my a
>> Baysian based filter, and now spam is gone for good!! I have well over
>> 99% accuracy. I highly recommend POPFile (popfile.sf.net)
>
>Yeah, it looks ok, but you have to download the spam for it to filter -
>and with the proliferation of viruses, I prefer to get shut of the crap
>at server level. How newer AV programs don`t go apeshit at downloaded
>viral attachments in spam prior to the bayesian filtering and subsequent
>deletion is beyond me. My old version of AVP clamps the machine
>immediately one appears even as a temp file.
>
>I was spending a lot of time updating the lists too, until I tried
>mailwasher on a friends old clanking machine. It was impossibly
>horribly slow.
>
>I deleted almost all the domains in my blacklist (80k+ of them) and now
>just block country codes in the blacklist (.uk .ie .ru etc) and have the
>filters as shown on the webpage I pointed at.
>
>So far in 4 days i`ve had to manually mark 5 spams, which when
>considering I use 8 accounts, of which 4 accounts are industrial strength
>spam magnets, I don`t think that`s a bad track record :-}

It's easier, if expensive, to use a spam deletion service, but even
that isn't perfect.

If spam comes in between the times they scan you mailbox and you access
it before their next scan you still get the spam.

Nothing's perfect it seems!

--
Jock.

Colin Wilson <(E-Mail Removed)> wrote:

> > Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
> > it still cost me time constantly updating the lists. I moved my a
> > Baysian based filter, and now spam is gone for good!! I have well over
> > 99% accuracy. I highly recommend POPFile (popfile.sf.net)
>
> Yeah, it looks ok, but you have to download the spam for it to filter -
> and with the proliferation of viruses, I prefer to get shut of the crap
> at server level. How newer AV programs don`t go apeshit at downloaded
> viral attachments in spam prior to the bayesian filtering and subsequent
> deletion is beyond me. My old version of AVP clamps the machine
> immediately one appears even as a temp file.

I use POPfile as well (and also highly recommend it!) in conjunction
with a server-side rule (using mailfilter) which blocks anything over a
certain size from an unknown sender. Catches most of the crap. Of
course, that may not fit with the way you use email.

--
TimH
pull tooth to reply by email

In message <ivUPb.26724$(E-Mail Removed)>, Zapp Brannigan
<(E-Mail Removed)> writes
>Colin Wilson wrote:
>
>> RE: Mailwasher
>>
>
>Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
>it still cost me time constantly updating the lists. I moved my a
>Baysian based filter, and now spam is gone for good!! I have well over
>99% accuracy. I highly recommend POPFile (popfile.sf.net)
>
Or K9 for Windows users.
--
Julian Knight,
Sheffield, South Yorkshire, United Kingdom.
Security, Directory, Messaging, Network & PC Consultant
http://www.knightnet.org.uk/

In message <(E-Mail Removed)> , Colin
Wilson <(E-Mail Removed)> writes
>> Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
>> it still cost me time constantly updating the lists. I moved my a
>> Baysian based filter, and now spam is gone for good!! I have well over
>> 99% accuracy. I highly recommend POPFile (popfile.sf.net)
>
>Yeah, it looks ok, but you have to download the spam for it to filter -
>and with the proliferation of viruses, I prefer to get shut of the crap
>at server level. How newer AV programs don`t go apeshit at downloaded
>viral attachments in spam prior to the bayesian filtering and subsequent
>deletion is beyond me. My old version of AVP clamps the machine
>immediately one appears even as a temp file.
>
>I was spending a lot of time updating the lists too, until I tried
>mailwasher on a friends old clanking machine. It was impossibly
>horribly slow.
>
>I deleted almost all the domains in my blacklist (80k+ of them) and now
>just block country codes in the blacklist (.uk .ie .ru etc) and have the
>filters as shown on the webpage I pointed at.
>
>So far in 4 days i`ve had to manually mark 5 spams, which when
>considering I use 8 accounts, of which 4 accounts are industrial strength
>spam magnets, I don`t think that`s a bad track record :-}
>
K9 is written from scratch in C (or C++?) and has very low overheads.
Even in the first few days, it is pretty accurate (80+%) and runs at
98+% for me now. (Based on an average of 50 emails per day of which >70%
are spam).
--
Julian Knight,
Sheffield, South Yorkshire, United Kingdom.
Security, Directory, Messaging, Network & PC Consultant
http://www.knightnet.org.uk/

Jock <(E-Mail Removed)> wrote:

>If spam comes in between the times they scan you mailbox and you access
>it before their next scan you still get the spam.
>
>Nothing's perfect it seems!

I find mailwasher to be pretty good, better than 95%, in its
identification of spam using various built-in rules and by checking
against known spam sources. But it's not 100% - what could be when
today's spam uses a different subject etc for every recipient.

An *ISP* could do a better job because when their system sees 10k
emails from the same source to 10k of their customers, it is obviously
spam. But there is a limit to what any individual email recipient will
be able to do. You can start with viagra, v1agra, penis, pen1s,
septic, porn, sex, etc in the subject header, and end up with a
dictionary analysis rejecting every email with more than a certain %
of mis-spelt words (a lot of spam uses randomly generated text within
it). But it will never be 100%.


Peter.
--
Return address is invalid to help stop junk mail.
E-mail replies to (E-Mail Removed) but remove the X and the Y.
Please do NOT copy usenet posts to email - it is NOT necessary.

On Mon, 26 Jan 2004 12:45:22 +0000, Julian Knight wrote:

> In message <(E-Mail Removed)> , Colin
> Wilson <(E-Mail Removed)> writes
>>> Hmm, I tried blacklisting programs like Mail washer, and got fed up, as
>>> it still cost me time constantly updating the lists. I moved my a
>>> Baysian based filter, and now spam is gone for good!! I have well over
>>> 99% accuracy. I highly recommend POPFile (popfile.sf.net)
>>
>>Yeah, it looks ok, but you have to download the spam for it to filter -
>>and with the proliferation of viruses, I prefer to get shut of the crap
>>at server level. How newer AV programs don`t go apeshit at downloaded
>>viral attachments in spam prior to the bayesian filtering and subsequent
>>deletion is beyond me. My old version of AVP clamps the machine
>>immediately one appears even as a temp file.
>>
>>I was spending a lot of time updating the lists too, until I tried
>>mailwasher on a friends old clanking machine. It was impossibly
>>horribly slow.
>>
>>I deleted almost all the domains in my blacklist (80k+ of them) and now
>>just block country codes in the blacklist (.uk .ie .ru etc) and have the
>>filters as shown on the webpage I pointed at.
>>
>>So far in 4 days i`ve had to manually mark 5 spams, which when
>>considering I use 8 accounts, of which 4 accounts are industrial strength
>>spam magnets, I don`t think that`s a bad track record :-}
>>
> K9 is written from scratch in C (or C++?) and has very low overheads.
> Even in the first few days, it is pretty accurate (80+%) and runs at
> 98+% for me now. (Based on an average of 50 emails per day of which >70%
> are spam).

You might also like to take a look at the add-on Filter and Blacklist for
K9 here www.edcottrell.com/k9.cfm

Regards

Bll

Peter <(E-Mail Removed)> wrote:

> But there is a limit to what any individual email recipient will
> be able to do. You can start with viagra, v1agra, penis, pen1s,
> septic, porn, sex, etc in the subject header, and end up with a
> dictionary analysis rejecting every email with more than a certain %
> of mis-spelt words (a lot of spam uses randomly generated text within
> it). But it will never be 100%.

Well, that's where Bayesian filtering scores (no pun intended). It
adapts as the spammers do, and you don't have to waste time tweaking
rules manually. The one I'm using (POPfile) is currently scoring 99.57%
accuracy.

--
TimH
pull tooth to reply by email