How to stop spammers bringing our server down?

Hi

I hope somebody can give me a few pointers which I can pass on.

We are running a www/email server on FreeBSD on a fixed IP 448/8192
ADSL link.

Particularly since upgrading the line from 512k to 8192k, the server
has been increasingly hit.

It gets flood attacks, which create multiple sendmail connections and
exhaust the preconfigured sendmail connection limit, so effectively
the email server is lost.

We can set up a "valid user" list in sendmail (currently it is
implemented further downstream, using a Python email filtering script,
so all emails are currently being received whole) but that still won't
stop the attacker exhausting the max # of sendmail connections.

I have a friend who knows a lot more than I do (I am a
hardware/software developer but know nothing about unix) doing the
server admin part-time on Monday mornings but he is not aware of any
way to limit the sendmail max connections # in such a way that some
IPs (i.e. me and a few others) can still access the machine.

The pop box (implemented with something called "popper", I believe) is
also being blocked however, so the problem may be elsewhere. There is
a firewall running on the server and I wonder whether the firewall is
blocking same or a lower # of connections to what sendmail is
blocking, so reaching the sendmail limit blocks the whole machine? Is
that possible?

The # of spam connections is about 20k per day... The actual data
volume is "only" a few hundred MB per month so this is not a problem.
The server is a 3GHz PC with 1GB RAM.

The server has a Draytek 2900 router in front of it, which contains
some facilities for blocking traffic, but nothing that would obviously
help. I have enabled some of the countermeasures e.g. Smurf attack,
teardrop attack but I don't think it made any difference. The router
has only the required ports forwarded through NAT, and we know about
the "Draytek port 443 bug" which is fixed by forwarding 443 to an
unused IP on the other side.

Until a week ago, 443 was being forwarded to the server IP instead
where the firewall was blocking it, and this was bringing the server
down. This was fixed and fixed the problem for a few days but the
attacker has moved to something else...

I would appreciate any tips for things to try.

Peter wrote:
> Hi
>
> I hope somebody can give me a few pointers which I can pass on.
>
> We are running a www/email server on FreeBSD on a fixed IP 448/8192
> ADSL link.
>
> Particularly since upgrading the line from 512k to 8192k, the server
> has been increasingly hit.
>
> It gets flood attacks, which create multiple sendmail connections and
> exhaust the preconfigured sendmail connection limit, so effectively
> the email server is lost.
....
> I would appreciate any tips for things to try.

You could try limiting at firewall level. Probably this solution is not
practical for a high-volume system; but at home, I use pf, and have a
perl script monitoring the firewall log. A pf table is either empty, or
is loaded with '0/0' (ie catch-all) by this script depending on the port
25 connection history. Said table is used to block incoming smtp
connections. There's a prior pf rule to allow "friendly" connections
irrespective. Stops floods dead, even if spread across a number of
machines. Downside is that you'll delay mail from unlisted but
nevertheless kosher machines, but you can't have it all ways :-)


--
Mike Scott (unet <at> scottsonline.org.uk)
Harlow Essex England

I hesitate to speak because I'm neither Unix nor mail expert, but
surely it should be possible to set up firewall rules to prevent this?

Particularly, if the attacks are coming from known IPs, then it should
relatively simple to prevent connections of any sort from those IPs?

On Sat, 03 Nov 2007 11:00:39 +0000, Peter
<occassionally-(E-Mail Removed)> wrote:

>There is
> a firewall running on the server

Java Jive <(E-Mail Removed)> wrote:

>I hesitate to speak because I'm neither Unix nor mail expert, but
>surely it should be possible to set up firewall rules to prevent this?
>
>Particularly, if the attacks are coming from known IPs, then it should
>relatively simple to prevent connections of any sort from those IPs?

Yes, if the IPs were constant I could just block them in the router;
easy.

They vary though. Today, I could block 90% of the traffic by blocking
one IP. Tomorrow, it will be a different one.

The CPU loading on the server is around 97% due to the python scripts
running the check against a whitelist, keywords, etc.

Peter wrote:
> Hi
>
> I hope somebody can give me a few pointers which I can pass on.
>
> We are running a www/email server on FreeBSD on a fixed IP 448/8192
> ADSL link.
>
> Particularly since upgrading the line from 512k to 8192k, the server
> has been increasingly hit.
>
> It gets flood attacks, which create multiple sendmail connections and
> exhaust the preconfigured sendmail connection limit, so effectively
> the email server is lost.................


> I would appreciate any tips for things to try.


How about a different approach - move your email onto a hosting company who
already manage mail servers, and thus avoid the time and expense of solving
the problem with scripts, firewalls, running a server, electricity, etc.. ?

I have three different mail domains, two for hobbies, one for a small
company. All running on various hosting firm's machines. We don't seem to
suffer serious downtime from the hosting firms, nor spam problems. Costs are
a few tens of pounds per year (actually £25-£50 per year, but that includes
website hosting).

All of the hosts offer POP3 connection. One offers IMAP-4, though we decided
after trying that we don't really need it. One offers the ability to fork
inbound email; so important business incoming is sent to both the main
mailboxes (accessed through POP3), and also duplicate copies into a HotMail
(free) account. Thus if we loose our local mail files, the backups fail,
and loose access to our ISP, we still have access to the mail archive on
HotMail !



regards,


- Nigel


--
Nigel Cliffe,
Webmaster at http://www.2mm.org.uk/

"Nigel Cliffe" <(E-Mail Removed)> wrote

>I have three different mail domains, two for hobbies, one for a small
>company. All running on various hosting firm's machines. We don't seem to
>suffer serious downtime from the hosting firms, nor spam problems. Costs are
>a few tens of pounds per year (actually £25-£50 per year, but that includes
>website hosting).
>
>All of the hosts offer POP3 connection. One offers IMAP-4, though we decided
>after trying that we don't really need it. One offers the ability to fork
>inbound email; so important business incoming is sent to both the main
>mailboxes (accessed through POP3), and also duplicate copies into a HotMail
>(free) account. Thus if we loose our local mail files, the backups fail,
>and loose access to our ISP, we still have access to the mail archive on
>HotMail !

I didn't want to open this can of worms here again but the short
reason is that ISPs don't offer what we need.

I have a small business which for various reasons (probably as a
result of having a domain name, plus having in past years exposed the
mailto: links openly on the company website) gets about 20k spams per
day.

Among this will be 10-50 emails/day from known contacts (some of whom
are occassionally on spamcop...) and perhaps 1-5 emails/day from
completely new contacts.

Currently, all ISPs manage their spam problem with a mixture of

1 - spamcop and similar IP blacklists
2 - keyword searches, bayesian filters, etc
3 - in some cases, by looking at patterns of incoming emails

I know (from server stats) that 1 removes about 75% of spam but also
drops a few % of real good emails.

2 has been the #1 defence for years but no longer works because
much/most spam is plain English with a GIF attached. Well, you can
make it work 99.x% but it will dump a lot of good emails. THIS is why
a lot of emails simply vanish. OK for private use perhaps but not a
business.

3 should work very well.

I can't take the risk of dropping say 1/4 of emails from previously
unknown contacts. So, we run our own mail server. On that, we filter
email in the following order

- allow anybody on a whitelist
- allow any emails containing any of a list of keywords (company
product names, etc)
- allow any emails addressed to specific usernames
- drop all emails to invalid usernames (this drops ~ 98%)
- challenge the small remainder with a human readable challenge which
requests a REPLY to the email

The whitelist is the accumulated result of all known contacts, plus
anybody who WE write to goes on it too. Plus, existing customers are
on there as a whole domain so any employee can contact us without
hassle (although I would hesitate putting *@ibm.com on there ).
Plus, any responses to the challenges go on there too.

I have enquired to some ISPs but none offer a whitelist + multiple
keywords kind of thing.

The www server doesn't matter - anybody can host a website. But one
day we will move to online shopping and there having the server in the
office has certain advantages.

In the past we used to get dictionary attacks against the system but
these are now less common.

On Sat, 3 Nov 2007 15:05:48 UTC, occassionally-(E-Mail Removed)
wrote:

>
> Java Jive <(E-Mail Removed)> wrote:
>
> >I hesitate to speak because I'm neither Unix nor mail expert, but
> >surely it should be possible to set up firewall rules to prevent this?
> >
> >Particularly, if the attacks are coming from known IPs, then it should
> >relatively simple to prevent connections of any sort from those IPs?
>
> Yes, if the IPs were constant I could just block them in the router;
> easy.
>
> They vary though. Today, I could block 90% of the traffic by blocking
> one IP. Tomorrow, it will be a different one.

You have to keep them up to date, but chances are they are already
blacklisted (or on a normal mail PBL).

Can you set up sendmail to reject IPs on blacklists? That might be a lot
more efficient. I know that in Postfix it's effectively just one line in
the config file, but God knows what that is in sendmail...
--
[ 7'ism - a condition by which the sufferer experiences an inability
to give concise answers, express reasoned argument or opinion.
Usually accompanied by silly noises and gestures - incurable, early
euthanasia recommended. ]

On Sat, 3 Nov 2007 16:32:22 UTC, Peter
<occassionally-(E-Mail Removed)> wrote:

> - challenge the small remainder with a human readable challenge which
> requests a REPLY to the email

OK until I got to this part. What it the sender has a similar system?

Generally, if I get a challenge like that I simply give up.

--
[ 7'ism - a condition by which the sufferer experiences an inability
to give concise answers, express reasoned argument or opinion.
Usually accompanied by silly noises and gestures - incurable, early
euthanasia recommended. ]

"Peter" <occassionally-(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hi
>
> I hope somebody can give me a few pointers which I can pass on.
>
> We are running a www/email server on FreeBSD on a fixed IP 448/8192
> ADSL link.
>
> Particularly since upgrading the line from 512k to 8192k, the server
> has been increasingly hit.

[snip]

The problem you describe is that the amount of incoming email overwhelms the
server.

Having read some of the responses, I see that any form of limiting at the
ISP would not meet your requirements. So to prevent the server from being
overwhelmed you will have to reduce the rate of arrival of email traffic.
Can you do this in the router?

Alternatively you could increase the power of the server so it can cope with
the workload delivered by the 8192kbits/sec ADSL service.

Is there a possibility that you are being explicitly targeted? Are you
running a business which might attract guerilla action?

--
Graham J

On Sat, 03 Nov 2007 11:00:39 +0000, Peter
<occassionally-(E-Mail Removed)> wrote:


>We are running a www/email server on FreeBSD on a fixed IP 448/8192
>ADSL link.

assuming a recent version of FreeBSD.

Implement empty ACK Prioritisation on the 448 k reverse path.
Use Dummynet or ALTQ to shape email traffic.
Use PF to rate limit inbound port 25 traffic.
Implement Greylisting.
Use PF's Spamd to catch and tarpit spammers.



greg
--
?¡aah, los gringos otra vez!?