ip_conntrack not full but dropping

Hi,

we have a problem with our router. It says:

ip_conntrack: table full, dropping packet

but in ip_conntrack there are only 2000 or less entries.

The value of ip_conntrack_max is 8192. When we change it to a higher value
e.g. 16384 the kernel stops dropping. But after a few days it will drop
again.

Our system is a PIII 1.2 GHz with 128MB ram and we are doing nat for 200
clients. The version of the kernel is 2.4.21.

Any ideas?

With regards,

Patrick

Patrick wrote:
> Hi,
>
> we have a problem with our router. It says:
>
> ip_conntrack: table full, dropping packet
>
> but in ip_conntrack there are only 2000 or less entries.
>
> The value of ip_conntrack_max is 8192. When we change it to a higher
value
> e.g. 16384 the kernel stops dropping. But after a few days it will
drop
> again.
>
> Our system is a PIII 1.2 GHz with 128MB ram and we are doing nat for
200
> clients. The version of the kernel is 2.4.21.
>
> Any ideas?

Be warned, these are the guesses of someone who has not had to support
connection tracking for 200+ hosts.

First, if you take a look at the source code in,
..../net/ipv4/netfilter/ip_conntrack_core.c (~ line 500) you will see
that not every available slot has to be filled before getting this
message. It's been several years since I've tracked down the various
constants and function calls to figure out "exactly" what's going on,
but it _does_ depend on just how your firewall/iptable rules are set
up, not simply whether every slot is used.

You've already tried reconfiguring the connection tracking table size,
but look here for some better idea on how/what you can adjust it (ie.,
hash table and list sizes):
http://www.wallfire.org/misc/netfilt...track_perf.txt

You probably need to monitor just what kind of connections are being
made that generate this message. One or more particular IPs?
Ports/services? It may be that simply enlarging the connection table
is _not_ the best or only remedy you need.

Filling the connection table _is_ a potential source of DoS (intended
or not), so take a close look at your filtering policies. Googling
Groups shows that conntrack table overflow is more prevalent in EU.
Are there some connections you should not allow?

In any case, you'll be well served to monitor/classify the IP traffic,
then devise a multi-layered plan. IIRC, some of the "dedicated"
firewall distros make some kernel source changes to accommodate more
"optimal" performance for connection tracking. It can be memory
intensive.

Wish I had a "quick fix", but as with most networking problems, the
"fix" depends on specific conditions.

good luck,
prg

Patrick <(E-Mail Removed)> wrote:

> we have a problem with our router. It says:
>
> ip_conntrack: table full, dropping packet
>
> but in ip_conntrack there are only 2000 or less entries.
>
> The value of ip_conntrack_max is 8192. When we change it to a higher
> value e.g. 16384 the kernel stops dropping. But after a few days it will
> drop again.
>
> Our system is a PIII 1.2 GHz with 128MB ram and we are doing nat for 200
> clients. The version of the kernel is 2.4.21.

Could it be that the connection tracking doesn't see
the end of the connections and so they remain in the
connection track table until they expire? The huge
number of connections you see, would be connections
not active anymore.

Does P2P filesharing traffic hit your router?

It might help if you post your iptables rules.

Ciao, Horst
--
»When pings go wrong (It hurts me too)« E.Clapton/E.James/P.Tscharn

Horst Knobloch wrote:

>
> Could it be that the connection tracking doesn't see
> the end of the connections and so they remain in the
> connection track table until they expire? The huge
> number of connections you see, would be connections
> not active anymore.

Yes, that's true.
But I don't understand, why the kernel starts dropping though there are less
entries as defined in ip_conntrack_max. ( I think I will have to take a
look at the kernel-sources as prg wrote.)

>
> Does P2P filesharing traffic hit your router?

Yes, because only students reside in our dormitory. We applied the ipp2p
patch from http://rnvs.informatik.uni-leipzig.de/ipp2p/ to filter
p2p-traffic. This helped us much in the past.

But perhaps this could be a problem. Our iptables rule for this is:

iptables -A FORWARD -p tcp -m ipp2p --ipp2p -j DROP

Can it be, that the DROP keeps an established connection in the table?
I changed it to REJECT --reject-with tcp-reject. Perhaps it will help.
The rest of our ruleset is not very restrictive. We only drop connections on
well known p2p ports.

>
> It might help if you post your iptables rules.
>
> Ciao, Horst

Patrick <(E-Mail Removed)> wrote:

> Horst Knobloch wrote:
>>
>> Could it be that the connection tracking doesn't see
>> the end of the connections and so they remain in the
>> connection track table until they expire? The huge
>> number of connections you see, would be connections
>> not active anymore.
>
> Yes, that's true.
> But I don't understand, why the kernel starts dropping though there are
> less entries as defined in ip_conntrack_max. ( I think I will have to
> take a look at the kernel-sources as prg wrote.)

This mismatch is a different angle of your problem
but you should not forget why you have the huge number
of (inactive) connections in the first place.

Does your ISP force your Internet connection down every
24 hours and are your stale connections left-overs from
the last dial-up? If yes, you could remove and reload
the connection tracking module in ip-down. This should
also clear all connections in the tracking table. Of
course this is a kludge but worth a try.


>> Does P2P filesharing traffic hit your router?
>
> Yes, because only students reside in our dormitory. We applied the ipp2p
> patch from http://rnvs.informatik.uni-leipzig.de/ipp2p/ to filter
> p2p-traffic. This helped us much in the past.
>
> But perhaps this could be a problem. Our iptables rule for this is:
>
> iptables -A FORWARD -p tcp -m ipp2p --ipp2p -j DROP
>
> Can it be, that the DROP keeps an established connection in the table?
> I changed it to REJECT --reject-with tcp-reject. Perhaps it will help.

I don't know how the ipp2p module works, so I can't say.
For connection tracking in general I propose to read:
http://www.sns.ias.edu/~jns/security...conntrack.html
http://iptables-tutorial.frozentux.n...l#STATEMACHINE


> The rest of our ruleset is not very restrictive. We only drop connections
> on well known p2p ports.

If I should guess, I would say that the students have
found ways to circumvent your p2p-blocking ruleset and
share stuff like hell. ;-)


Ciao, Horst
--
»When pings go wrong (It hurts me too)« E.Clapton/E.James/P.Tscharn