PMTU cisco ipsec problem

I have a cisco -- cisco tunnel using ipsec. It uses
path-mtu-discovery.

|-------------| |-------------- |--------------|
|----------|
| linux |----->| cisco router |------>| cisco router |----->
exchange |
|-------------| | ipsec | | ipsec |
|----------|
|--------------| |--------------|

Now the problem.

I have a linux RH9 server with postfix sending over the tunnel. It
fails on large messages to exechange server 192.168.100.25 at the
other end of the tunnel

ping -s 1472 -M do 192.168.100.25

From 192.168.1.233 icmp_seq=1 Frag needed and DF set (mtu = 1456)
ping: local error: Message too long, mtu=1456
ping: local error: Message too long, mtu=1456
ping: local error: Message too long, mtu=1456
ping: local error: Message too long, mtu=1456

so far so good

ping -s 1428 -M do 192.168.100.25

just hangs

tcpdump shows packets leaving but none returning

ping -s 1393 -M do 192.168.100.25

PING 192.168.100.25 (192.168.100.25) from 192.168.1.233 : 1393(1421)
bytes of data.
1401 bytes from 192.168.100.25: icmp_seq=1 ttl=125 time=281 ms
1401 bytes from 192.168.100.25: icmp_seq=2 ttl=125 time=281 ms

1393 is the largest packet size that works. What is going on I thought
a MTU of 1456 would allow a packetsize of 1428

Any ideas as to what is happening

Stephen Collier

On 2004-05-12, Stephen Collier <(E-Mail Removed)> wrote:
>
> Now the problem.
>
> I have a linux RH9 server with postfix sending over the tunnel. It
> fails on large messages to exechange server 192.168.100.25 at the
> other end of the tunnel
>
> ping -s 1472 -M do 192.168.100.25
>
> From 192.168.1.233 icmp_seq=1 Frag needed and DF set (mtu = 1456)
> ping: local error: Message too long, mtu=1456
> ping: local error: Message too long, mtu=1456
> ping: local error: Message too long, mtu=1456
> ping: local error: Message too long, mtu=1456
>
> so far so good
>
> ping -s 1428 -M do 192.168.100.25
>
> just hangs
>
well it would. By default ICMP traffic has (and should have really) the 'do
not fragment' (aka 'DF') flag set; this is what ping is warning you of, its
telling you its sending packets larger but probably chopping them.

> tcpdump shows packets leaving but none returning
>
> ping -s 1393 -M do 192.168.100.25
>
> PING 192.168.100.25 (192.168.100.25) from 192.168.1.233 : 1393(1421)
> bytes of data.
> 1401 bytes from 192.168.100.25: icmp_seq=1 ttl=125 time=281 ms
> 1401 bytes from 192.168.100.25: icmp_seq=2 ttl=125 time=281 ms
>
> 1393 is the largest packet size that works. What is going on I thought
> a MTU of 1456 would allow a packetsize of 1428
>
MTU is the maximum transmissible packet including all the headers. For TCP
traffic you are looking at a _payload_ of (MTU-40)==1416 (as IP has 20 bytes
and TCP adds a further 20 bytes of header). ICMP of course is different.

The problem is that you are forcing packets *above* the MTU to be send and
they are probably being chopped. When you tell ping '-s 1472' you are asking
for a _payload_ of 1472 bytes plus about the 28 bytes of header (1500). You
should find that '-s 1424' works fine. Do you see the bit in the brackets?

Regards

Alex

Sorry I didn't explain the problem well enough.

The MTU is being set at 1456 by PMTU discovery thus should allow icmp of
1428 through.
although it seems that it should be 1421 which is the value obtained from
the other end of the route. I suspect a broken router in the route but I
don't understand why the problem is asymetric. the other end on 2 different
paths have their PMTU working perfectly.

regards
Stephen
"Alexander Clouter" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> On 2004-05-12, Stephen Collier <sam123assammysdotcomdotau> wrote:
> >
> > Now the problem.
> >
> > I have a linux RH9 server with postfix sending over the tunnel. It
> > fails on large messages to exechange server 192.168.100.25 at the
> > other end of the tunnel
> >
> > ping -s 1472 -M do 192.168.100.25
> >
> > From 192.168.1.233 icmp_seq=1 Frag needed and DF set (mtu = 1456)
> > ping: local error: Message too long, mtu=1456
> > ping: local error: Message too long, mtu=1456
> > ping: local error: Message too long, mtu=1456
> > ping: local error: Message too long, mtu=1456
> >
> > so far so good
> >
> > ping -s 1428 -M do 192.168.100.25
> >
> > just hangs
> >
> well it would. By default ICMP traffic has (and should have really) the
'do
> not fragment' (aka 'DF') flag set; this is what ping is warning you of,
its
> telling you its sending packets larger but probably chopping them.
>
> > tcpdump shows packets leaving but none returning
> >
> > ping -s 1393 -M do 192.168.100.25
> >
> > PING 192.168.100.25 (192.168.100.25) from 192.168.1.233 : 1393(1421)
> > bytes of data.
> > 1401 bytes from 192.168.100.25: icmp_seq=1 ttl=125 time=281 ms
> > 1401 bytes from 192.168.100.25: icmp_seq=2 ttl=125 time=281 ms
> >
> > 1393 is the largest packet size that works. What is going on I thought
> > a MTU of 1456 would allow a packetsize of 1428
> >
> MTU is the maximum transmissible packet including all the headers. For
TCP
> traffic you are looking at a _payload_ of (MTU-40)==1416 (as IP has 20
bytes
> and TCP adds a further 20 bytes of header). ICMP of course is different.
>
> The problem is that you are forcing packets *above* the MTU to be send and
> they are probably being chopped. When you tell ping '-s 1472' you are
asking
> for a _payload_ of 1472 bytes plus about the 28 bytes of header (1500).
You
> should find that '-s 1424' works fine. Do you see the bit in the
brackets?
>
> Regards
>
> Alex