Incorrect TCP sequence numbers.

I have a tcpdump that shows a problem I'm having with the 2.4.20
kernel. I'm trying to open a ftp connection (server is 172.16.1.140)
from a client machine (vxWorks 5.5 kernel, 172.16.1.99 -- vxWorks uses
a BSD networking core). The Linux server is sending incorrect ACK
sequence numbers. If I read the RFC correctly, it should have sent
sequence number 2078917054 in its ACK, not 2078917104.

This doesn't happen every time. Perhaps 60-70% of the time, it works
properly. If I try a linux-linux ftp session, it works correctly
every time. The Linux server will always respond to a Linux client
with the correct sequence numbers. I compared the packet dumps, and
there are essentially no differences between the initial SYN packet of
a Linux client and the initial SYN packet of a vxWorks client (except
for some TCP options -- sackOK, etc).

Is there a known bug in the Linux kernel, or is the vxWorks client
just overreacting to the situation by sending a RST (should it just
ignore and SYN again?)

tcpdump (-nnxevvv -s 2000) is below.
Aaron


vxWorks client contacting a Linux ftp server, in kernel 2.4.20:

10:48:45.995029 0:30:60:80:6f:8a ff:ff:ff:ff:ff:ff 0806 60: arp
who-has 172.16.1.140 tell 172.16.1.99
0x0000 0001 0800 0604 0001 0030 6080 6f8a ac10
0x0010 0163 0000 0000 0000 ac10 018c 0000 0000
0x0020 0000 0000 0000 0000 0000 0000 0000
10:48:45.995057 0:b0:d0:7:1a:e6 0:30:60:80:6f:8a 0806 42: arp reply
172.16.1.140 is-at 0:b0:d0:7:1a:e6
0x0000 0001 0800 0604 0002 00b0 d007 1ae6 ac10
0x0010 018c 0030 6080 6f8a ac10 0163
10:48:51.495361 0:30:60:80:6f:8a 0:b0:d0:7:1a:e6 0800 74:
172.16.1.99.1024 > 172.16.1.140.21: S [tcp sum ok]
2078917053:2078917053(0) win 8192 <mss 1460,nop,wscale
0,nop,nop,timestamp 11 0> (ttl 64, id 1, len 60)
0x0000 4500 003c 0001 0000 4006 1fac ac10 0163
0x0010 ac10 018c 0400 0015 7be9 c1bd 0000 0000
0x0020 a002 2000 8e31 0000 0204 05b4 0103 0300
0x0030 0101 080a 0000 000b 0000 0000
10:48:51.495398 0:b0:d0:7:1a:e6 0:30:60:80:6f:8a 0800 66:
172.16.1.140.21 > 172.16.1.99.1024: . [tcp sum ok]
219158104:219158104(0) ack 2078917104 win 5792 <nop,nop,timestamp
52151 12> (DF) (ttl 64, id 54327, len 52)
0x0000 4500 0034 d437 4000 4006 0b7d ac10 018c
0x0010 ac10 0163 0015 0400 0d10 1658 7be9 c1f0
0x0020 8010 16a0 d3f2 0000 0101 080a 0000 cbb7
0x0030 0000 000c
10:48:51.495599 0:30:60:80:6f:8a 0:b0:d0:7:1a:e6 0800 60:
172.16.1.99.1024 > 172.16.1.140.21: R [tcp sum ok]
2078917104:2078917104(0) win 8192 (ttl 64, id 2, len 40)
0x0000 4500 0028 0002 0000 4006 1fbf ac10 0163
0x0010 ac10 018c 0400 0015 7be9 c1f0 0000 0000
0x0020 5004 2000 f2e1 0000 0204 05b4 0103

The client and server will then both perform no actions for another
20-25 seconds, at which time the client will try connecting again (and
usually succeed).

Aaron Graham <(E-Mail Removed)> wrote:

> I have a tcpdump that shows a problem I'm having with the 2.4.20
> kernel. I'm trying to open a ftp connection (server is 172.16.1.140)
> from a client machine (vxWorks 5.5 kernel, 172.16.1.99 -- vxWorks uses
> a BSD networking core). The Linux server is sending incorrect ACK
> sequence numbers. If I read the RFC correctly, it should have sent
> sequence number 2078917054 in its ACK, not 2078917104.

It should have sent a SYN-ACK with SEQ 2078917054.

[...]
> tcpdump (-nnxevvv -s 2000) is below.
[...]
> vxWorks client contacting a Linux ftp server, in kernel 2.4.20:

Trace is slightly edited by me:

> 10:48:45.995029 arp who-has 172.16.1.140 tell 172.16.1.99
> 10:48:45.995057 arp reply 172.16.1.140 is-at 0:b0:d0:7:1a:e6
> 10:48:51.495361 172.16.1.99.1024 > 172.16.1.140.21: S [tcp sum ok] \
> 2078917053:2078917053(0) win 8192 <mss 1460,nop,\
> wscale 0,nop,nop,timestamp 11 0> (ttl 64, id 1, len 60)
> 10:48:51.495398 172.16.1.140.21 > 172.16.1.99.1024: . [tcp sum ok] \
> 219158104:219158104(0) ack 2078917104 win 5792 <nop,nop,\
> timestamp 52151 12> (DF) (ttl 64, id 54327, len 52)
> 10:48:51.495599 172.16.1.99.1024 > 172.16.1.140.21: R [tcp sum ok] \
> 2078917104:2078917104(0) win 8192 (ttl 64, id 2, len 40)

This is no proper 3-Way Handshake, at least the second segment
with SYN-ACK and the third segment with ACK is missing in the
trace. I suppose that you are also missing one data segment with
49 bytes TCP payload.

However what speaks against lost segments in the trace is, that
it looks (due to the IP id field) like your client machine has
no more than the two segments sent. Assuming that the BSD core
monotonously increments the IP ID field by one for each send packet.

Have you taken the trace on the Linux Box (FTP server) or
on another box on the LAN? Was the box on which you have
taken the trace under heavy load?

Are you implementing your own FTP server via raw sockets on
the linux box?

Check error counters listed by ifconfig.

Do an iptables-save and check whether you have iptables rules
in place. If yes, get rid of them for a short test.

Ciao, Horst
--
»When pings go wrong (It hurts me too)« E.Clapton/E.James/P.Tscharn