Userspace packet queuing with libipq: ip_conntrack does not defragment?

Hi all,

I'm using libipq to pass certain packets to my userspace application
on Fedora 6 / Kernel 2.6.21.1 and ipTables 1.3.5.

I do:
modprobe iptable_filter
modprobe ip_queue
modprobe ip_conntrack
iptables -A INPUT -p tcp -j QUEUE

Works fine. However, since ip_conntrack is loaded I would expect that
the packets are defragmented before they are passed to my userspace
appliation (as indicated here for example:
http://lists.netfilter.org/pipermail...y/034006.html).
This does not seem the case, i.e., the maximum size of the packets
which I get (through ->data_len) is 1500 bits.

The len parameter of the ipq_read method is well over 1500, as is the
buffer size.

Any suggestions what I'm doing wrong?

Many thanks,
Michael

ps: I also tried to use "OUTPUT" in my rule since I read somewhere
that connection tracking only works in OUTPUT and PREROUTING: Same
result - maximum packet size is 1500, i.e., no defragmentation :-(

Hi again,

I would like to use an example to clarify what I want to achieve:

Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
What I see is that each picture of the video is send as a single TCP/
IP packet (?) - usually with a size of around 10 KB. This TCP/IP
packet is fragmented according to the MTU and therefore I see the
following fragments on the wire:
Fragment 1: <Ethernet header><IP header><TCP header>application data
Fragment 2: <Ethernet header><IP header>application data
Fragment 3: <Ethernet header><IP header>application data
....

On my router/firewall I would like to inspect the video picture as a
whole. In my current (windows) application I have to reassemble the
fragments manually into the original TCP/IP packet. I hoped that
netfilter would make my life easier by performing the reassembling for
me (and potentially also the recalculation of the checksum(s) if the
video picture is changed).

Is there support in netfilter for this kind of stuff?

Many thanks again for your help,
Michael

Hello,

I do not know about packet queuing and will reply only about connection
tracking and packet reassembly.

Daneel a écrit :
>
> I'm using libipq to pass certain packets to my userspace application
> on Fedora 6 / Kernel 2.6.21.1 and ipTables 1.3.5.
>
> I do:
> modprobe iptable_filter
> modprobe ip_queue
> modprobe ip_conntrack
> iptables -A INPUT -p tcp -j QUEUE
>
> Works fine. However, since ip_conntrack is loaded I would expect that
> the packets are defragmented before they are passed to my userspace
> appliation

Yes, but actually you do not need connection tracking for that.
Reassembly occurs between the input routing decision and the INPUT
chains even without the connection tracking. So the INPUT chains always
see complete datagrams.

> (as indicated here for example:
> http://lists.netfilter.org/pipermail...y/034006.html).

Err.. your URL is incomplete.

> This does not seem the case, i.e., the maximum size of the packets
> which I get (through ->data_len) is 1500 bits.

Did you send fragmented datagrams with a size > 1500 bytes ?

> ps: I also tried to use "OUTPUT" in my rule since I read somewhere
> that connection tracking only works in OUTPUT and PREROUTING: Same
> result - maximum packet size is 1500, i.e., no defragmentation :-(

Same question, did you send datagrams with a size > 1500 bytes ?

Notes :
1) Connection tracking does not *work* in OUTPUT and PREROUTING ; it
*takes places* there. Actually it is the packet classification, which is
one part of the connection tracking process, which occurs before the
PREROUTING and OUTPUT chains of the 'mangle' table (but after the
PREROUTING and OUTPUT chains of the 'raw' table). Fragment reassembly is
required for proper connection tracking operation, so it is performed
just before packet classification.

2) In recent 2.6 kernels at least, including 2.6.21, fragmentation of
locally generated packets occurs - if needed - only after the OUTPUT
chains, so these chains do not see fragmentation even without the
connection tracking.

Daneel a écrit :
>
> Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
> What I see is that each picture of the video is send as a single TCP/
> IP packet (?) - usually with a size of around 10 KB. This TCP/IP
> packet is fragmented according to the MTU and therefore I see the
> following fragments on the wire:
> Fragment 1: <Ethernet header><IP header><TCP header>application data
> Fragment 2: <Ethernet header><IP header>application data
> Fragment 3: <Ethernet header><IP header>application data

This sounds highly unusual. Usually TCP connections limit the MSS
(maximum segment size) according to the MTU size in order to avoid IP
fragmentation.

On May 9, 9:07 pm, Pascal Hambourg <boite-a-s...@plouf.fr.eu.org>
wrote:
> Daneel a écrit :
>
>
>
> > Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
> > What I see is that each picture of the video is send as a single TCP/
> > IP packet (?) - usually with a size of around 10 KB. This TCP/IP
> > packet is fragmented according to the MTU and therefore I see the
> > following fragments on the wire:
> > Fragment 1: <Ethernet header><IP header><TCP header>application data
> > Fragment 2: <Ethernet header><IP header>application data
> > Fragment 3: <Ethernet header><IP header>application data
>
> This sounds highly unusual. Usually TCP connections limit the MSS
> (maximum segment size) according to the MTU size in order to avoid IP
> fragmentation.

So what you say is that I should (!) only see:
<Ethernet header><IP header><TCP header>application data
without any
<Ethernet header><IP header>application data
in-between for a TCP stream (given a proper TCP/IP stack)?

Thanks,
Michael

Daneel a écrit :
>>
>>>Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
>>>What I see is that each picture of the video is send as a single TCP/
>>>IP packet (?) - usually with a size of around 10 KB. This TCP/IP
>>>packet is fragmented according to the MTU and therefore I see the
>>>following fragments on the wire:
[...]
>>This sounds highly unusual. Usually TCP connections limit the MSS
>>(maximum segment size) according to the MTU size in order to avoid IP
>>fragmentation.
>
> So what you say is that I should (!) only see:
> <Ethernet header><IP header><TCP header>application data
> without any
> <Ethernet header><IP header>application data
> in-between for a TCP stream (given a proper TCP/IP stack)?

Indeed this is what I would expect to see, unless the path MTU is lower
than the interface MTU and the hosts do not use path MTU discovery
(PMTUd), thus causing fragmentation on some router along the path.
However I would not expect to see TCP segments bigger than the interface
MTU. But I am not a TCP/IP specialist at all, so my view of what
actually exists may be limited. Someone with better knowledge and/or
experience may correct me if I'm wrong.

Could you please provide a capture of the beginning of a connection,
showing at least the SYN packets with the MSS option and one ~10kB
segment ? This will not help you, but I'm curious. ;-)