Tie UDP reply to incoming interface

I have a box where an openvpn server is running. This box has two internet
connections, and openvpn answers on both addresses. What I'd like to
accomplish is to somehow have openVPN's answers to incoming connections go
out the same interface they came in. In other words, if a client connects
on eth1, all the traffic to/from that client should use eth1; the same for
eth2. Of course, many clients should be able to connect at the same time.
This article: http://www.linuxjournal.com/article/7291 seems to suggest
that doing what I want is as easy as setting up two rules, which I did:

# ip rule
0: from all lookup local
20000: from 1.1.1.1 lookup 1
20001: from 2.2.2.2 lookup 2
32766: from all lookup main
32767: from all lookup default

# ip route shot table 1
1.1.1.2 dev eth1 scope link src 1.1.1.1
1.1.1.0/24 dev eth1 proto kernel scope link src 1.1.1.1
default via 1.1.1.2 dev eth1

# ip route show table 2
2.2.2.3 dev eth2 scope link src 2.2.2.2
2.2.2.0/24 dev eth2 proto kernel scope link src 2.2.2.2
default via 2.2.2.3 dev eth2

# ip route show table main
1.1.1.0/24 dev eth1 proto kernel scope link src 1.1.1.1
2.2.2.0/24 dev eth2 proto kernel scope link src 2.2.2.2
default via 1.1.1.2 dev eth1 metric 100
default via 2.2.2.3 dev eth2 metric 100

However, I see erratic behavior in client connections; if a client connects
to 1.1.1.1, everything seems to work fine, but if another client (shortly
after the first) connects to 2.2.2.2, it doesn't get an answer.

My suspicion is that, for the reply packets, for some reason the "main"
routing table is used to lookup the destination, and since the default
route via 1.1.1.2 comes first, that one is used regardless of the incoming
interface.

I know that UDP is stateless, but I would expect that the source address of
the reply packet matches the destination address of the received packet, as
described in the article. That would trigger one of the two routing rules
(20000 or 20001), and the right routing table (and thus outgoing interface)
would be used. Or does it all depend on the application?

I also know that I could run two instances of openvpn, each tied to one
address, and that would solve the problem.
But I was just curious to understand why the way I'm doing it isn't working,
and whether there are ways around it.

Thanks in advance.

"pk" <(E-Mail Removed)> wrote in message news:h8bgjs$n92$(E-Mail Removed)...
> I have a box where an openvpn server is running. This box has two
internet
> connections, and openvpn answers on both addresses. What I'd like to
> accomplish is to somehow have openVPN's answers to incoming connections
go
> out the same interface they came in. In other words, if a client connects
> on eth1, all the traffic to/from that client should use eth1; the same
for
> eth2. Of course, many clients should be able to connect at the same time.
> This article: http://www.linuxjournal.com/article/7291 seems to suggest
> that doing what I want is as easy as setting up two rules, which I did:

I seem to remember something about a kernel setting (might be /proc
accessible) where one can do this....

pk <(E-Mail Removed)> wrote:
> I know that UDP is stateless, but I would expect that the source
> address of the reply packet matches the destination address of the
> received packet, as described in the article.

Well... if the UDP socket on which the request is recieved is bound
to an IP address, then that will be the source IP of the reply. if
the socket is unbound, then the routing lookup of the destination will
imply the source IP to be used.

rick jones
--
Wisdom Teeth are impacted, people are affected by the effects of events.
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Hello,

pk a écrit :
>
> # ip route shot table 1
> 1.1.1.2 dev eth1 scope link src 1.1.1.1

Redundant with the prefix route.

> 1.1.1.0/24 dev eth1 proto kernel scope link src 1.1.1.1
> default via 1.1.1.2 dev eth1
>
> # ip route show table 2
> 2.2.2.3 dev eth2 scope link src 2.2.2.2

Redundant with the prefix route.

> 2.2.2.0/24 dev eth2 proto kernel scope link src 2.2.2.2
> default via 2.2.2.3 dev eth2
>
> # ip route show table main
> 1.1.1.0/24 dev eth1 proto kernel scope link src 1.1.1.1
> 2.2.2.0/24 dev eth2 proto kernel scope link src 2.2.2.2
> default via 1.1.1.2 dev eth1 metric 100
> default via 2.2.2.3 dev eth2 metric 100

There are two conflicting default routes with the same metric. Only one
is actually used, the other one is just ignored.

> However, I see erratic behavior in client connections; if a client connects
> to 1.1.1.1, everything seems to work fine, but if another client (shortly
> after the first) connects to 2.2.2.2, it doesn't get an answer.
>
> My suspicion is that, for the reply packets, for some reason the "main"
> routing table is used to lookup the destination, and since the default
> route via 1.1.1.2 comes first, that one is used regardless of the incoming
> interface.

Have you first checked that the UDP layer actually receives the incoming
packet (this can be done with a LOG iptables rule in the INPUT chain) ?
If not, check that source validation by reversed path is disabled on
eth1 and eth2 (sysctl net.ipv4.conf.<interface>.rp_filter or
net.ipv4.conf.all.rp_filter are set to 0).

> I know that UDP is stateless, but I would expect that the source address of
> the reply packet matches the destination address of the received packet,

Only when the local socket is bound to this address. If the socket is
bound to the wildcard address (0.0.0.0), then source address selection
is up to the application or the routing decision by default.

Rick Jones wrote:

> pk <(E-Mail Removed)> wrote:
>> I know that UDP is stateless, but I would expect that the source
>> address of the reply packet matches the destination address of the
>> received packet, as described in the article.
>
> Well... if the UDP socket on which the request is recieved is bound
> to an IP address, then that will be the source IP of the reply. if
> the socket is unbound, then the routing lookup of the destination will
> imply the source IP to be used.

Yes that was indeed the case. Further investigation indicates that it's a
known openVPN issue, and a patch to enable multihoming was added starting
from 2.1rc4 (IIRC). What that patch does is still bind to 0.0.0.0 but enable
the server to set the source address in the UDP packet based on the
destination address found in the incoming packet (because it DOES know how
to match request packets and replies). With the multihome patch, each
response packet has a source address before the routing decision is made (as
far as I understand it), so the rules that send packets out of a specific
interface are now used.

David Schwartz wrote:

> On Sep 10, 11:27 am, pk <p...@pk.invalid> wrote:
>
>> I have a box where an openvpn server is running. This box has two
internet
>> connections, and openvpn answers on both addresses. What I'd like to
>> accomplish is to somehow have openVPN's answers to incoming connections
go
>> out the same interface they came in. In other words, if a client connects
>> on eth1, all the traffic to/from that client should use eth1; the same
for
>> eth2. Of course, many clients should be able to connect at the same time.
>> This article:http://www.linuxjournal.com/article/7291seems to suggest
>> that doing what I want is as easy as setting up two rules, which I did:
>
> This makes no sense and is not likely what you actually want.
>
> First, why it makes no sense: A client does not connect on an
> interface. They connect to a machine on a destination IP address.

Which is bound to a physical interface, so if the client connects to that IP
address, the request comes in through that interface. Bad wording in my
original post.

> Second, why it's not actually what you want. If you are sending the
> packet to 1.2.3.4 and 1.2.3.4 is directly connected to eth1, why would
> you ever want to send the packet out eth2?

That is obviously not the case (sorry if that was not clear). The connected
networks only have two IP addresses in use, one is assigned to the openvpn
server interface, the other to the upstream router interface facing the
server.
All the clients are remote, and no packet will ever come in with a source
address belonging to a connected network.

(and in any case, the way the routing roules and tables are populated will
ensure that packets for connected networks go via the relevant interface).

Pascal Hambourg wrote:

> Hello,
>
> pk a écrit :
>>
>> # ip route shot table 1
>> 1.1.1.2 dev eth1 scope link src 1.1.1.1
>
> Redundant with the prefix route.

Fair enough.

>> 1.1.1.0/24 dev eth1 proto kernel scope link src 1.1.1.1
>> default via 1.1.1.2 dev eth1
>>
>> # ip route show table 2
>> 2.2.2.3 dev eth2 scope link src 2.2.2.2
>
> Redundant with the prefix route.
>
>> 2.2.2.0/24 dev eth2 proto kernel scope link src 2.2.2.2
>> default via 2.2.2.3 dev eth2
>>
>> # ip route show table main
>> 1.1.1.0/24 dev eth1 proto kernel scope link src 1.1.1.1
>> 2.2.2.0/24 dev eth2 proto kernel scope link src 2.2.2.2
>> default via 1.1.1.2 dev eth1 metric 100
>> default via 2.2.2.3 dev eth2 metric 100
>
> There are two conflicting default routes with the same metric. Only one
> is actually used, the other one is just ignored.

Sure, but my point is that those routes shouldn't be reached at all in my
scenario.

>> My suspicion is that, for the reply packets, for some reason the "main"
>> routing table is used to lookup the destination, and since the default
>> route via 1.1.1.2 comes first, that one is used regardless of the
incoming
>> interface.
>
> Have you first checked that the UDP layer actually receives the incoming
> packet (this can be done with a LOG iptables rule in the INPUT chain) ?
> If not, check that source validation by reversed path is disabled on
> eth1 and eth2 (sysctl net.ipv4.conf.<interface>.rp_filter or
> net.ipv4.conf.all.rp_filter are set to 0).

It was an application issue (see my other reply). No change in the OS was
necessary, other than the already existing routing rules and tables.

>> I know that UDP is stateless, but I would expect that the source address
of
>> the reply packet matches the destination address of the received packet,
>
> Only when the local socket is bound to this address. If the socket is
> bound to the wildcard address (0.0.0.0), then source address selection
> is up to the application or the routing decision by default.

Right, this was exactly the case.

> > First, why it makes no sense: A client does not connect on an
> > interface. They connect to a machine on a destination IP address.

> Which is bound to a physical interface, so if the client connects to
> that IP address, the request comes in through that interface. Bad
> wording in my original post.

And thus we have the beginnings of a discussion on the relative merits
of the so called "strong" vs "weak" end system model...

The Linux stack is very firmly in the "weak" end system model camp
where IP addresses are attributes of systems and interfaces are merely
means by which packets enter and leave the system. IP datagrams for
any IP will be accepted on any interface. ARP will be happy to
respond to ARP requests for any IP address in the system on any
interface.

(That last one drives me bonkers when I am running netperf tests with
multiple interfaces connected to the same broadcast domain, I always
set the "arp_ignore" sysctl to deal with it)

A full acceptance of the "strong" end system model is one where IP
addresses are attributes of interfaces - for example when one sets
ip_strong_es_model to a value of one in HP-UX (and perhaps Solaris,
they are network stack cousins) the system will only accept IP
datagrams via the interface to which that IP address was assigned.

Much of the rest of the world (including the UX and Solaris default
behaviour) tends to wobble about somewhere in the middle.

I'm sure there is much more discussion about the topic on the web.

rick jones
--
No need to believe in either side, or any side. There is no cause.
There's only yourself. The belief is in your own precision. - Joubert
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Rick Jones wrote:

>> > First, why it makes no sense: A client does not connect on an
>> > interface. They connect to a machine on a destination IP address.
>
>> Which is bound to a physical interface, so if the client connects to
>> that IP address, the request comes in through that interface. Bad
>> wording in my original post.
>
> And thus we have the beginnings of a discussion on the relative merits
> of the so called "strong" vs "weak" end system model...

Ok, I'm aware of the difference between strong and weak host model. Let's
assume that for my scenario one interface can never receive an ARP request
for the IP address assigned to the "other" interface (which is true in
practice, since they're two different physical links to two different ISP,
and I strongly doubt either ISP will let that happen).

> The Linux stack is very firmly in the "weak" end system model camp
> where IP addresses are attributes of systems and interfaces are merely
> means by which packets enter and leave the system. IP datagrams for
> any IP will be accepted on any interface. ARP will be happy to
> respond to ARP requests for any IP address in the system on any
> interface.

Right. I seem to remember, however, that some knob in /proc can be exploited
to somewhat modify that behavior. But I can't check now, so I might as well
be wrong.

> (That last one drives me bonkers when I am running netperf tests with
> multiple interfaces connected to the same broadcast domain, I always
> set the "arp_ignore" sysctl to deal with it)
>
> A full acceptance of the "strong" end system model is one where IP
> addresses are attributes of interfaces - for example when one sets
> ip_strong_es_model to a value of one in HP-UX (and perhaps Solaris,
> they are network stack cousins) the system will only accept IP
> datagrams via the interface to which that IP address was assigned.
>
> Much of the rest of the world (including the UX and Solaris default
> behaviour) tends to wobble about somewhere in the middle.
>
> I'm sure there is much more discussion about the topic on the web.

Yes, that has also been discussed to death on lkml and various networking-
related mailing lists.

David Schwartz wrote:

> Suppose a machine has IP addresses 192.168.1.10/24 and
> 192.168.2.10/24. If I'm on 192.168.1.3 and I connect to 192.168.2.10
> (assuming I know a route), the packet may be received by that machine
> on the interface in the 192.168.1.0/24 network.
>
> Even if I don't know the route, if I send my packet to my default
> router, say 192.168.1.1, it may know that 192.168.2.10 and
> 192.168.1.10 are the same machine, so it will send the packet to
> 192.168.2.10 to the interface assigned 192.168.1.20 since it has a
> direct path to that interface.

Last time: that is NOT the case for me. My interfaces are in DIFFERENT IP
subnets.

> Since no such packet will ever come in, it doesn't matter how you
> handle such a packet. So your proposal and my proposal will do the
> same thing, except mine is logical and yours is not. So you should
> prefer my solution, because it doesn't break if your topology changes.

Erm, and your solution would be?