NAT and port-based routing in a multi-homed enviroment

Hello,

I have 2 ISP connections, and I am trying to setup port-based routing
using iptables and iproute2 using the example documented at:
http://www.linuxhorizon.ro/iproute2.html

eth1 connects to the main ISP and is the default route in the main
table. All in/out traffic now goes through this interface.
eth0 connects to the alternate ISP. I want to divert internal web
browsing from our internal network to go out through eth0.
eth2 connects to the internal network.

eth0's public IP (making these up) is 10.0.0.2 with gateway 10.0.0.1.
I am trying to browse out to public IP 144.89.40.111

Forwaridng for all internal traffic is enabled:
-A FORWARD -i eth2 -j ACCEPT

Forwarding for all inbound related/established traffic is enableD:
-A FORWARD -d 192.168.1.0/255.255.255.0 -p tcp -m state --state
RELATED,ESTABLISHED -j ACCEPT

I have enabled SNAT for eth0 with iptables as follows:
-t nat -A POSTROUTING -o eth0 -j SNAT --to-source 10.0.0.2

To redirect all outbound web browsing traffic to eth0, I mark the
packets as follows:
-t mangle -A PREROUTING -s 192.168.1.0/255.255.255.0 -p tcp -m tcp --
dport 80 -j MARK --set-mark 0x1

I created a table T1 to hold the routing table for eth0:
192.168.1.0/24 dev eth2 scope link
127.0.0.0/8 dev lo scope link
default via 10.0.0.1 dev eth0

Finally I add a rule to use table T1 for traffic marked with mark 1
ip rule add from all fwmark 1 lookup T1

When I try to load a webpage from internal host 192.168.1.21, I see
the gateway forward the initial SYN packet to the remote host. The
remote host then replies to the correct interface (eth0) with SYN/ACK.

On the gateway, I can see the following ip_conntrack entry:
tcp 6 55 SYN_RECV src=192.168.1.21 dst=144.89.40.111 sport=51618
dport=80 packets=1 bytes=60 src=144.89.40.111 dst=10.0.0.2 sport=80
dport=51618 packets=3 bytes=132 mark=0 use=1

Packet sniffing on the internal host (192.168.1.21) shows it keeps
sending SYN packets, but never receives the SYN/ACK. So the gateway is
not forwarding the returned SYN/ACK packet back to the internal host.

If I remove the rule added above, I am able to browse the web using
the primary eth1 connection so basic NAT is working just fine.

I'm using FC3 (2.6.12-1.1381_FC3)

Anyone have any pointers on how to further debug this problem? Is
there something glaring I'm missing?

Thanks in advance,

-- Ron

Hello,

RDub wrote :
[...]
> I created a table T1 to hold the routing table for eth0:
> 192.168.1.0/24 dev eth2 scope link
> 127.0.0.0/8 dev lo scope link

You don't need this route. Routes to local destinations are already in
the 'local' routing table which has higher precedence.

> default via 10.0.0.1 dev eth0
>
> Finally I add a rule to use table T1 for traffic marked with mark 1
> ip rule add from all fwmark 1 lookup T1
>
> When I try to load a webpage from internal host 192.168.1.21, I see
> the gateway forward the initial SYN packet to the remote host. The
> remote host then replies to the correct interface (eth0) with SYN/ACK.
>
> On the gateway, I can see the following ip_conntrack entry:
> tcp 6 55 SYN_RECV src=192.168.1.21 dst=144.89.40.111 sport=51618
> dport=80 packets=1 bytes=60 src=144.89.40.111 dst=10.0.0.2 sport=80
> dport=51618 packets=3 bytes=132 mark=0 use=1
>
> Packet sniffing on the internal host (192.168.1.21) shows it keeps
> sending SYN packets, but never receives the SYN/ACK. So the gateway is
> not forwarding the returned SYN/ACK packet back to the internal host.

Check that source validation by reversed path is disabled at least for
eth0, i.e. either /proc/sys/net/ipv4/conf/eth0/rp_filter or
/proc/sys/net/ipv4/conf/all/rp_filter is 0. If not, set
/proc/sys/net/ipv4/conf/eth0/rp_filter to 0.

On Sep 9, 6:36 pm, Pascal Hambourg <boite-a-s...@plouf.fr.eu.org>
wrote:

> Check that source validation by reversed path is disabled at least for
> eth0, i.e. either /proc/sys/net/ipv4/conf/eth0/rp_filter or
> /proc/sys/net/ipv4/conf/all/rp_filter is 0. If not, set
> /proc/sys/net/ipv4/conf/eth0/rp_filter to 0.- Hide quoted text -

EUREKA! THANK YOU THANK YOU THANK YOU!!!

echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter seems to have done the
trick!

Woondabar!