Multiple uplinks/routes to Internet with one ethernet - MAC problem

Hi networkers,

i'm triying to setup a firewall with 2 Internet links. Followed instructions
from


Linux Advanced Routing & Traffic Control HOWTO http://lartc.org/howto/

and It work only if one connect to process in the firewall itself, but not
to masqueraded servewrs.

The real problems is that the packets are sent with correct source IP but
wrong mac, ie., the MAC of the other uplink router, so one uplink is
getting all the outbound traffic.

here is mi conf.


INTERFACES


eth0 Link encap:Ethernet HWaddr 00:80:C8:E4:3F:48
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1597982 errors:0 dropped:0 overruns:0 frame:3323
TX packets:2006989 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:266792533 (254.4 MiB) TX bytes:2048005415 (1.9 GiB)
Interrupt:16 Base address:0xe000

eth0:gtd0 Link encap:Ethernet HWaddr 00:80:C8:E4:3F:48
inet addr:200.55.216.130 Bcast:200.55.216.255
Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:16 Base address:0xe000

eth0:gtd1 Link encap:Ethernet HWaddr 00:80:C8:E4:3F:48
inet addr:200.55.216.131 Bcast:200.55.216.255
Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:16 Base address:0xe000

eth0:ifx0 Link encap:Ethernet HWaddr 00:80:C8:E4:3F:48
inet addr:200.73.16.162 Bcast:200.73.16.255 Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:16 Base address:0xe000

eth0:ifx1 Link encap:Ethernet HWaddr 00:80:C8:E4:3F:48
inet addr:200.73.16.163 Bcast:200.73.16.255 Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:16 Base address:0xe000

eth1 Link encap:Ethernet HWaddr 00:80:AD:74:85:64
inet addr:192.168.1.129 Bcast:192.168.1.255 Mask:255.255.255.128
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1952639 errors:0 dropped:0 overruns:0 frame:0
TX packets:1531224 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:1965270247 (1.8 GiB) TX bytes:235635162 (224.7 MiB)
Interrupt:17 Base address:0xec00



RULES

0: from all lookup local
32760: from 192.168.254.0/24 to 192.168.1.128/25 lookup main
32761: from 192.168.254.1 lookup main
32762: from 192.168.254.50 lookup main
32764: from 200.55.216.130 lookup gtd
32765: from 200.73.16.162 lookup ifx
32766: from all lookup main
32767: from all lookup default


ROUTING TABLES

amazing:/home/qlsoft# ip rou ls table gtd
200.55.216.128/28 dev eth0 scope link src 200.55.216.130
127.0.0.0/8 dev lo scope link
default via 200.55.216.129 dev eth0 src 200.55.216.130

amazing:/home/qlsoft# ip rou ls table ifx
200.73.16.160/28 dev eth0 scope link src 200.73.16.162
127.0.0.0/8 dev lo scope link
default via 200.73.16.161 dev eth0 src 200.73.16.162

amazing:/home/qlsoft#amazing:/home/qlsoft# ip rou ls
200.73.16.160/28 dev eth0 proto kernel scope link src 200.73.16.162
200.55.216.128/28 dev eth0 proto kernel scope link src 200.55.216.130
192.168.1.0/25 dev eth1 proto kernel scope link src 192.168.1.1
192.168.1.128/25 dev eth1 proto kernel scope link src 192.168.1.129

default via 200.73.16.161 dev eth0 (*)
amazing:/home/qlsoft#

(*) same thing happens if i setup the default route like this

ip route add default scope global nexthop via 200.55.216.129 dev eth0 weight
1 nexthop via 200.73.16.161 dev eth0 weight 1


ARP

amazing:/home/qlsoft# arp -n
Address HWtype HWaddress Flags Mask
Iface
200.55.216.129 ether 00:0B:6A:72:61:62 C
eth0
200.73.16.161 ether 00:03:6C:36:F0:00 C
eth0



TCPDUMP

CORRECT MAC
------------

09:46:32.584185 0:3:6c:36:f0:0 0:80:c8:e4:3f:48 0800 74:
200.113.10.242.38839 > 200.73.16.162.80: S 2414790943:2414790943(0) win
5808 <mss 1452,sackOK,timestamp 27088859 0,nop,wscale 2> (DF)

09:46:32.584344 0:80:c8:e4:3f:48 0:3:6c:36:f0:0 0800 74: 200.73.16.162.80 >
200.113.10.242.38839: S 2363380759:2363380759(0) ack 2414790944 win 5792
<mss 1460,sackOK,timestamp 13832165 27088859,nop,wscale 0> (DF)


WRONG MAC ( but correct source ip ) WHEN CONNECTING TO MASQUERADED SERVERS
--------------------------------------------------------------------------

09:48:12.703202 0:b:6a:72:61:62 0:80:c8:e4:3f:48 0800 74:
200.113.10.242.38859 > 200.55.216.130.80: S 2506498875:2506498875(0) win
5808 <mss 1452,sackOK,timestamp 27188994 0,nop,wscale 2> (DF)

09:48:12.703440 0:80:c8:e4:3f:48 0:3:6c:36:f0:0 0800 74: 200.55.216.130.80 >
200.113.10.242.38859: S 2478324347:2478324347(0) ack 2506498876 win 5792
<mss 1460,sackOK,timestamp 13842177 27188994,nop,wscale 0> (DF)


Conecting directly to firewall CORRECT MAC
-------------------------------------------
09:50:11.558471 0:b:6a:72:61:62 0:80:c8:e4:3f:48 0800 74:
200.113.10.242.38875 > 200.55.216.130.1433: S 2640321865:2640321865(0) win
5808 <mss 1452,sackOK,timestamp 27307857 0,nop,wscale 2> (DF)

09:50:11.558535 0:80:c8:e4:3f:48 0:b:6a:72:61:62 0800 54:
200.55.216.130.1433 > 200.113.10.242.38875: R 0:0(0) ack 2640321866 win 0
(DF)


amazing:/home/qlsoft# uname -a
Linux amazing 2.4.18-1-k7 #1 Wed Apr 14 19:20:42 UTC 2004 i686 unknown

------------------------------------------------------------------


Thanx for 4 help!

--

Felipe Alvarez Harnecker. QlSoftware.

María Luisa Santander 568, Providencia, Santiago.

Tels. 204.56.21 - 09.874.60.17
e-mail: (E-Mail Removed)
MSN: (E-Mail Removed) IRC: (E-Mail Removed)

http://qlsoft.cl/
http://ql.cl/
__________________________________________________ ____

On Thu, 24 Feb 2005 10:00:19 -0300, Felipe Alvarez <(E-Mail Removed)>
wrote:

>The real problems is that the packets are sent with correct source IP but
>wrong mac, ie., the MAC of the other uplink router, so one uplink is
>getting all the outbound traffic.

I suspect a confused ARP cache at your ISP, so My Wild Guess:
echo 1 >/proc/sys/net/ipv4/ip_forward
echo 1 >/proc/sys/net/ipv4/conf/eth0/proxy_arp

My setup uses Julian's route patch, which for me was required, not
just advisable.
--
buck