balancing and conntrack

Hi all!

My problem looks like this:

I have 2xDSL and i am trying to do load balancing (with NAT)
what i do i exactly like in HOWTO

ip link set $IFI up
ip addr flush dev $IFI
ip addr add $IPI/$NMI brd + dev $IFI
ip rule add prio 50 table main
ip route del default table main


ip link set $IFE1 up
ip addr flush dev $IFE1
ip addr add $IPE1/$NME1 brd $BRD1 dev $IFE1

ip link set $IFE2 up
ip addr flush dev $IFE2
ip addr add $IPE2/$NME2 brd $BRD2 dev $IFE2


ip rule add prio 201 from $NWE1/$NME1 table 201
ip route add default via $GWE1 dev $IFE1 src $IPE1 proto static
table 201
ip route append prohibit default table 201 metric 1 proto
static

ip rule add prio 202 from $NWE2/$NME2 table 202
ip route add default via $GWE2 dev $IFE2 src $IPE2 proto static
table 202
ip route append prohibit default table 202 metric 1 proto
static

ip rule add prio 222 table 222
ip route add default table 222 proto static \
nexthop via $GWE1 dev $IFE1 weight 1\
nexthop via $GWE2 dev $IFE2 weight 2


then


iptables -t filter -N keep_state
iptables -t filter -A keep_state -m state --state RELATED,ESTABLISHED
-j ACCEPT
iptables -t filter -A keep_state -j RETURN

iptables -t nat -N keep_state
iptables -t nat -A keep_state -m state --state RELATED,ESTABLISHED -j
ACCEPT
iptables -t nat -A keep_state -j RETURN

iptables -t nat -A PREROUTING -j keep_state
iptables -t nat -A POSTROUTING -j keep_state
iptables -t nat -A OUTPUT -j keep_state
iptables -t filter -A INPUT -j keep_state
iptables -t filter -A FORWARD -j keep_state

iptables -t filter -A OUTPUT -j
keep_state
iptables -t mangle -A PREROUTING -s 192.168.0.0/16 -i $IFI -j IMQ
--todev 0
iptables -t mangle -A POSTROUTING -d 192.168.0.0/16 -o $IFI -j IMQ
--todev 1


and every thing seems to be ok load balancing works very well BUT
somteimes it breaks TCP sessions what i see in TCPDUMP output when
session is resetting that sometimes packets from opened session which
should go same interface goes through another one and then session
breaks... and ideas ?


best regards
David

On 2 Jun 2006 04:38:56 -0700, (E-Mail Removed) wrote:

>Hi all!
>
>My problem looks like this:
>
>I have 2xDSL and i am trying to do load balancing (with NAT)
>what i do i exactly like in HOWTO
>
> ip link set $IFI up
> ip addr flush dev $IFI
> ip addr add $IPI/$NMI brd + dev $IFI
> ip rule add prio 50 table main
> ip route del default table main
>
>
> ip link set $IFE1 up
> ip addr flush dev $IFE1
> ip addr add $IPE1/$NME1 brd $BRD1 dev $IFE1
>
> ip link set $IFE2 up
> ip addr flush dev $IFE2
> ip addr add $IPE2/$NME2 brd $BRD2 dev $IFE2
>
>
> ip rule add prio 201 from $NWE1/$NME1 table 201
> ip route add default via $GWE1 dev $IFE1 src $IPE1 proto static
>table 201
> ip route append prohibit default table 201 metric 1 proto
>static
>
> ip rule add prio 202 from $NWE2/$NME2 table 202
> ip route add default via $GWE2 dev $IFE2 src $IPE2 proto static
>table 202
> ip route append prohibit default table 202 metric 1 proto
>static
>
> ip rule add prio 222 table 222
> ip route add default table 222 proto static \
> nexthop via $GWE1 dev $IFE1 weight 1\
> nexthop via $GWE2 dev $IFE2 weight 2
>
>
>then
>
>
>iptables -t filter -N keep_state
>iptables -t filter -A keep_state -m state --state RELATED,ESTABLISHED
>-j ACCEPT
>iptables -t filter -A keep_state -j RETURN
>
>iptables -t nat -N keep_state
>iptables -t nat -A keep_state -m state --state RELATED,ESTABLISHED -j
>ACCEPT
>iptables -t nat -A keep_state -j RETURN
>
>iptables -t nat -A PREROUTING -j keep_state
>iptables -t nat -A POSTROUTING -j keep_state
>iptables -t nat -A OUTPUT -j keep_state
>iptables -t filter -A INPUT -j keep_state
>iptables -t filter -A FORWARD -j keep_state
>
> iptables -t filter -A OUTPUT -j
>keep_state
>iptables -t mangle -A PREROUTING -s 192.168.0.0/16 -i $IFI -j IMQ
>--todev 0
>iptables -t mangle -A POSTROUTING -d 192.168.0.0/16 -o $IFI -j IMQ
>--todev 1
>
>
>and every thing seems to be ok load balancing works very well BUT
>somteimes it breaks TCP sessions what i see in TCPDUMP output when
>session is resetting that sometimes packets from opened session which
>should go same interface goes through another one and then session
>breaks... and ideas ?
>
>
>best regards
>David

During the last week or so a solution was posted to the LARTC mailing
list. Go to the archive and look for "Problems with Routing and
Masquerading".

If I understood the postings correctly, the usual thing to "break" the
routing is SSH.

CONNTRACK doesn't work but CONNMARK does, so keep_state is not
sufficient.
--
buck

(E-Mail Removed) wrote:
> Hi all!
>
> My problem looks like this:
>
> I have 2xDSL and i am trying to do load balancing (with NAT)
> what i do i exactly like in HOWTO
>
> ip link set $IFI up
> ip addr flush dev $IFI
> ip addr add $IPI/$NMI brd + dev $IFI
> ip rule add prio 50 table main
> ip route del default table main
>
>
> ip link set $IFE1 up
> ip addr flush dev $IFE1
> ip addr add $IPE1/$NME1 brd $BRD1 dev $IFE1
>
> ip link set $IFE2 up
> ip addr flush dev $IFE2
> ip addr add $IPE2/$NME2 brd $BRD2 dev $IFE2
>
>
> ip rule add prio 201 from $NWE1/$NME1 table 201
> ip route add default via $GWE1 dev $IFE1 src $IPE1 proto static
> table 201
> ip route append prohibit default table 201 metric 1 proto
> static
>
> ip rule add prio 202 from $NWE2/$NME2 table 202
> ip route add default via $GWE2 dev $IFE2 src $IPE2 proto static
> table 202
> ip route append prohibit default table 202 metric 1 proto
> static
>
> ip rule add prio 222 table 222
> ip route add default table 222 proto static \
> nexthop via $GWE1 dev $IFE1 weight 1\
> nexthop via $GWE2 dev $IFE2 weight 2
>

Have you applied Julian Anastasov's kernel patches? If not, I
don't think these rules are enough. Another method to assure that
interface is not changed during connection is using CONNMARK in
iptables.

>
> then
>
>
> iptables -t filter -N keep_state
> iptables -t filter -A keep_state -m state --state RELATED,ESTABLISHED
> -j ACCEPT
> iptables -t filter -A keep_state -j RETURN
>
> iptables -t nat -N keep_state
> iptables -t nat -A keep_state -m state --state RELATED,ESTABLISHED -j
> ACCEPT

Are you sure it was in HOWTO? ACCEPT in nat can cause omitting of
possible next "real" nat rules.

> iptables -t nat -A keep_state -j RETURN
>
> iptables -t nat -A PREROUTING -j keep_state
> iptables -t nat -A POSTROUTING -j keep_state
> iptables -t nat -A OUTPUT -j keep_state
> iptables -t filter -A INPUT -j keep_state
> iptables -t filter -A FORWARD -j keep_state
>
> iptables -t filter -A OUTPUT -j
> keep_state

I hope you have some more filter rules or this one only for
RELATED and ESTABLISHED would be unnecessary.

Jarek P.

that was a prolem, i did not applied those patches
best regards
and thanx a lot


Jarek Poplawski napisal(a):
> (E-Mail Removed) wrote:
> > Hi all!
> >
> > My problem looks like this:
> >
> > I have 2xDSL and i am trying to do load balancing (with NAT)
> > what i do i exactly like in HOWTO
> >
> > ip link set $IFI up
> > ip addr flush dev $IFI
> > ip addr add $IPI/$NMI brd + dev $IFI
> > ip rule add prio 50 table main
> > ip route del default table main
> >
> >
> > ip link set $IFE1 up
> > ip addr flush dev $IFE1
> > ip addr add $IPE1/$NME1 brd $BRD1 dev $IFE1
> >
> > ip link set $IFE2 up
> > ip addr flush dev $IFE2
> > ip addr add $IPE2/$NME2 brd $BRD2 dev $IFE2
> >
> >
> > ip rule add prio 201 from $NWE1/$NME1 table 201
> > ip route add default via $GWE1 dev $IFE1 src $IPE1 proto static
> > table 201
> > ip route append prohibit default table 201 metric 1 proto
> > static
> >
> > ip rule add prio 202 from $NWE2/$NME2 table 202
> > ip route add default via $GWE2 dev $IFE2 src $IPE2 proto static
> > table 202
> > ip route append prohibit default table 202 metric 1 proto
> > static
> >
> > ip rule add prio 222 table 222
> > ip route add default table 222 proto static \
> > nexthop via $GWE1 dev $IFE1 weight 1\
> > nexthop via $GWE2 dev $IFE2 weight 2
> >
>
> Have you applied Julian Anastasov's kernel patches? If not, I
> don't think these rules are enough. Another method to assure that
> interface is not changed during connection is using CONNMARK in
> iptables.
>
> >
> > then
> >
> >
> > iptables -t filter -N keep_state
> > iptables -t filter -A keep_state -m state --state RELATED,ESTABLISHED
> > -j ACCEPT
> > iptables -t filter -A keep_state -j RETURN
> >
> > iptables -t nat -N keep_state
> > iptables -t nat -A keep_state -m state --state RELATED,ESTABLISHED -j
> > ACCEPT
>
> Are you sure it was in HOWTO? ACCEPT in nat can cause omitting of
> possible next "real" nat rules.
>
> > iptables -t nat -A keep_state -j RETURN
> >
> > iptables -t nat -A PREROUTING -j keep_state
> > iptables -t nat -A POSTROUTING -j keep_state
> > iptables -t nat -A OUTPUT -j keep_state
> > iptables -t filter -A INPUT -j keep_state
> > iptables -t filter -A FORWARD -j keep_state
> >
> > iptables -t filter -A OUTPUT -j
> > keep_state
>
> I hope you have some more filter rules or this one only for
> RELATED and ESTABLISHED would be unnecessary.
>
> Jarek P.