Bonding does not work on a switch?

Hello!

I try to configure bonding in active backup mode for an embedded device
booting a Linux 2.4.20 kernel.

I found out by monitoring /proc/net/bonding/info that a lost link is
perfectly detected and the kernel (module) switches from eth1 to eth2.

However, if I connect these two NICs to a switch, the connection to my
embedded system gets lost (ping replies stop, telnet sessions are
interrupted). If I connect the two devices to a dumb hub (sorry all you
brave hubs out there, no offense :-)) everything works as expected.

I tried three different models of switches, from a small SOHO model to a
manageable enterprise class switch.

As far as I understood the documentation, active backup mode should work
with any layer 2 switch.

Are there any parameters I have to tweak?

Thank you in advance,
kind regards,

Heiko


Heiko Gerstung

In article <s28i32-(E-Mail Removed)>, Heiko Gerstung wrote:
>
> However, if I connect these two NICs to a switch, the connection to my
> embedded system gets lost (ping replies stop, telnet sessions are
> interrupted). If I connect the two devices to a dumb hub (sorry all you
> brave hubs out there, no offense :-)) everything works as expected.

Switches remember which ports it has seen various mac addresses on and
will forward packets for those mac addresses specifically to those ports.

I think you can use two switches in parallel with one leg of each bonded
channel going to each switch.

I also suspect that you might be misusing channel bonding. It isn't for
providing failover in the event that a link goes down, it is for using
two physical links as though they were one link.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
NotDashEscaped: You need GnuPG to verify this message

In comp.os.linux.networking Bruno Wolff III <(E-Mail Removed)>:
> In article <s28i32-(E-Mail Removed)>, Heiko Gerstung wrote:
>>
>> However, if I connect these two NICs to a switch, the connection to my
>> embedded system gets lost (ping replies stop, telnet sessions are
>> interrupted). If I connect the two devices to a dumb hub (sorry all you
>> brave hubs out there, no offense :-)) everything works as expected.

> Switches remember which ports it has seen various mac addresses on and
> will forward packets for those mac addresses specifically to those ports.

> I think you can use two switches in parallel with one leg of each bonded
> channel going to each switch.

Yup, if you don't like a single point of failure, it's
recommended.

> I also suspect that you might be misusing channel bonding. It isn't for
> providing failover in the event that a link goes down, it is for using
> two physical links as though they were one link.

Nope, it can do both at the same time, ha and aggregating
bandwidth, or only run in ha mode with one or more backup devices.

Cheapo switches usually work out of the box, $$ switches need to
be configured for bonding, they don't like seeing the same MAC on
different ports and unexpected things might happen. Even if you
use it over multiple switches they need to be configured.

I suspect some configuration error, following
/usr/src/linux/Documentation/bonding.txt should enable the OP to
configure bonding probably.

--
Michael Heiming (GPG-Key ID: 0xEDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBZsnxAkPEju3Se5QRAkrSAJ0d7jqMRDMVA4zkKlCEHT 6dlXtrZgCeLhfK
oDmJCRpDhVhd0AoE5Lh9Vd0=
=F0s3
-----END PGP SIGNATURE-----

Bruno,

thank you for your answer. As Michael already corrected, the bonding
module for Linux is supposed to provide link aggregation, but also can
be run in "active backup mode", which means exactly that if one link
fails, the next one in the corresponding "bonding group" is used
(without major interruption). It's possible but it does not work for me,
unfortunately.

Kind regards,
Heiko


Bruno Wolff III wrote:
> In article <s28i32-(E-Mail Removed)>, Heiko Gerstung wrote:
>
>>However, if I connect these two NICs to a switch, the connection to my [...]
>
> Switches remember which ports it has seen various mac addresses on and
> will forward packets for those mac addresses specifically to those ports.
>
> I think you can use two switches in parallel with one leg of each bonded
> channel going to each switch.
>
> I also suspect that you might be misusing channel bonding. It isn't for
> providing failover in the event that a link goes down, it is for using
> two physical links as though they were one link.



--
Meinberg radio clocks: 25 years of accurate time worldwide

Meinberg Radio Clocks
www.meinberg.de

Stand alone ntp time servers and radio clocks based on GPS, DCF77 and
IRIG. Rackmount and desktop versions and PCI slot cards.

Hello Michael,

thank you very much. Please find my comments below:

Michael Heiming wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> NotDashEscaped: You need GnuPG to verify this message
> [...]
>>I think you can use two switches in parallel with one leg of each bonded
>>channel going to each switch.
>
>
> Yup, if you don't like a single point of failure, it's
> recommended.
>
Yes, that's why our customer wants to use it and that's why we
implemented it in our configuration interface.

>>I also suspect that you might be misusing channel bonding. It isn't for[...]
>
>
> Nope, it can do both at the same time, ha and aggregating
> bandwidth, or only run in ha mode with one or more backup devices.
Correct. Thanks for clarifying this for me :-)

> Cheapo switches usually work out of the box, $$ switches need to
> be configured for bonding, they don't like seeing the same MAC on
> different ports and unexpected things might happen. Even if you
> use it over multiple switches they need to be configured.
The problem is that if you are interested to eliminate single points of
failure and want to be on the safer side, you normally buy $$ switches,
not the cheapo types ...

> I suspect some configuration error, following
> /usr/src/linux/Documentation/bonding.txt should enable the OP to
> configure bonding probably.
I checked it and found this section:

::: cut here :::
While the switch does not need to be configured when the active-backup,
balance-tlb or balance-alb policies (mode=1,5,6) are used, it does need
to be configured for the round-robin, XOR, broadcast, or 802.3ad
policies (mode=0,2,3,4).
::: cut here :::

So, it works on a hub but not on a switch. And no configuration needed
for a switch in mode 1 ...

Again, thanks for your reply. Any other ideas?

Kind regards,
Heiko




--
Meinberg radio clocks: 25 years of accurate time worldwide

Meinberg Radio Clocks
www.meinberg.de

Stand alone ntp time servers and radio clocks based on GPS, DCF77 and
IRIG. Rackmount and desktop versions and PCI slot cards.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
NotDashEscaped: You need GnuPG to verify this message

In comp.os.linux.networking Heiko Gerstung <heiko.gerstung_removeme_@meinberg.de>:
> Michael Heiming wrote:
[..]

> ::: cut here :::
> While the switch does not need to be configured when the active-backup,
> balance-tlb or balance-alb policies (mode=1,5,6) are used, it does need
> to be configured for the round-robin, XOR, broadcast, or 802.3ad
> policies (mode=0,2,3,4).
> ::: cut here :::

> So, it works on a hub but not on a switch. And no configuration needed
> for a switch in mode 1 ...

Not quite, you need to configure $$ switches to make use of
bonding anyway or a disaster is likely to happen. If you want to
use multiple switches, you need to configure something like ISL,
(cicso). Check your switches documentation or ask the people in
charge for them.

--
Michael Heiming (GPG-Key ID: 0xEDD27B94)
mail: echo (E-Mail Removed) | perl -pe 'y/a-z/n-za-m/'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBbjirAkPEju3Se5QRAlaYAKDSL3oxb6bmgpoBG/5wr8zvsxaY5gCeJBUl
+mtP4Y12QD+TJfncpfwdWYQ=
=adT3
-----END PGP SIGNATURE-----

Hello Michael,

I found out how to configure my 3Com (=$$$) switch and now it works like
a charm. I already filed a "Request for Document Change" (RFDC) at the
Sourceforge Bugtracker of the Bonding module, because the documentation
(bonding.txt) is just plain wrong in this point.

Thank you very much for your assistance,
kind regards,
Heiko



Michael Heiming wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
[...]

>>So, it works on a hub but not on a switch. And no configuration needed
>>for a switch in mode 1 ...
>
>
> Not quite, you need to configure $$ switches to make use of
> bonding anyway or a disaster is likely to happen. If you want to
> use multiple switches, you need to configure something like ISL,
> (cicso). Check your switches documentation or ask the people in
> charge for them.
>


--
Meinberg radio clocks: 25 years of accurate time worldwide

Meinberg Radio Clocks
www.meinberg.de

Stand alone ntp time servers and radio clocks based on GPS, DCF77 and
IRIG. Rackmount and desktop versions and PCI slot cards.