HP ProLiant ML370, SLES8, Gigabit + Fast Ethernet NICs

Dear all,

we have severe network problems within our environment consisting of

- HP ProLiant ML370 system running SLES8, Service Pack 2a
- Broadcom BCM5700 Gigabit on-board chip ====> eth0
- Intel Pro 100 Dual Server card ====> eth1, eth2

Here is our network configuration:

eth0 Link encap:Ethernet HWaddr 00:0B:CD:6A:8B:B5
inet addr:149.243.226.32 Bcast:149.243.226.255
Mask:255.255.255.0
inet6 addr: fe80::20b:cdff:fe6a:8bb5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:180352 errors:0 dropped:0 overruns:0 frame:0
TX packets:205001 errors:0 dropped:0 overruns:0 carrier:0
collisions:28403 txqueuelen:100
RX bytes:43088491 (41.0 Mb) TX bytes:119177298 (113.6 Mb)
Interrupt:29 Memory:f7ad0000-f7ae0000

eth1 Link encap:Ethernet HWaddr 00:02:B3:AC:63:45
inet addr:192.9.205.1 Bcast:192.9.205.255
Mask:255.255.255.0
inet6 addr: fe80::202:b3ff:feac:6345/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1719407 errors:0 dropped:0 overruns:0 frame:0
TX packets:1690980 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:451397306 (430.4 Mb) TX bytes:390577233 (372.4 Mb)
Interrupt:18 Base address:0x4000 Memory:f7bf0000-f7bf0038

eth2 Link encap:Ethernet HWaddr 00:02:B3:AC:63:46
inet addr:192.9.206.1 Bcast:192.9.206.255
Mask:255.255.255.0
inet6 addr: fe80::202:b3ff:feac:6346/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:40 dropped:0 overruns:0 carrier:40
collisions:0 txqueuelen:100
RX bytes:0 (0.0 b) TX bytes:16160 (15.7 Kb)
Interrupt:18 Base address:0x4040 Memory:f7bb0000-f7bb0038

We can establish any connection into the 192.9.205.0 net but trying to
get a system in the ...206... net, for examples

telnet 192.9.206.2

then this appears

telnet:connect to address 192.9.206.2: no route to host

This worries us since the routing table (netstat -r) seems to be o.k.:

Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
149.243.226.0 * 255.255.255.0 U 40 0 0 eth0
192.9.205.0 * 255.255.255.0 U 40 0 0 eth1
192.9.206.0 * 255.255.255.0 U 40 0 0 eth2
default 149.243.226.1 0.0.0.0 UG 40 0 0 eth0

For analysis reasons pls. also see also the following ouputs:

cat /proc/interrupts
======================

CPU0 CPU1 CPU2 CPU3
0: 891022 1707693 905850 1974549 IO-APIC-edge timer
1: 1 8 0 6 IO-APIC-edge keyboard
2: 0 0 0 0 XT-PIC cascade
8: 2 0 0 0 IO-APIC-edge rtc
9: 0 0 0 0 IO-APIC-edge acpi
11: 0 0 0 0 IO-APIC-edge usb-ohci
12: 1 0 0 25 IO-APIC-edge PS/2 Mouse
14: 1 0 2 31 IO-APIC-edge ide0
18: 870521 1330974 912644 1405877 IO-APIC-level eth1, eth2
29: 32887 98520 67531 114136 IO-APIC-level eth0
30: 4047 3929 3728 5130 IO-APIC-level aic7xxx
31: 26375 64925 25715 67218 IO-APIC-level aic7xxx
NMI: 0 0 0 0
LOC:5478739 5478738 5478738 5478738
ERR: 0
MIS: 0

cat /proc/pci
=============

Bus 1, device 4, function 0:
Ethernet controller: Broadcom Corporation NetXtreme BCM5703X
Gigabit Ethernet (rev 2).
IRQ 29.
Master Capable. Latency=64. Min Gnt=64.
Non-prefetchable 64 bit memory at 0xf7ad0000 [0xf7adffff].
Bus 2, device 2, function 0:
PCI bridge: Intel Corp. 21154 PCI-to-PCI Bridge (rev 0).
Master Capable. Latency=64. Min Gnt=7.
Bus 10, device 2, function 0:
PCI bridge: Intel Corp. 21154 PCI-to-PCI Bridge (#2) (rev 0).
Master Capable. Latency=64. Min Gnt=7.
Bus 3, device 4, function 0:
Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (rev 13).
IRQ 18.
Master Capable. Latency=64. Min Gnt=8.Max Lat=56.
Non-prefetchable 32 bit memory at 0xf7bf0000 [0xf7bf0fff].
I/O at 0x4000 [0x403f].
Non-prefetchable 32 bit memory at 0xf7bc0000 [0xf7bdffff].
Bus 3, device 5, function 0:
Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (#2) (rev 13).
IRQ 18.
Master Capable. Latency=64. Min Gnt=8.Max Lat=56.
Non-prefetchable 32 bit memory at 0xf7bb0000 [0xf7bb0fff].
I/O at 0x4040 [0x407f].
Non-prefetchable 32 bit memory at 0xf7b80000 [0xf7b9ffff].

ethtool eth0,1,2
================

Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 10Mb/s
Duplex: Half
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes

Settings for eth1:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: puag
Wake-on: g
Link detected: yes

Settings for eth2:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Speed: Unknown! (65535) ????????????????????????????
Duplex: Unknown! (255) ????????????????????????????
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: puag
Wake-on: g
Link detected: no ?????????????????????????????


In the first attempt we installed a new Intel card but the eth2 is
still not working.

Any hints, tips are welcome!

TIA,
Rainer Blaes
EADS Space Transportation GmbH, Bremen (Germany)

On 11 Jul 2003 00:14:23 -0700, (E-Mail Removed) (Rainer
Blaes) wrote:

>Dear all,
>
>we have severe network problems within our environment consisting of
>
>- HP ProLiant ML370 system running SLES8, Service Pack 2a
>- Broadcom BCM5700 Gigabit on-board chip ====> eth0
>- Intel Pro 100 Dual Server card ====> eth1, eth2
>
>Here is our network configuration:
>
>eth0 Link encap:Ethernet HWaddr 00:0B:CD:6A:8B:B5
> inet addr:149.243.226.32 Bcast:149.243.226.255
> Mask:255.255.255.0
> inet6 addr: fe80::20b:cdff:fe6a:8bb5/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:180352 errors:0 dropped:0 overruns:0 frame:0
> TX packets:205001 errors:0 dropped:0 overruns:0 carrier:0
> collisions:28403 txqueuelen:100
> RX bytes:43088491 (41.0 Mb) TX bytes:119177298 (113.6 Mb)
> Interrupt:29 Memory:f7ad0000-f7ae0000
>
>eth1 Link encap:Ethernet HWaddr 00:02:B3:AC:63:45
> inet addr:192.9.205.1 Bcast:192.9.205.255
> Mask:255.255.255.0
> inet6 addr: fe80::202:b3ff:feac:6345/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1719407 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1690980 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:100
> RX bytes:451397306 (430.4 Mb) TX bytes:390577233 (372.4 Mb)
> Interrupt:18 Base address:0x4000 Memory:f7bf0000-f7bf0038
>
>eth2 Link encap:Ethernet HWaddr 00:02:B3:AC:63:46
> inet addr:192.9.206.1 Bcast:192.9.206.255
> Mask:255.255.255.0
> inet6 addr: fe80::202:b3ff:feac:6346/64 Scope:Link
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:40 dropped:0 overruns:0 carrier:40
> collisions:0 txqueuelen:100
> RX bytes:0 (0.0 b) TX bytes:16160 (15.7 Kb)
> Interrupt:18 Base address:0x4040 Memory:f7bb0000-f7bb0038
>
>We can establish any connection into the 192.9.205.0 net but trying to
>get a system in the ...206... net, for examples
>
> telnet 192.9.206.2
>
>then this appears
>
> telnet:connect to address 192.9.206.2: no route to host
>
Well, there isn't is there! Look at the stats and you'll see that,
although the interface is up, it's received no traffic ( RX bytes:0 ).
Either the cable's cut, or the port at the server or switch end is
damaged.
>This worries us since the routing table (netstat -r) seems to be o.k.:
>
> Kernel IP routing table
>Destination Gateway Genmask Flags MSS Window irtt Iface
>149.243.226.0 * 255.255.255.0 U 40 0 0 eth0
>192.9.205.0 * 255.255.255.0 U 40 0 0 eth1
>192.9.206.0 * 255.255.255.0 U 40 0 0 eth2
>default 149.243.226.1 0.0.0.0 UG 40 0 0 eth0
>
>For analysis reasons pls. also see also the following ouputs:
>
>cat /proc/interrupts
>======================
>
> CPU0 CPU1 CPU2 CPU3
> 0: 891022 1707693 905850 1974549 IO-APIC-edge timer
> 1: 1 8 0 6 IO-APIC-edge keyboard
> 2: 0 0 0 0 XT-PIC cascade
> 8: 2 0 0 0 IO-APIC-edge rtc
> 9: 0 0 0 0 IO-APIC-edge acpi
> 11: 0 0 0 0 IO-APIC-edge usb-ohci
> 12: 1 0 0 25 IO-APIC-edge PS/2 Mouse
> 14: 1 0 2 31 IO-APIC-edge ide0
> 18: 870521 1330974 912644 1405877 IO-APIC-level eth1, eth2
> 29: 32887 98520 67531 114136 IO-APIC-level eth0
> 30: 4047 3929 3728 5130 IO-APIC-level aic7xxx
> 31: 26375 64925 25715 67218 IO-APIC-level aic7xxx
>NMI: 0 0 0 0
>LOC:5478739 5478738 5478738 5478738
>ERR: 0
>MIS: 0
>
>cat /proc/pci
>=============
>
>Bus 1, device 4, function 0:
> Ethernet controller: Broadcom Corporation NetXtreme BCM5703X
> Gigabit Ethernet (rev 2).
> IRQ 29.
> Master Capable. Latency=64. Min Gnt=64.
> Non-prefetchable 64 bit memory at 0xf7ad0000 [0xf7adffff].
>Bus 2, device 2, function 0:
> PCI bridge: Intel Corp. 21154 PCI-to-PCI Bridge (rev 0).
> Master Capable. Latency=64. Min Gnt=7.
> Bus 10, device 2, function 0:
> PCI bridge: Intel Corp. 21154 PCI-to-PCI Bridge (#2) (rev 0).
> Master Capable. Latency=64. Min Gnt=7.
>Bus 3, device 4, function 0:
> Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (rev 13).
> IRQ 18.
> Master Capable. Latency=64. Min Gnt=8.Max Lat=56.
> Non-prefetchable 32 bit memory at 0xf7bf0000 [0xf7bf0fff].
> I/O at 0x4000 [0x403f].
> Non-prefetchable 32 bit memory at 0xf7bc0000 [0xf7bdffff].
>Bus 3, device 5, function 0:
> Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (#2) (rev 13).
> IRQ 18.
> Master Capable. Latency=64. Min Gnt=8.Max Lat=56.
> Non-prefetchable 32 bit memory at 0xf7bb0000 [0xf7bb0fff].
> I/O at 0x4040 [0x407f].
> Non-prefetchable 32 bit memory at 0xf7b80000 [0xf7b9ffff].
>
>ethtool eth0,1,2
>================
>
>Settings for eth0:
> Supported ports: [ TP ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> Advertised auto-negotiation: Yes
> Speed: 10Mb/s
> Duplex: Half
> Port: Twisted Pair
> PHYAD: 0
> Transceiver: internal
> Auto-negotiation: on
> Supports Wake-on: g
> Wake-on: d
> Link detected: yes
>
>Settings for eth1:
> Supported ports: [ TP MII ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> Advertised auto-negotiation: Yes
> Speed: 100Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 1
> Transceiver: internal
> Auto-negotiation: on
> Supports Wake-on: puag
> Wake-on: g
> Link detected: yes
>
>Settings for eth2:
> Supported ports: [ TP MII ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> Supports auto-negotiation: Yes
> Speed: Unknown! (65535) ????????????????????????????
> Duplex: Unknown! (255) ????????????????????????????
> Port: Twisted Pair
> PHYAD: 1
> Transceiver: internal
> Auto-negotiation: on
> Supports Wake-on: puag
> Wake-on: g
> Link detected: no ?????????????????????????????
>
>
>In the first attempt we installed a new Intel card but the eth2 is
>still not working.
>
>Any hints, tips are welcome!
>
>TIA,
>Rainer Blaes
>EADS Space Transportation GmbH, Bremen (Germany)