Network Puzzle with 2 NICs

Discussion in 'Linux Networking' started by circuit_breaker, Apr 9, 2004.

  1. Hi,

    "Now you see it, now you don't". I have 2 PCs connected on a HUB.
    One is a Windows box, the other one is a RH 9.1.

    The Setup:
    ----------
    PC1:
    RH9.1
    eth0 (e100) 192.168.0.100, 255.255.255.0
    eth1 (3c59x) 192.168.0.101, 255.255.255.0

    PC2:
    Windows, one NIC, 192.168.0.10, 255.255.255.0

    The Test:
    ---------
    I can ping any nic from any other nic. If I type "ifdown eth0",
    Windows can no longer ping eth0 but it can PING eth1 and eth1 can PING
    Windows. Same thing with "ifdown eth1". Life is good.

    The Problem:
    ------------
    If I disconnect the eth1 network cable, I can't PING any NIC from
    anywhere. Strange, this should be the same result as "ifdown eth1"
    right? So I put back the eth1 network cable and I disconnect eth0.
    This time I can PING any NIC from anywhere. Geeezzz

    Question: Why? Is this something about "routing tables"? Any help
    will be appreciated.

    Thanks.
     
    circuit_breaker, Apr 9, 2004
    #1
    1. Advertisements

  2. circuit_breaker

    Arun Dev Guest

    I bet 2 cents
    o
    You have two interfaces on two physical networks but they belong
    to the _same_ logical network 192.168.0.0/24.

    For that to work your PC1 has to be a _bridge_
    I'm not saying that this is impossible, but need more knowledge
    about networking ;-)

    My suggestion create two logical networks like 192.168.0.0/24 and
    192.168.1.0/24

    Have fun!

    Arun
     
    Arun Dev, Apr 9, 2004
    #2
    1. Advertisements

  3. circuit_breaker

    jack Guest

    Please send the routing tables of Your linux box ("route -n"), with
    both interfaces activated.

    You run Yourself into trouble when connecting two NICs in one box to
    the same logical subnet.

    In Your case, I predict that if You pull the cable _and_ ifdown eth1,
    You will be able to ping eth0 again.

    The reason for that is simple: When adding a NIC with ifconfig, the
    kernel routing table will be updated accordingly. Say, when You add
    one NIC with IP 192.168.0.100/24, You will get a route to that subnet
    via that NIC. So far, so good. - If now You add a second NIC with, say,
    192.168.0.101/24, You'll get the same result, i. e. You now have two
    routes to that same subnet. But, only the first matching rule in the
    table will be used by the kernel (with no respect to whether the link
    is functional or not).

    What You're seeing is, in fact, that when both NICs are up and connec-
    ted, Your ICMP echo request ("ping") will get to the respective inter-
    face in Your linux box, but from there, the ICMP echo reply ("pong")
    will always come from one and the same NIC, namely that one that the
    route is set for first. - So even if You ping eth0, the reply may come
    from eth1, and vice versa.


    Cheers, Jack.
     
    jack, Apr 9, 2004
    #3
  4. Do you mean that each NIC should be on its own network? The subnet
    mask is not enough specific in that case?

    Thanks.
     
    circuit_breaker, Apr 9, 2004
    #4
  5. Here's my routing tables:
    -------------------------
    Destination Gateway Genmask Flags Metric Ref
    Use Iface
    192.168.0.0 0.0.0.0 255.255.255.0 U 0 0
    0 eth1
    192.168.0.0 0.0.0.0 255.255.255.0 U 0 0
    0 eth1
    169.254.0.0 0.0.0.0 255.255.0.0 U 0 0
    0 eth1
    127.0.0.0 0.0.0.0 255.0.0.0 U 0 0
    0 lo
    0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0
    0 eth1

    This morning, it behaves differently. While both NICs are active,
    "ifdown eth1" makes both interfaces unable to communicate with PC2.
    Then I did "ifdown eth0" and of course, still no reply from any
    interface, and then "ifup eth1". Guess what, eth1 was replying... :)

    Looks like there's something to add or remove in those routing tables.
    Thanks for your help
     
    circuit_breaker, Apr 10, 2004
    #5
  6. circuit_breaker

    jack Guest

    Is that Your routing table? Did You "cut and paste"?

    See that there is a duplicate entry at the top. So everything that goes
    into that 192.168.0/24 subnet will be sent via eth1. If the second entry
    had eth0 as device, it would not change things because only the first
    matching entry will be used. From the kernels point of view: It gets a
    packet that shall be sent to, say, 192.168.0.20. The kernel will look at
    its routing table and find that 192.168.0/24, which .20 belongs to, can
    be accessed via eth1. Good, so that packet will be shoved out via eth1,
    done. All other entries in that routing table will not be used, because
    in a properly configured system, there's exactly one route to any given
    destination. So if one route matches the target, all other rules will
    not match, so there's no need to even bother.

    Yes, have a look at the routing table after taking down eth1. Normally,
    ifconfig will remove all routes from the table that refer to the inter-
    face that is being taken down. In Your case, all routes to 192.168.0/24
    plus the default route. Since there was no route to Your subnet via eth0
    anymore, that subnet is unreachable for Your box. You still can receive
    incoming packets via eth0, no problem, but You cannot send replies. Bad
    thing in a two-way communication...

    Normally, when bringing up a NIC, a route will be added to the routing
    table based on the information You provide to configure the card. If
    You specify its IP to be 192.168.0.100, and the netmask 255.255.255.0,
    this information is sufficient to know that everybody in that subnet
    can be reached via that respective NIC.

    Try this: After adding or removing ("up"-ing and "down"-ing) the NICs,
    do a "route -n" as a first thing. Try to find how the kernel will try
    to send data into that subnet of Yours. Then, generate such traffic and
    see whether Your guess was right or not.

    Again, if the route to Your subnet is through eth1, You cann still send
    things to eth0 from that other box. In any case, the reply will come
    from eth1, and hence things will fail when You pull the plug from eth1.
    eth0 will not send away anything at all, except for ARP, perhaps.


    Hope that clears things a bit, Jack.
     
    jack, Apr 10, 2004
    #6
  7. You did more than I expected. It makes sense to me now, Thanks a lot!
     
    circuit_breaker, Apr 11, 2004
    #7
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.