MTU size for a 3g mobile broadband dongle

Discussion in 'Linux Networking' started by Mark Hobley, Sep 27, 2010.

  1. Mark Hobley

    Mark Hobley Guest

    My LAN is ethernet based. However, for connectivity to the internet, I have
    an Edimax 3g-6200n router, which connects to the internet via a 3g mobile
    broadband stick.

    Can a 3g mobile broadband stick route the 1500 byte packets arriving from
    the LAN without fragmentation, or should I reduce the size of the MTU for
    outbound traffic? If so, what should I reduce it to? (My internet service
    provider is Three Mobile, if that matters.)

    My next question is ...

    Will Linux allow me to configure two MTU sizes on a single network interface,
    depending on whether or not the destination address is local or remote?

    If I need to reduce the MTU for outbound traffic, can I keep using a 1500
    byte MTU for LAN traffic?

    (The computers have only one ethernet connection onto a single subnet. There
    is a single ethernet based router (Netgear) before the Edimax unit, which
    forwards internet bound traffic via the Edimax unit).

    Mark.
     
    Mark Hobley, Sep 27, 2010
    #1
    1. Advertisements

  2. Mark Hobley

    Rick Jones Guest

    No, but you can manually introduce PMTU routes up in your routing
    table to accomplish that task.

    rick jones
     
    Rick Jones, Sep 27, 2010
    #2
    1. Advertisements

  3. Mark Hobley

    Jorgen Grahn Guest

    ["Followup-To:" header set to comp.protocols.tcp-ip.]

    Yeah.

    I can add that there's most likely at least one bottleneck: at one
    point your packets go over GTP over UDP over IP, on Ethernet.
    *Someone* will get hurt if you use 1500-octet packets -- possibly the
    operator.
    You've mentioned this several times recently. What about PMTU
    discovery -- doesn't that work for you? It seems to work well enough
    (or never kick in) for me.

    /Jorgen
     
    Jorgen Grahn, Sep 27, 2010
    #3
  4. Mark Hobley

    Andy Furniss Guest

    Telling us what the somewhere he pinged has set their mtu to may be a
    bit misleading as to the nature of his ISP settings, though.
     
    Andy Furniss, Sep 28, 2010
    #4
  5. Mark Hobley

    Andy Furniss Guest

    Don't know, but if it works OK for all sites then I guess so.

    That's a bit oversimplistic because I have no idea about 3G devices, eg.
    they may be doing some mss clamping that reduces the size of packets in
    tcp connections.

    Tcpdump/wireshark etc will show this - but don't just rely on looking at
    one connection as you may be just seeing how they have set things up
    rather than learning about your ISP.

    I have seen threads where people claim better results by using smaller
    MTU due to smaller packets having more chance of getting through rf
    noise type loss, so experimenting if you have problems could improve
    things - if you don't have problems I wouldn't bother.

    Like Rick has said you can do things with ip route. Another way would be
    to use iptables to mss clamp non local tcp connections.
    A quick test I just did using ip route shows it is possible to just
    modify the default route, which I assume is all you would need.

    Normally if you set mtu on eth then the tcp advertised mss will also
    adjust so that incoming tcp packets are also limited in size. Using ip
    route this doesn't happen, but you can use an extra commant to specify.

    Testing on this PC which already has a default route setup to
    192.168.0.1, the command -

    ip route change dev eth0 default via 192.168.0.1 mtu 1000 advmss 960

    seems to do the trick and leaves traffic to local net unchanged.
     
    Andy Furniss, Sep 28, 2010
    #5
  6. Mark Hobley

    Andy Furniss Guest

    Resent - forgot to re-add comp.os.linux.networking
    They shouldn't get hurt if they know what they are doing - most recent
    ISP type kit should be able to run MTU > 1500 to handle this.

    In the UK alot of ISPs use the dominant teleco for transport and they
    use L2TP - and explicitly state in the specs that the ISP must run a
    higher MTU to allow for this.
    Does your ISP restrict your inbound to < 1500, though. If it doesn't you
    are not likely to have problems.

    In the case of pppoe type ISPs the 1492 they (may) run is overcome by
    the fact that consumer dsl modem/routers mss clamp TCP avoids PMTUD
    black hole faliures.
     
    Andy Furniss, Sep 28, 2010
    #6
  7. Mark Hobley

    Rick Jones Guest

    True, the reply coming-back fragmented means either the remote, or the
    ISP(s) in the middle had a smaller-than-echo-size MTU. However, if
    the reply comes-back unfragmented we know that both the remote and
    ISP(s) in the middle had an MTU of at least that size.

    rick jones
     
    Rick Jones, Sep 28, 2010
    #7
  8. Mark Hobley

    Mark Hobley Guest

    What is the simplest way to send a max-sized ping packet? Is there
    something I can type at the command line to do this?

    Mark.
     
    Mark Hobley, Sep 29, 2010
    #8
  9. Mark Hobley

    Andy Furniss Guest

    ping -s 1472 will send a 1500 packet (1472 + 8 bytes ICMP header + 20 IP
    header)
     
    Andy Furniss, Sep 29, 2010
    #9
  10. Mark Hobley

    Mark Hobley Guest

    Right. I didn't realize that ping could carry a payload. Cheers, my results
    are as follows:

    On a system with PMTUD enabled, the maximum value for the -s switch is 1384:

    ping -s 1384 www.yahoo.com
    PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1384(1412) bytes of data.
    1392 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_seq=2 ttl=45 time=305 ms

    If I disable PMTUD, I can use a larger maximum value as follows:

    echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc

    ping www.yahoo.com -s 1472
    PING any-fp.wa1.b.yahoo.com (67.195.160.76) 1472(1500) bytes of data.
    1480 bytes from ir1.fp.vip.ac4.yahoo.com (67.195.160.76): icmp_seq=1 ttl=45 time=1455 ms

    So, am I right in thinking that datagrams bigger than 1412 cannot be carried
    without fragmentation, and that I would be better to reduce my MTU to a value of
    1412 for externally bound traffic?

    Mark.
     
    Mark Hobley, Sep 29, 2010
    #10
  11. Mark Hobley

    Mark Hobley Guest

    On Wed, 29 Sep 2010 17:16:12 +0000, Mark Hobley wrote:

    I have some more weirdness now. If I ping from one host to another, connected
    via ethernet, I would expect a maximum datagram size of 1500. However, I
    actually get a result of 65535, as follows:

    # PMTUD is disabled on both hosts:
    echo 0 > /proc/sys/net/ipv4/ip_no_pmtu_disc

    ping -s 65507 miranda
    PING miranda.markhobley.yi.org (10.0.0.30) 65507(65535) bytes of data.
    65515 bytes from miranda.markhobley.yi.org (10.0.0.30): icmp_seq=1 ttl=64
    time=13.8 ms

    Does this mean that jumbo frames have become enabled on my network?

    Mark.
     
    Mark Hobley, Sep 29, 2010
    #11
  12. Mark Hobley

    Jorgen Grahn Guest

    ["Followup-To:" header set to comp.protocols.tcp-ip.]

    No, it means you should install and use[1] tcpdump to see what this
    actually translates to on the Ethernet level. Hint: IP fragmentation.

    /Jorgen

    [1] If you have root access on either machine, that is.
     
    Jorgen Grahn, Sep 29, 2010
    #12
  13. Mark Hobley

    Rick Jones Guest

    I guess there is an opportunity to fix the "linux" ping manpage then -
    that would be 64 IP data bytes when the 56 bytes of ICMP data are
    added to the 8 bytes of ICMP header...

    Since comp.protocols.tcp-ip is included, here is how for the ping
    which ships with HP-UX :)

    SYNOPSIS
    ping [-oprv] [-f address-family] [-i address] [-I interval] [-t ttl]
    host [-n count [-m timeout]]

    ping [-oprv] [-f address-family] [-i address] [-I interval] [-t ttl]
    host packet-size [ [-n] count [-m timeout]]

    rick jones
     
    Rick Jones, Sep 29, 2010
    #13
  14. Mark Hobley

    Mark Hobley Guest

    Right. I have done a tcpdump, at first examining the internet bound stuff:

    With PMTUD enabled:

    21:13:47.557102 IP venus.markhobley.yi.org > ir1.fp.vip.ac4.yahoo.com: ICMP echo request, id 52485, seq 4, length 1392
    21:13:47.811766 IP ir1.fp.vip.ac4.yahoo.com > venus.markhobley.yi.org: ICMP echo reply, id 52485, seq 4, length 1392

    With PMTUD disabled:
    21:19:24.453588 IP venus.markhobley.yi.org > ir1.fp.vip.re1.yahoo.com: ICMP echo request, id 58885, seq 6, length 1480
    21:19:24.976825 IP ir1.fp.vip.re1.yahoo.com > venus.markhobley.yi.org: ICMP echo reply, id 58885, seq 6, length 1480

    So, with PMTUD disabled, I can send and receive packets of 1480 to and from yahoo,
    presumably fragmentation and reassembly is taking place en-route and I cannot see this.

    With PMTUD enabled, my maximum packet size if 1392.

    Now, if PMTUD is disabled, why do I now have a limit of 1480? I would have expected packets
    larger than this to become fragmented and transmitted successfully.

    This actually does happen on the LAN. Here are those 65535 long being fragmented at
    1480 for transportation over ethernet:

    21:34:00.635725 IP venus.markhobley.yi.org > miranda.markhobley.yi.org: ICMP echo request, id 64005, seq 2, length 1480
    21:34:00.635760 IP venus.markhobley.yi.org > miranda.markhobley.yi.org: icmp

    blah blah blah ...

    21:34:00.640468 IP venus.markhobley.yi.org > miranda.markhobley.yi.org: icmp
    21:34:00.643685 IP miranda.markhobley.yi.org > venus.markhobley.yi.org: ICMP echo reply, id 64005, seq 2, length 1480
    21:34:00.643804 IP miranda.markhobley.yi.org > venus.markhobley.yi.org: icmp

    blah blah blah ...

    21:34:00.648989 IP miranda.markhobley.yi.org > venus.markhobley.yi.org: icmp
    21:34:04.638350 ARP, Request who-has venus.markhobley.yi.org tell miranda.markhobley.yi.org, length 46
    21:34:04.638376 ARP, Reply venus.markhobley.yi.org is-at 00:11:2f:bc:df:19 (oui Unknown), length 28

    I guess upstream have set a limit of 1480, even though they have to fragment to carry this.

    There is another strange thing I noticed. Why does miranda make an arp request for venus at
    the end of the exchange? I would have thought that miranda already knows the mac address
    of venus, because she has just been communicating with it.

    Mark.
     
    Mark Hobley, Sep 29, 2010
    #14
  15. Mark Hobley

    Andy Furniss Guest

    Remember - don't just test one address, try the f.root-servers.net that
    Morten suggested.
    If you use the -v switch for tcpdump you will see whether the replies
    have DF set or not.
    That 1480 is IP data so add 20 = 1500 it's a bit hit or miss whether
    you'll get a server on the internet to answer to a > 1500.

    ICMPs with DF set aren't the same as other packets with DF set because
    AIUI ICMP packets aren't allowed to generate further ICMPs so there will
    be no PMTUD to help them get through.

    I would have expected packets
    target doesn't answer tcpdump should show you outgoing fragged echo request.
     
    Andy Furniss, Sep 30, 2010
    #15
  16. Mark Hobley

    Rick Jones Guest

    internet is broken.
    Yet... unless the routing changes based on DF being set, if the max
    before things stopped working with DF set was 13whatever, shouldn't
    the responses for the DF clear and 14whatever have come back
    fragmented? Or is the hypothesis that the brokenness is on the way
    out?

    rick jones
     
    Rick Jones, Sep 30, 2010
    #16
  17. Mark Hobley

    Rick Jones Guest

    If it happens all the time at the end of the exchange, that would be
    odd. If you just see it once and a while, that stems from ARP
    maintaining a cache, and periodically refreshing the entries. ARP
    does not see or know that traffic is being exchanged successfully,
    thus the periodic refreshes.

    rick jones
     
    Rick Jones, Sep 30, 2010
    #17
  18. Mark Hobley

    Rick Jones Guest

    I think there must be an exception in there somewhere - until it was
    found to be a vector for an amplification attack, years ago HP-UX did
    a "hybrid" PathMTU discovery by default. When it first started
    talking to a remote IP, it would send IP datagrams with DF cleared -
    however, at the same time, it would send an ICMP Echo Request to the
    remote with the DF bit set. It would lather, rinse and repeat until
    it could get the Echo reply without generating any datagram too big
    messages. I believe that was to "deal" with black holes where the
    ICMP Datagram Too Big messages were being filtered.

    rick jones
     
    Rick Jones, Sep 30, 2010
    #18
  19. Mark Hobley

    Andy Furniss Guest

    Ugh, yes, I really should have tested that rather than relying on
    something I once read.

    On my LAN with vanilla Linux kernel settings I can indeed get a frag
    needed in response to an echo request.
     
    Andy Furniss, Sep 30, 2010
    #19
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.