tc and bandwidth limiting: What am I doing wrong?

Discussion in 'Linux Networking' started by Dances With Crows, Jun 14, 2008.

  1. So I thought I'd limit the bandwidth on one of my network interfaces to
    run a few tests on a few things. I tried:

    tc qdisc add dev eth1 root tbf burst 20480 limit 20480 \
    mtu 1514 rate 32000bps

    ....as was mentioned in the Traffic Control HOWTO. However, sustained
    transfer rates weren't 64K/sec, but were 15K/sec. Doubling and
    quadrupling the numbers for rate, burst, and limit made no difference to
    the sustained transfer rates I observed.

    Kernel 2.6.22.1 (vanilla), iproute2 2.6.22.20070710 . Is the HOWTO
    outdated? Am I doing something stupid? Is there an easier way? The
    machines are using forcedeth and e1000 cards, and sustained transfer
    rates without tc are ~10M/sec. If you need more information, say what
    you need and I'll find it. Thanks in advance.

    (I ended up limiting bandwidth with an old 10bT DEC Tulip card I had
    lying around. Yay for obsolete hardware, I guess.)
     
    Dances With Crows, Jun 14, 2008
    #1
    1. Advertisements

  2. It's hard to figure out what exactly happened. You claim you set a
    traffic limit of 32,000 bits per second, and expected a transfer rate
    of 64,000 somethings per second (bits? bytes?) but got 15,000. Why
    would you expect 64,000 if you asked for 32,000? No conceivable bits/
    bytes conversion can make that work.

    What was the command you used? What was the rate you got (and specify
    if it's in bits or bytes) and what did you expect? And how did you
    measure it?

    The problem may be that you are confusing bits with bytes. It also may
    be that you are expecting *application* cooked data throughput to
    equal raw network throughput.

    It's hard to tell though because your command seems not to match any
    expectation, with or without such errors.

    DS
     
    David Schwartz, Jun 15, 2008
    #2
    1. Advertisements

  3. David Schwartz staggered into the Black Sun and said:
    That's 64K/sec (kilobytes) expected, got 15K/sec. (Actually I should
    have expected 32K/sec, but 32 still != 15.)
    See first paragraph. That was the exact syntax given in the HOWTO for
    limiting an interface to 256 kbit/second. All right, they might have
    gotten their math wrong, but I usually expect HOWTOs to be reasonably
    accurate. Please also note that removing the rule, then replacing it
    with the numbers doubled, quadrupled or *ed by 8 for burst, limit, and
    rate produced the same measured 15K/sec. I said that in the first
    message, but it's worth repeating.
    Sustained transfer rate using scp.
    The overhead isn't *that* high with scp, and see above.
    That's sort of why I posted here, since what I saw is so different from
    what I expected. So: Who's using tc, how are you using it, and how
    do you make it work the way you want it to?
     
    Dances With Crows, Jun 15, 2008
    #3
  4. Dances With Crows

    Andy Furniss Guest

    Just to make things even more confusing TC reads bps as bytes/sec you
    have to use bit/kbit/mbit for bits/sec.

    When using tc on gig nics you really need to turn off tcp segmentation
    offload with IIRC ethtool -k. With it on you can get strange results.

    When you put qdiscs on the root of eths you need to consider arp - these
    will be delayed/dropped as well - which can mess things up.

    Andy.
     
    Andy Furniss, Jun 15, 2008
    #4
  5. One last clarification -- changing the configured 'tc' *rate* had no
    effect on the transfer speed?!

    DS
     
    David Schwartz, Jun 15, 2008
    #5
  6. David Schwartz staggered into the Black Sun and said:
    The bits in my notes here say that:

    tc qdisc add dev eth1 root tbf burst 131122 limit 131122 mtu 1514 \
    rate 2012144

    gave the same results as--

    tc qdisc add dev eth1 root tbf burst 262244 limit 262244 mtu 1514 \
    rate 4024288

    which gave the same results as--

    tc qdisc add dev eth1 root tbf burst 524488 limit 524488 mtu 1514 \
    rate 8048576

    ....which is why I said that I was using 2.6.22.1 (vanilla) and iproute2
    2.6.22.20070710 , just in case there was some sort of stupid bug in
    either/both that was fixed in later revisions.
     
    Dances With Crows, Jun 15, 2008
    #6
  7. Andy Furniss staggered into the Black Sun and said:
    eth1 in this case was a forcedeth CK804 10/100 card. The other
    machine's eth0 is an e1000, but since the tc'ing is being done on the
    machine with the forcedeth, I didn't think that would make a difference.
    Maybe it does. I can check later.
    This could have screwed things up as well. However, I tried this as
    well:

    tc qdisc add dev eth1 handle 1:0 root dsmark indices 1 default_index 0
    tc qdisc add dev eth1 handle 2:0 parent 1:0 tbf burst 20480 limit 20480
    mtu 1514 rate 32000bps

    ....and got exactly the same results. Yeah, I'm confused, which is why I
    posted here, hoping to find someone who knows more about tc than I do.
     
    Dances With Crows, Jun 15, 2008
    #7
  8. Dances With Crows

    Andy Furniss Guest

    Strange - I have a forcedeth 100mbit nic and just tried your tbf rule
    from the first post and it works OK.

    Could be unlucky kernel version I suppose, or maybe timers, if you have
    multicore/smp you shouldn't use tsc. I haven't tested noHZ on the box I
    shape on I still use Hz=1000.

    Does htb work eg -

    tc qdisc del dev eth1 root

    Forgetting to do this between tests can sometimes cause weirdness

    tc qdisc add dev eth1 root handle 1:0 htb
    tc class add dev eth1 parent 1:0 classid 1:1 htb rate 256kbit
    tc qdisc add dev eth1 parent 1:1 bfifo limit 20k
    tc filter add dev eth1 protocol ip parent 1:0 u32 match u32 0 0 classid 1:1

    It's not quite the same - ie it doesn't catch arp and I didn't use
    burst/cburst because it doesn't seem to behave the same as burst on tbf.
     
    Andy Furniss, Jun 15, 2008
    #8
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.