PPPoE and fragmentation

Hi folks.

I have a DSL connection from AT&T (was SBC, sort of still is in a sense I
guess). I'm using a SpeedStream 5260 DSL modem, and a Linksys WRT54G
running OpenWRT for wireless, switch, firewall and doing the PPPoE
processing.

From time to time, my connections get stuck. Especially ssh connections,
but also AIM and HTTP sometimes too. If I try to rerun an ssh connection
enough times, it'll eventually be good for the long term, but that's a
pain.

I've heard on a number of occasions that PPPoE cannot do packet
fragmentation correctly, so one just needs to reduce one's MTU to
something like 1400, to be rid of the problem.

And I'm finding that this works, but only to a point. I've got my linux
box, and my WRT54G set up with an MTU of 1400, and it works fine for a few
weeks, but then it stops working again. Reboot the SpeedStream, followed
by a reboot of the WRT54G, and all is well again - for another couple of
weeks.

I check the MTU's on my linux box and the WRT54G, and they're both still
1400 during the trouble.

What's the deal here? Why does it keep forgetting how to do a proper TCP
session?

Thanks!

Dan Stromberg wrote:
>
> [snip]
>
> I've heard on a number of occasions that PPPoE cannot do packet
> fragmentation correctly, so one just needs to reduce one's MTU to
> something like 1400, to be rid of the problem.
>
> And I'm finding that this works, but only to a point. I've got my linux
> box, and my WRT54G set up with an MTU of 1400, and it works fine for a few
> weeks, but then it stops working again. Reboot the SpeedStream, followed
> by a reboot of the WRT54G, and all is well again - for another couple of
> weeks.
>
> I check the MTU's on my linux box and the WRT54G, and they're both still
> 1400 during the trouble.
>
> What's the deal here? Why does it keep forgetting how to do a proper TCP
> session?

PPPoE can do fragmentation just fine, but fragmentation is usually to be
avoided. It's not uncommon for firewalls to block packet fragments.

You only need to reduce your MTU to 1492. The PPP header is 8 bytes and
counts as part of the 1500 IP bytes, leaving 1492 of "real" data.

What can also sometimes happen is that people get too freaked out by
ICMP, which is just what it is, Internet Control Message Protocol. If a
packet is too big to go through a device, the device sends an ICMP error
back to the sender. If the sender is blocking ICMP, he never sees the
error, thus never figuring out he needs to reduce his MTU. Whatever
combination of HW and SW you've got, make sure MTU Discovery isn't
getting blocked (ICMP Type 3).

FWIW, I set MTU to 1492 only on my PPPoE connection, letting all the
other devices on my internal net figure out they need to scale back to
1492 on their own for talking to the Internet.

HTH

On Sun, 23 Jul 2006, Dan Stromberg <(E-Mail Removed)> wrote:
>
> Hi folks.
>
> I have a DSL connection from AT&T (was SBC, sort of still is in a sense I
> guess). I'm using a SpeedStream 5260 DSL modem, and a Linksys WRT54G
> running OpenWRT for wireless, switch, firewall and doing the PPPoE
> processing.

Not sure what was different with the 5260 (maybe no auto VCI/VPI, or is it
VPI/VCI?). But I have been using a 5360 on Ameritech, then SBC, now ATT
dynamic ADSL for years. When a hardware router was doing PPPoE, the only
time I had any problem with mtu was when "receiving" smtp on a mail server
behind the router, and the only adjustment I had to do to make it work was
to adjust LAN eth mtu of the sendmail server to mtu 1492 (with ifconfig).

Since I have been using Linux as pppoe/firewall/router (an old Celeron 300
box), I left its nic to modem at default mtu 1500 (so pppoe would be 1492)
and set its LAN nics to mtu 1492 in my network scripts (one nic to wired
LAN and another nic to WAP11 proxy_arp wireless subnet of main LAN). No
problems at all. Everything works to internet without any mtu settings on
other LAN boxes, and local LAN traffic ends up mtu 1500 despite mtu
settings on router LAN nics.

I do not know what settings WRT54G has, but its PPPoE should be mtu 1492,
and you should likely set mtu 1492 on LAN nic of any server behind it
(that has external ports or DMZ forwarded to it).

Do you have keepalive set for ssh? If you get dropped due to PPPoE idle
timeout setting or from peer (ATT), it would throw a wrench into ssh,
since you would end up with new public IP (if dynamic). On occasion I have
left ssh from a LAN box connected to my old Unix ISP overnight and it
still worked the next morning (pppoe idle timeout = 0, ie, never).

To find my dynamic IP for incoming ssh, I use no-ip.com dynamic DNS
(updated from /etc/ppp/ip-up anytime my Linux router connects).

On Sun, 23 Jul 2006, David Efflandt <(E-Mail Removed)> wrote:
> On Sun, 23 Jul 2006, Dan Stromberg <(E-Mail Removed)> wrote:
>>
>> I have a DSL connection from AT&T (was SBC, sort of still is in a sense I
>> guess). I'm using a SpeedStream 5260 DSL modem, and a Linksys WRT54G
>> running OpenWRT for wireless, switch, firewall and doing the PPPoE
>> processing.

I just thought of one other thing that can interfere with a connection.
I sometimes have trouble with Dlink 530TX+ nic (supplied by SBC) when
using Linux 8139too module (nic goes into a frenzy with packet loss,
interrupting network traffic). I did not think of that until after I had
trouble after shutting down my Linux router during prolonged power loss
(normally on 24/7 w/UPS).

Since I have 2 of those nics, I made a script that slows them down to
10baseT (I should automate it, but currently just run from root home):

/sbin/mii-tool -A 10baseT-FD,10baseT-HD eth0
/sbin/mii-tool -A 10baseT-FD,10baseT-HD eth1

After that, everything works fine and mii-tool returns:

eth0: negotiated 10baseT-FD, link ok
eth1: negotiated 10baseT-FD, link ok
eth2: negotiated 100baseTx-FD, link ok

So it is possible that you may have a glitch with a nic or its module if
you get any constant flickering LED's and/or more than rare packet loss.