Frustrated that I don't UNDERSTAND why my network times out

Discussion in 'Linux Networking' started by billy, Oct 8, 2013.

  1. billy

    Chris Davies Guest

    Yes and no.

    The data for a website is carried over TCP. The control commands
    (e.g. "slow down you're filling up the pipe", or "that packet's waayyy
    too big; make it smaller") are sent over ICMP. Pings and some traceroute
    programs also use ICMP, but they use a different ICMP message type.

    A competently configured firewall might be set up to block ICMP ping
    requests. But there's no way that same firewall should be completely
    and naively blocking all ICMP packets.
     
    Chris Davies, Oct 9, 2013
    #21
    1. Advertisements

  2. billy

    Chris Davies Guest

    That's a good starting point, yes. Now you need to see whether your web
    browser can access centos.org. If it does, increase the MTU until it
    breaks and then back it off a little.

    If it still can't access centos.org, even though you've got your MTU
    down at 500 then the chances are that this suggestion is inappropriate
    for your situation.

    Chris
     
    Chris Davies, Oct 9, 2013
    #22
    1. Advertisements

  3. billy

    Mike Easter Guest

    TCP. But the 'process' involves TCP/IP.

    It would be of value for you to learn about the protocols.

    http://en.wikipedia.org/wiki/Internet_protocol_suite

    The article breaks down the different layers of the suite.

    But your 'breakdown' is more than just the browser, so you should also
    understand the protocols used by your tools of investigation.

    Standard ping = ICMP (also there is UDP pinger)
    traceroute and tracert also can come in 'flavors' ICMP, UDP, and TCP

    Some hop obstructions or 'holes' are related to protocol.
     
    Mike Easter, Oct 9, 2013
    #23
  4. billy

    Tauno Voipio Guest


    PLEASE do get a text on TCP, read and understand it.
    It explains the usage.

    ICMP is a helper protocol inside the TCP/IP protocol suite.
    TCP uses ICMP to detect a suitable segment (transmission
    block) size.

    Your web browser needs UDP for name resolution, TCP for
    page transfer and ICMP to help TCP work, and all run
    using IP.

    Can you ping the target from your computer?

    It works perfectly from here:

    --- clip clip ---

    tauno-voipios-macbook-pro-2:~ tauno$ ping centos.org
    PING centos.org (72.232.194.162): 56 data bytes
    64 bytes from 72.232.194.162: icmp_seq=0 ttl=52 time=161.925 ms
    64 bytes from 72.232.194.162: icmp_seq=1 ttl=52 time=164.736 ms
    c64 bytes from 72.232.194.162: icmp_seq=2 ttl=52 time=161.303 ms
    64 bytes from 72.232.194.162: icmp_seq=3 ttl=52 time=165.196 ms
    64 bytes from 72.232.194.162: icmp_seq=4 ttl=52 time=165.483 ms
    ^C

    --- clip clip ---

    I can also access the web site with a browser.
     
    Tauno Voipio, Oct 9, 2013
    #24
  5. billy

    unruh Guest

    BEcause you were also worried about the problem of traceroute failing
    and/or ping failing.

    You could try
    telnet centos.com 80
    (or whatever the address is)
    to see if there is any response
    (QUIT should get you out)

    eg
    info:0.0[unruh]>telnet centos.com 80
    Trying 87.106.187.200...
    Connected to centos.com (87.106.187.200).
    Escape character is '^]'.
    QUIT
    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>501 Method Not Implemented</title>
    </head><body>
    <h1>Method Not Implemented</h1>
    <p>QUIT to /index.html not supported.<br />
    </p>
    </body></html>
    Connection closed by foreign host.
     
    unruh, Oct 9, 2013
    #25
  6. billy

    unruh Guest

    You have connection to the web. That is not the problem since you say
    you can connect to other places. There is something about centos.org
    that is behaving weirdly. Now it could be something on your end, and it
    is almost certainly something on their end. They may have decided that
    you are a pain in the ass and blackballed you (I have no idea why), or
    it might be some sort of technical incompatibility between you and them.
    The only way to clear it up is to talk to them about it. If you cannot
    then I cannot see how you are going to figure it out.
    Note you could take your computer with you to some other place which
    connects to the web in some other way than your microwave link and see
    if that works there.
     
    unruh, Oct 9, 2013
    #26
  7. billy

    Rick Jones Guest

    The protocols layer and in some cases intertwine. What you call "the
    web" sits on top of a protocol called HTTP (HyperText Transfer
    Protocol). HTTP makes use of the services of TCP (Transmission
    Control Protocol). TCP makes use of IP (Internet Protocol). IP will
    make use of whatever link-layer protocol is available in its
    sitaution. For your end systems that will be "Ethernet" protocol over
    your WiFi connection.

    HTTP
    TCP
    IP
    Ethernet/WiFi/whatnot

    Within/alongside IP is ICMP (Internet Control Message Protocol). ICMP
    is used to provide some "Hey, you should know this" kinds of messages
    back to traffic sources.

    So, simplifying a bit, when you try to access www.centos.org from your
    web browser, the browser will generate an HTTP message the browswer
    wants to get to HTTP at www.centos.org. That HTTP message will be
    handed-off to TCP to transmit in one or more TCP segments to TCP at
    www.centos.org (segment = what a packet is called in the context of
    TCP). The TCP segment(s) will be handed off to IP, which will
    encapsulate them in one or more IP datagrams (datagram = what a packet
    is called in the context of IP) destined to the IP address of
    www.centos.org. IP will then hand its datagrams to the "data link"
    layer (eg Ethernet etc) which will do its encapsulation. It will then
    go across the data link destined for IP at the "next hop" where the
    data-link layer message will decapsulate the IP datagram and hand it
    to IP at that hop. IP at that hop will then decide what to do with
    the datagram, which is being sent to the/an IP address for
    www.centos.org. There will probably be another "next hop" and so on
    until the traffic arrives at www.centos.org. IP at www.centos.org
    will then decapsulate the TCP segment from the IP datagram, and TCP
    will decapsulate the HTTP message and hand it to the HTTP server.

    The HTTP response will flow back to you via a similar mechanism.

    Those different lines you see in the output of traceroute are the
    "next hops" along the way from your system to www.centos.org.

    Now, the data links at all those next hops may have different limits
    for how large a packet they can transmit. If an IP datagram arrives
    at a "next hop" (aka an IP router), and IP there determines that where
    it needs to be sent-out next is over a data link with a packet size
    smaller than the IP datagram it needs to send, IP decides to send back
    to the traffic's source an ICMP message saying "Hey, the datagram you
    sent is too large for me to send, you should send smaller IP
    datagrams" (There is more to it than this, but I'm simplifying).

    There are other sorts of ICMP "Hey!" messages. One of them is
    "Maximum Hop Count Reached" - that is the ICMP message that (default)
    traceroute relies upon receiving. In the header of an IP datagram
    there is a field called the "TTL" (Time To Live) which is really a
    maximum hop count. As the IP datagram passes through an IP router,
    the router decrements the count by one, and when it hits zero, that IP
    is supposed to send back an ICMP message that says "Hey! The datagram
    you sent had its maximum hop count expire before it reached its
    destination." Traceroute works by sending an IP datagram over and
    over again, with the TTL increased by one each time.

    Ping sends a different sort of ICMP message, one that says "Please
    tell me that you got this" - more formally known as an ICMP Echo
    Request. The destination IP will send an ICMP Echo Response back to
    the originator when it receives one.

    Now, all these things I've described rely on ICMP messages making it
    back to the source of the IP/ICMP datagram triggering them. But some
    network administrators feel that allowing (certain) ICMP traffic to
    pass represents a security vulnerablity, so they block ICMP traffic.

    If ICMP traffic is blocked at a point part-way between you and
    www.centos.org, you will see the traceroute "stop" (those '*' lines
    starting indicating there was no response with the TTL set to that
    many hops). You will also not have ping "work" - it won't see any
    ICMP Echo Reply messages.

    If TCP tries to send a TCP segment that was carried in an IP datagram
    which needed to be fragmented somewhere in the middle there, then when
    it goes to send traffic it will appear to disappear - it hits an IP
    router (next hop) that tried to send it but couldn't and the "Hey!"
    message that IP router tried to send back was blocked.

    Now, by default, ping and traceroute do not send very large IP
    datagrams, so the chances of those packets arriving at a next hop
    where fragmentation is needed is pretty small.

    Also, not all TCP segments are the same size - the TCP segments used
    to setup a TCP connection for example are pretty small. That is what
    was behind some of the "try telnet www.centos.org 80" suggestions -
    that is an old school way to just try establishing a TCP connection
    but not actually send any data on it. Those TCP segments will be
    small.

    Why try that? (Or for that matter the pings or traceroutes with
    options set to send larger packets) Because it is how one tries to see
    if 'the problem" centers on packets which are too large arriving at a
    small data link somewhere, or simply a routing issue somewhere in the
    middle of the Internet. If the "telnet www.centos.org 80" establishes
    the TCP connection, you can assume that it isn't a routing issue. In
    that case, a/the remedy is to ensure that TCP at your system and TCP
    at the system(s) to which you are connecting never try to send TCP
    segments which become IP datagrams which are too large to send without
    fragmentation. One can do that by shrinking the MTU of the network
    interface of your system. When TCP establishes a connection, part of
    that is exchanging with the other side (eg www.centos.org) the Maximum
    Segment Size (MSS) for the connection. There are several inputs to
    that exchange, one of which is the MTU of your local interface. The
    smallest MSS (your's or that of www.centos.org) is the one that will
    be used for traffic in both directions.

    Now, if the telnet www.centos.org 80 does not succeed, that means
    there is some sort of routing issue. That was already partially
    suggested by the ping and traceroute "failures" you've already
    reported (if memory serves) but since the ICMP traffic on which they
    rely can be blocked, their failure is not conclusive.

    What sort of routing issue might it be? Well, one could be that there
    is a simply problem at or after one of those next hops you see in the
    traceroute output. That is why some have suggested you lookup the
    information about who runs those and contact them.

    It is possible that your source IP has been "black listed" for some
    reason or other - either at/past the ISP responsible for the last
    "next hop" you see in traceroute or perhaps even at www.centos.org.
    Perhaps at one point the IP address you have from your ISP was
    involved in something considered nefarious or anti-social.

    That you could still get to www.centos.org using TOR would remain
    consistent with your IP address (the one being assigned to your home
    gateway by your ISP) being blacklisted because the entire point of TOR
    (as I understand it) is to hid your actual IP address. It does that
    by adding some additional layers to the top of what I mentioned
    above. (As I understand it, I don't actually have any practical
    experience with TOR).

    rick jones
     
    Rick Jones, Oct 9, 2013
    #27
  8. billy

    ~BD~ Guest

    Have you been in touch here as suggested by Pooh?

    Technical Contact:
    Internap Network Services Corporation
    Domain Administrator
    One Ravinia Drive Suite 1300
    Atlanta, GA 30346
    US
    Phone: +1.8778434662
    Email:

    If not, WHY not?
     
    ~BD~, Oct 9, 2013
    #28
  9. billy

    unruh Guest

    But he has stated that they all work on other web sites. Only on
    centos.org do they fail, apparently.
    He says he cannot, although he keeps showing the output of traceroute
    rather than ping.
    Yes, he also says that his neighbors have no problem either. So it is
    something peculiar to the interaction between his machine and that
    particular site. -- eg firewall on centos which rejects him. Firewall on
    his side which rejects the far response. (probably he should make sure
    that all firewalls are disabled and try to contact them, just to rule
    out the firewall on his machines.)
     
    unruh, Oct 9, 2013
    #29
  10. billy

    unruh Guest

    Well, a firewall might be rejecting you or the reply.
    Switch off all firewalls-- on your machine, your router, etc to see if
    that might be the problem.

    (switch them on again after the test)
     
    unruh, Oct 9, 2013
    #30
  11. billy

    unruh Guest

    ["Followup-To:" header set to comp.os.linux.networking.]
    What do you mean "the login works fine"? You told us you could not
    connect in any way to that machine? Was that wrong?
    It is really hard to give help if information supplied is missing or
    wrong.
     
    unruh, Oct 9, 2013
    #31
  12. billy

    unruh Guest

    Well, no. He tried
    telnet centos.org 80
    which generates no HTTP message. Just a bare connection request.
    It fails.

    He sent really small packets.
    ping is I believe 80 bytes.
    And they are not rejecting ping packets (I just checked.)

    Except they do not. They come back to me with no trouble. They come back
    to his neighbour with no trouble.
    Yup, but why only to him?

    And they failed as well, so I think we can eliminate packet size as an
    issue.

    So, why does the route up to the last hop work, but not that last hop?
    And why does it work for everyone else. It is really really hard to see
    it as a routing issue.


    Yes, I does look like a firewall issue, but he claims that centos claims
    that there is no such issue.
     
    unruh, Oct 9, 2013
    #32
  13. billy

    unruh Guest

    Of course it is possible.
     
    unruh, Oct 9, 2013
    #33
  14. billy

    unruh Guest

    For you? No. Get in touch with them.
     
    unruh, Oct 9, 2013
    #34
  15. billy

    Mike Easter Guest

    Personally when I'm troubleshooting something I take into account the
    fact that some hops handle ICMP, UDP, and TCP differently. One of my
    preferred tools for testing access to a server is Steve Gibson's IDServe
    which runs under windows or WINE in linux and doesn't require
    installation in either.

    For this target I would be using TCP port 80 on/at the webserver at
    centos. The tool shows my system resolving the IP and connecting to the
    webserver there.

    Initiating server query ...
    Looking up IP address for domain: www.centos.org
    The IP address for the domain is: 72.232.194.162
    Connecting to the server on standard HTTP port: 80
    [Connected] Requesting the server's default page.
    The server returned the following response headers:
    [redact html from the centos Apache server]
    Query complete.
    From the other info you've posted here, your system when not reaching
    centos on port 80 TCP IDServe would show you resolving the name but not
    connecting, which IDServe interprets as the server being stealthed or
    not online, so its use wouldn't add anything to this particular
    problem's solution; but I mention it as an indication that 'typically' I
    don't use an ICMP tool to test some TCP-related server issues.
     
    Mike Easter, Oct 9, 2013
    #35
  16. billy

    Mike Easter Guest

    Login? Are you talking about login here?

    https://www.centos.org/user.php

    The difference between http and https protocols is the layering of http
    over the SSL/TLS protocol and the other difference is the use of port
    443 instead of 80.

    Or perhaps you are talking about logging in to some other server - the
    centos forums are at https centos as well.

    The bugs.centos server login is at a different IP 83.149.86.133
     
    Mike Easter, Oct 10, 2013
    #36
  17. billy

    unruh Guest

    Which was why I suggested
    telnet centos.org 80
    but that did not return anything for him. Everyone else in the world has
    no trouble connecting but he does and he wants to know why.
     
    unruh, Oct 10, 2013
    #37
  18. billy

    billy Guest

    That's a perfect summary!
    I can't imagine that I'm singled out in this world for not
    connecting, for months at a time, to centos.org (it happened
    with one other web site - but it was so long ago that I've
    forgotten which it was).

    One question I have is WHICH site is dropping my packets?

    Is it the last site that returns traceroute values?
    Os is it the first site that has all asterisks?

    # traceroute -M icmp www.centos.org
    traceroute to www.centos.org (72.232.194.162), 30 hops max, 60 byte
    packets
    1 192.168.1.1 (192.168.1.1) 2.840 ms 2.832 ms 2.829 ms
    2 MYIPADDRESS (xxx.xxx.xxx.xxx) 2.827 ms 5.830 ms 5.834 ms
    3 10.50.0.1 (10.50.0.1) 9.369 ms 14.568 ms 14.566 ms
    4 10.25.0.1 (10.25.0.1) 14.562 ms 18.614 ms 18.610 ms
    5 10.20.0.1 (10.20.0.1) 79.603 ms 83.180 ms 83.177 ms
    6 10.0.0.1 (10.0.0.1) 454.682 ms 251.479 ms 251.450 ms
    7 69.36.226.193 (69.36.226.193) 251.420 ms 402.683 ms 402.652 ms
    8 vl2.core1.scl.layer42.net (69.36.225.129) 402.624 ms 397.148 ms
    397.127 ms
    9 216.156.84.141.ptr.us.xo.net (216.156.84.141) 397.107 ms 284.336
    ms 284.314 ms
    10 207.88.14.233.ptr.us.xo.net (207.88.14.233) 287.164 ms 153.090 ms
    162.910 ms
    11 vb15.rar3.dallas-tx.us.xo.net (207.88.12.45) 162.878 ms 156.560
    ms 156.544 ms
    12 207.88.14.34.ptr.us.xo.net (207.88.14.34) 156.516 ms 2751.856 ms
    2751.839 ms
    13 207.88.185.74.ptr.us.xo.net (207.88.185.74) 2586.242 ms 97.179 ms
    97.152 ms
    14 border1.pc1-bbnet1.dal004.pnap.net (216.52.191.19) 97.123 ms
    border1.pc2-bbnet2.dal004.pnap.net (216.52.191.81) 74.418 ms
    border1.pc1-bbnet1.dal004.pnap.net (216.52.191.19) 87.348 ms
    15 layered-11.border1.dal004.pnap.net (63.251.44.74) 87.329 ms
    65.781 ms 120.012 ms
    16 * * *
    17 * * *
    18 * * *
    19 * * *
    20 * * *
    21 * * *
    22 * * *
    23 * * *
    24 * * *
    25 * * *
    26 * * *
    27 * * *
    28 * * *
    29 * * *
    30 * * *
    #

    Is it hop 15 or 16 above which is dropping my packets?
     
    billy, Oct 10, 2013
    #38
  19. billy

    billy Guest

    All I do, on TOR, is type 'http://www.centos.org' or "centos.org",
    and it gives me a screen where I can enter in my login and password
    and it logs me in. On Firefox, I do the same, when everything is
    working.

    Then, for months at a time, I can't even connect to www.centos.org
    or to 'centos.org' - getting the standard "connection timed out"
    message.

    When I run a traceroute (any type of traceroute), the result is
    that the last hop before centos.org is the last hop that returns
    anything. The centos.org hop is asterisked out.

    There's absolutely nothing wrong with my login (I easily log in
    with TOR, and I have an old thread asking THEM what's going on,
    and they don't know either).

    So, I 'think' it's the ISP that they use.

    I remember having a similar problem with telephones not being able
    to call certain numbers, and, we found out it was a routing thing
    at the Ooma headquarters. Nobody on the planet could call these
    specific numbers, one of whom was a relative of mine so I knew that
    the number existed. Yet, nobody could call the number from VOIP.

    Same kind of idea seems to be here. Some router somewhere is very
    badly set up.

    The funny thing, is that I'm the only one with the problem.
    Worse yet, there are no debugging tools for when a router is
    dropping packets. That's the saddest part.

    I've been trying to solve this for two years running.
    Or, at least understand how it could possibly happen.

    When I point my browser to http://83.149.86.133 I get the message:
    Forbidden
    You don't have permission to access / on this server.
    Apache/2.2.3 (CentOS) Server at 83.149.86.133 Port 80

    # telnet 83.149.86.133 80
    Trying 83.149.86.133...
    Connected to 83.149.86.133.
    Escape character is '^]'.
    HEAD HTTP/1.0
    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>400 Bad Request</title>
    </head><body>
    <h1>Bad Request</h1>
    <p>Your browser sent a request that this server could not
    understand.<br />
    </p>
    <hr>
    <address>Apache/2.2.3 (CentOS) Server at mirror.centos.org Port
    80</address>
    </body></html>
    Connection closed by foreign host.

    # traceroute -M icmp 83.149.86.133
    # traceroute -M icmp 83.149.86.133
    traceroute to 83.149.86.133 (83.149.86.133), 30 hops max, 60 byte
    packets
    1 192.168.1.1 (192.168.1.1) 3.423 ms 3.405 ms 6.261 ms
    2 MY_WISP_ADDRESS (xx.xx.xx.xx) 9.844 ms 9.842 ms 9.839 ms
    3 10.50.0.1 (10.50.0.1) 15.284 ms 139.930 ms 139.937 ms
    4 10.25.0.1 (10.25.0.1) 148.735 ms 148.734 ms 148.729 ms
    5 10.20.0.1 (10.20.0.1) 164.306 ms 169.567 ms 169.564 ms
    6 10.0.0.1 (10.0.0.1) 344.049 ms 70.997 ms 189.575 ms
    7 69.36.226.193 (69.36.226.193) 189.531 ms 34.070 ms 136.902 ms
    8 vl2.core2.scl.layer42.net (69.36.225.130) 139.000 ms 47.370 ms
    133.156 ms
    9 xe3-4.core1.mpt.layer42.net (69.36.239.110) 133.125 ms 35.300 ms
    61.929 ms
    10 any2ix.coresite.com (206.223.143.213) 73.838 ms 32.061 ms 94.368
    ms
    11 te5-2.msc2.wdc.leaseweb.net (217.20.125.40) 155.304 ms 99.444 ms
    109.215 ms
    12 * * *
    13 31.31.32.5 (31.31.32.5) 260.392 ms 220.540 ms 229.781 ms
    14 ten5-4.sbp-sr2.leaseweb.net (85.17.100.202) 229.737 ms 166.166 ms
    243.165 ms
    15 centos.yourname.nl (83.149.86.133) 239.189 ms 176.139 ms 194.943
    ms

    So, I have absolutely no problem getting to this site.
    But, why would I have a problem getting to centos.org?

    Could it be that anything after 15 hops fails no matter what?
    To test, do you know of a site that would be more than 15 hops
    away from San Francisco, CA?
     
    billy, Oct 10, 2013
    #39
  20. billy

    billy Guest

    I'm beginning to wonder if anything after about 15 hops is
    dropping the connection for me?

    Could it be that the path to centos.org just happens to be
    longer than any other path I've tried?

    The path to the centos bug server you gave me worked fine, but
    it only took 15 hops.

    Do you have a way for me to pick somewhere far far away from
    San Francisco CA to see if it's simply the long path that is
    causing the problem (i.e., more than 16 hops)?
     
    billy, Oct 10, 2013
    #40
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.