sendmail+TLS causing unwarranted TCP RST packets

Discussion in 'Linux Networking' started by Damian Menscher, May 3, 2005.

  1. [crossposted to comp.mail.sendmail and comp.os.linux.networking since
    the bug might be in sendmail, the linux tcp stack, ssl, iptables, or
    somewhere else ....]

    I've managed to discover something fairly disturbing when sending mail
    from a linux machine running sendmail: if the connection is encrypted
    via TLS, the connection won't close cleanly (with an exchange of FIN
    packets) but will terminate abruptly with a RST packet (from the
    machine that originated the email). I have not observed a RST packet
    from the originating machine when TLS is _not_ used.

    I just tested this with someone to try to figure out what's wrong.
    Here was the setup:
    My end is a RHEL 3 box:
    kernel-smp-2.4.21-27.0.2.EL
    sendmail-8.12.11-4.RHEL3.1
    Remote end is a Debian box:
    kernel: 2.6.12-rc2-mm1
    sendmail: 8.13.4
    We confirmed that if the RedHat box initiated the message transfer, it
    would be the one to issue a RST after the transaction was complete.
    Similarly, if the Debian box initiated the transfer, it would issue a
    RST.

    Mail *is* flowing, but I find it somewhat disturbing that there are RST
    packets being thrown about. My theory of what might be happening is
    that the sending sendmail process terminates after sending its final
    data (QUIT) to the SSL layer, rather than waiting for any final
    response (221 Goodbye). Since the process is no longer there, the TCP
    connection is broken, and a RST gets generated. That means the bug is
    in sendmail, right? I'd like to get some confirmation about this
    reasoning from others before filing a bug report, though, since my
    hypothesis is little more than an educated guess.

    Cheers,

    Damian Menscher
    --
    -=#| Physics Grad Student & SysAdmin @ U Illinois Urbana-Champaign |#=-
    -=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |#=-
    -=#| 4602 Beckman, VMIL/MS, Imaging Technology Group:(217)244-3074 |#=-
    -=#| <> www.uiuc.edu/~menscher/ Fax:(217)333-9819 |#=-
    -=#| The above opinions are not necessarily those of my employers. |#=-
     
    Damian Menscher, May 3, 2005
    #1
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.