Mirroring Socket FDs.

Hi

Apologies in advance if this should be in the kernel group.

I currently sysadmin various Linux systems that host Java applications
that make TCP/IP calls to remote hosts and have realtime data streamed
back to them.

What I want to be able to do, as efficiently as possible, is copy both
read & write channels on each of these connections, allowing me to
write this data to either another socket or to a filesystem without
having much (or any) impact on the system performance.

I know that I can examine these sockets using TCPDUMP or Socket
Filters, but that would require code to rebuild these packets back
into streams - I just want to be able to snoop the bi-directional data
streams directly without interacting with them. It occurs to me that
being able to snoop the FDs directly would achieve this, but I am at a
loss as to how to do this.

All (polite) suggestions welcome!

Gary

It would likely have a rather non-trivial effect, but you could strace
the relevant socket/filesystem calls on the socket fds.

Then you would have the data just as it was sent/recv'd by the
application.

rick jones
--
a wide gulf separates "what if" from "if only"
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

On Thu, 09 Aug 2007 10:47:27 -0700 (E-Mail Removed) wrote:

| What I want to be able to do, as efficiently as possible, is copy both
| read & write channels on each of these connections, allowing me to
| write this data to either another socket or to a filesystem without
| having much (or any) impact on the system performance.
|
| I know that I can examine these sockets using TCPDUMP or Socket
| Filters, but that would require code to rebuild these packets back
| into streams - I just want to be able to snoop the bi-directional data
| streams directly without interacting with them. It occurs to me that
| being able to snoop the FDs directly would achieve this, but I am at a
| loss as to how to do this.

Just about anything will require some sort of re-assembly of pieces of
data. TCP itself does that at each end, and then it's still broken up
through the system calls like read() and write() and the application has
to re-assemble. What's hard is parsing tcpdump output, or strace output.

If you can reconfigure the programs that make the TCP connections to make
them to different addresses and/or ports, you could use my "tcprelay"
program to capture the bi-directional traffic. If the security of these
connections depends on the host or port a connection comes from, then my
program could also be a security exposure for you.

You can get it as part of the LIBH package at http://libh.slashusr.org/

The command "tcprelay" is built as one of the test programs in LIBH to
test the "daemon" functions. It is started from a command line with a
syntax like:

tcprelay -c -d <directory> <listenport> <remotehost> <remoteport>

It runs in the background and will have 2 processes, plus one more per
connection being handled. Sending a SIGTERM to either of the 2 main
processes will terminate both.

It creates subdirectories and files in <directory> to capture traffic
with each connection in its own file. It stores data in a format that
also captures timestamps. It is a binary format that requires the
"tcprelayout" program (also from LIBH) to extract and reformat into
a readable form. Use the options "-et" to start with on that program
and pass it the capture file as stdin.

If you need the stream in another form, such as a raw file, you may need
to write your own processing program. You can use "tcprelayout" as a
starting point for that.

--
|---------------------------------------/----------------------------------|
| Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
| first name lower case at ipal.net / spamtrap-2007-08-11-(E-Mail Removed) |
|------------------------------------/-------------------------------------|