G'Day,
On Fri, 19 Nov 2004, AA wrote:
> I was wondering if anyone can recommend some tools (Opensource or
> commercial) to automate the parsing of very large (many GB) tcpdump
> files. I am trying to put together a generic toolset but in general some
> things I'd like to do are:
>
> 1. Filter out traffic to/from a specific IP address or range
> 2. Reconstruct all reconstructable sessions in an easy to parse way:
> emails, web sites visited (and content uploaded/downloaded), voip,
> anything else imaginable.
> 3. Be able to search all of this data for keywords.
>
> I know of a few tools to do individual tasks on a small scale, such as
> mailsnarf, vomit, ethereal, etc. but it's not practical to use ethereal
> or the others to parse these by hand. I've tried chaosreader.pl but it
> bogs down on files as small as 200 MB.
I wrote Chaosreader as a program to demonstrate the vulnerabilities of
plaintext protocols such as telnet, HTTP, FTP, X11, VNC, etc; while using
log files of around 10Mb.. (I had met some people who believed X11 to be
"safe" as the protocol was too hard to interpret and redisplay[1]).
A 200Mb demo is, erm, rather large. Don't use the "-ve" options as
they trigger Hex dumps - which consume a lot of memory. Someone did
explain a legitimate reason to me for processing huge files, so optimising
the memory footprint is on my todo list.
no worries,
Brendan Gregg
[Sydney, Australia]
[1] They were mostly right - raw X11 processing is awful.
|
Similar Threads
Linear Mode
