IPtables logging failure (multiple NICs)

However, after putting in about 45 to 50 hours of my own time over the
weekend researching and tinkering with configuration details, syslog,
iptables, tcpdump, and other troubleshooting details, including
recompiling the kernel with support for other cards, nothing works.

Here are the specs of the problem:

There are three networks that I am trying to monitor: two internal
subnets and the external network, set up as such

----------------------------------------------------------------------
10.0.0.0/24 public wireless subnet
(Monitor port on Cisco switch)
|
|
|
*eth1*

Log box *eth2* ------ 209.x.x.x/28
Mgmt Net-------------*eth0* (FC4) (public IP block
(to IS hub) connected to
*eth3* 10/100 hub)
|
|
|
192.168.10.0/24 business subnet
( Monitor port on HP switch )
----------------------------------------------------------------------

The only NIC that has an IP currently is the managment NIC, though it
still doesn't work even if I assign eth1-3 all IP's and/or gateways.
Only eth0 is able to receive all traffic using IPtables and tcpdump.
eth1-3 only show all traffic if I use "tcpdump -i ethx", otherwise they
only show traffic addressed to their given IP (if they don't have an
IP, they log nothing)

Here are more details on my configuration:

All switches have the port the logging box plugged into set for monitor
mode on all other ports. The hubs receive all traffic on all ports,
anyway.

I have set eth1-3 into promiscuous mode manually. (I used ifconfig -a
to confirm)

Syslog is set to log all kern.=debug messages to /var/log/iptables. I
can confirm this by changing the iptables rules to log all traffic in
and out on eth0 and watching /var/log/bandwidth grow. I force traffic
by scanning the eth1-3 addresses using nmap from another computer.

"tail -f /var/log/iptables"
shows that all traffic on the IS subnet shows up perfectly, just like
it's supposed to do, no matter what the source or destination IP's are.


"tcpdump -f -i eth0"
shows the same packets as are shown in /var/log/iptables

"tcpdump -f -i eth1" (or eth2/eth3)
shows all traffic that I want to log using IPtables, that iptables will
not log.

Also, I wanted to make sure the card still worked, so I switched the
cables to eth2 and eth0, making eth2 the management port and eth0 the
public logging port. At that point, eth0 stopped allowing all traffic
into the log and only showed traffic specifically addressed to it, and
eth2 allowed all traffic to the log

I'm stumped. I've read through all of the RHCE books, linux forums,
information on syslog, iptables, and tcpdump that I can get my hands
on, physically or electronically. Nothing tells me why this is
happening. Any help?


<BEGIN CONFIGURATION DATA>
The box is a Fedora Core 4 installation on a Dell GS400 workstation.
Updated with "yum upgrade" on 3/11/06 at approximately 0415. *yawn*

***NIC-CONFIGURATION***
Network configuration is as follows
NIC 1: (1 physical port)
eth0 : 3Com Corporation 3c905C-TX/TX-M [Tornado]
(3c59x)
NIC 2: (3 physical ports)
[Compaq Dual 10/100 Network Card 64Bit NC3131 (Intel make)]
eth 1, 2: Intel Corporation 82557/8/9 [Ethernet Pro 100]
(e100)
[NC7132 COPPER GIGABIT UPGRADE CTLRMODULE FOR NC3131 NC3]
eth 3 : Intel Corporation 82542 Gigabit Ethernet Controller
(e1000)


*** IFCFG-FILES ***
/etc/sysconfig/network-scripts/ifcfg-ethx:
ifcfg-eth0:
ONBOOT=yes
DEVICE=eth0
BOOTPROTO=static
HWADDR=[xx:xx:xx:xx:xx:xx]
NETMASK=255.255.255.0
IPADDR=192.168.1.17
GATEWAY=192.168.1.1
BROADCAST=192.168.1.255
NETWORK=192.168.1.0
ifcfg-eth1-3:
BOOTPROTO=none
TYPE=Ethernet
HWADDR=[xx:xx:xx:xx:xx:xx]
DEVICE=eth1
ONBOOT=yes


***IPTABLES-RULES***
IPtables rules (from /etc/sysconfig/iptables
# Generated by iptables-save v1.3.0 on Sat Mar 11 11:54:18 2006
*nat
:OUTPUT ACCEPT [1:70]
:POSTROUTING ACCEPT [1:70]
:PREROUTING ACCEPT [2:283]
COMMIT
# Completed on Sat Mar 11 11:54:18 2006
# Generated by iptables-save v1.3.0 on Sat Mar 11 11:54:18 2006
*mangle
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [62:5175]
:OUTPUT ACCEPT [44:6573]
:POSTROUTING ACCEPT [44:6573]
:PREROUTING ACCEPT [62:5175]
COMMIT
#
:FORWARD ACCEPT [0:0]
:INPUT DROP [0:0]
:RH-Firewall-1-INPUT - [0:0]
#
:OUTPUT DROP [0:0]
-A FORWARD -j RH-Firewall-1-INPUT
-A INPUT -i eth2 -j LOG --log-prefix "BANDWIDTH_2: " --log-level debug
-A INPUT -i eth1 -j LOG --log-prefix "BANDWIDTH_1: " --log-level debug
-A INPUT -i eth3 -j LOG --log-prefix "BANDWIDTH_3: " --log-level debug
-A INPUT -s 127.0.0.1 -j ACCEPT
-A INPUT ! -i eth0 -j DROP
#
-A OUTPUT -o eth2 -j LOG --log-prefix "BANDWIDTH_2: " --log-level
debug
-A OUTPUT -o eth0 -j ACCEPT
-A OUTPUT -o eth2 -j DROP
-A OUTPUT -o eth1 -j DROP
-A OUTPUT -o eth3 -j DROP
#
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp -i eth0 --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp -i eth0 --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
#
-A INPUT -j RH-Firewall-1-INPUT
COMMIT
# Completed on Sat Mar 11 11:54:18 2006

***SYSLOG.CONF***
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none
/var/log/messages

# The authpriv file has restricted access.
authpriv.* /var/log/secure

# Log all the mail messages in one place.
mail.*
-/var/log/maillog

# Log cron stuff
cron.* /var/log/cron

# Everybody gets emergency messages
*.emerg *

# Save news errors of level crit and higher in a special file.
uucp,news.crit
/var/log/spooler

# Save boot messages also to boot.log
local7.*
/var/log/boot.log

#log all network traffic for configuration testing
*.=debug
-/var/log/iptables

<END CONFIGURATION DATA>

On 13 Mar 2006 18:11:37 -0800, "(E-Mail Removed)" <(E-Mail Removed)> wrote:

>However, after putting in about 45 to 50 hours of my own time over the
>weekend researching and tinkering with configuration details, syslog,
>iptables, tcpdump, and other troubleshooting details, including
>recompiling the kernel with support for other cards, nothing works.

You must be doing this the hard way

>I have set eth1-3 into promiscuous mode manually. (I used ifconfig -a
>to confirm)

Break the problem down, just enable one extra NIC and get the logging
working.

>"tcpdump -f -i eth1" (or eth2/eth3)
>shows all traffic that I want to log using IPtables, that iptables will
>not log.

INPUT + FORWARD rules only see the start (first packet) of TCP traffic,
not each packet, for that you need the mangle table.

>-A INPUT -i eth2 -j LOG --log-prefix "BANDWIDTH_2: " --log-level debug

I use 'LOG --log-level info --log-prefix "prefix "' to go to
/var/log/messages, no problem

>***SYSLOG.CONF***
....
>#log all network traffic for configuration testing
>*.=debug
>-/var/log/iptables

And I don't see a mention of iptables in my /etc/syslog.conf

Perhaps you're trying to solve too many issues at once?

Grant.
--
Testing can show the presense of bugs, but not their absence.
-- Dijkstra

Grant,

I did not know that input didn't see all inbound traffic. Hmm. I'll
have to look into that. I don't suppose logging works from there,
then? But then why does it work on eth0? Maybe that's got something
to do with it. I'll give it a whirl and see what it does.

Also, about the one NIC at a time, that's the last thing I did. I
tried to use only the public block interface (eth2) from the machine's
console. Gave me the same problems. What's really strange, is that
everything's fine when it comes through eth0, works just like I want it
to, but only for that interface.

I added the /var/log/iptables to my syslog.conf file. Like I said, it
at least works for one interface, but I'm not sure why it doesn't work
for the others.

John