Routing/Filtering: scalability

Using a standard PC running linux what is the upper end of clients it
can deal with being used as a marking, rerouting, and filtering
firewall and router? When will it start dropping routes or killing
throughput?

James Knott wrote:
> Noah Roberts wrote:
>
>
>>Using a standard PC running linux what is the upper end of clients it
>>can deal with being used as a marking, rerouting, and filtering
>>firewall and router? When will it start dropping routes or killing
>>throughput?
>
>
> How long is a piece of string?
>
Depends on how many knotts are in it...not very.

Noah Roberts wrote:

> Using a standard PC running linux what is the upper end of clients it
> can deal with being used as a marking, rerouting, and filtering
> firewall and router? When will it start dropping routes or killing
> throughput?

How long is a piece of string?

> Using a standard PC running linux what is the upper end of clients it
> can deal with being used as a marking, rerouting, and filtering
> firewall and router? When will it start dropping routes or killing
> throughput?

(to the other poster, give us the string and we'll tell you how long it is.
although, point taken.)

I started out with a 486 w/ 12M ram that ran a firewall/NAT machine for my
cable modem and my bootlegged web host. Never had any problems. Updating
ipchains (what I was using back then) took about 10 minutes, but never had
any connection problems. I ran email, ftp, nntp, and web all from an
internal box that the linux box DNATed for. And it took care of my PC and
my wife's PC doing SNAT. No problems no complaints.

About 2 years ago I upgraded it to a pentium. Just last month I made it a
pentium II. No real need, just wanted to upgrade the kernel and tools and
didn't want a lot of downtime, which means bringing another box up to speed
and just swapping them out. And I didn't want to keep running the 486... I
kept my eye on the CPU several times while doing big downloads to see how
much usage it had, I never saw it climb above 10% on the 486.

I also have a PIII running in front of a web server with 80+ websites and a
T1 for the connection. Never noticed any problems with it either. It has
64M ram and currently has 16M in swap. But that's understandable because
eventually the kernel will swap stuff out anyway. I can easily stick
another 128M in it and the only things swapped out will be what the kernel
really doesn't think is necessary.

It really will depend on the amount of connections you support at a time,
and what type of throughput (T1, OC3...) you need, thus the other poster's
comment on measuring string.

As long as you have the memory, you can keep bumping up the connections it
can track by echoing numbers into /proc/sys/net/ipv4/ip_conntrack_max. I've
settled in with PIII w/ 128M - 256M of memory as being a good machine for
the nat/firewall stuff that I do, including mid-range connections like the
T1.

If you have a machine to use, start with the PIII. If you plan on buying
one, you can either spring $50 and get a PIII or go ahead and spend $200 and
get a PIV. I wouldn't worry about getting one of these 2.0+ GHz PIVs,
you'll just be wasting your money on them. Depending on your network, it
wouldn't hurt to get the Gig NICs. On my T1 my 100M NIC is bigger than the
T1 so I don't have to worry about the T1 ever sending more data than I can
handle (or me not being able to send enough data to use the entire T1 as
needed). But if I had a much larger pipe than a T1 I would consider going
with a Gig card *even* if my connection to the pipe was only going to be
100M, because at least I know the card itself is built to handle huge
amounts of traffic and max-out the 100M connection. And if you ever
upgraded that 100M connection, well your equipment would already be ready
for it. Now I wouldn't put a gig card in a PIII, I don't think the PCI bus
can handle a gig, it may take a PIV with a faster PCI bus before you get
above the gig NIC. I'm sure there are some docs out there that show the
throughput on various PCI busses and you can figure out what's needed for a
gig NIC (or maybe someone here will just tell us).

And to answer that string question:

This string is 34 characters long.

Measure it and see!

/dev/null wrote:

> It really will depend on the amount of connections you support at a
time,
> and what type of throughput (T1, OC3...) you need, thus the other
poster's
> comment on measuring string.

Ok, how bout 200+ computers running web, kaza, bittorrent, etc, being
routed to the internet or to local SMB shares. Firewall tracking
connection state, marking registered hardware for passthrough, and
redirecting unregistered traffic to a captured portal registration
system.

200+ computers are on a 100BaseT network connected to main campus by
gigabit fiber. SMB shares on 100BaseT inside main campus. Internet
connection is a fiber line running upstairs to a 6Mbit site2site
wireless connection through PPPoE. Internet traffic is smashed by a
packet shaper post routing by network appliance between firewall and
internet router. Internet router does NAT.

What kind of hardware does the firewall/router have to be in order to
not be a bandwidth bottleneck?

I was going to provide diagram but google groups bugged it up really
good...no point.

Noah Roberts wrote:
> /dev/null wrote:
>
> > It really will depend on the amount of connections you support at a
> time,
> > and what type of throughput (T1, OC3...) you need, thus the other
> poster's
> > comment on measuring string.
>
> Ok, how bout 200+ computers running web, kaza, bittorrent, etc, being
> routed to the internet or to local SMB shares. Firewall tracking
> connection state, marking registered hardware for passthrough, and
> redirecting unregistered traffic to a captured portal registration
> system.
>
> 200+ computers are on a 100BaseT network connected to main campus by
> gigabit fiber. SMB shares on 100BaseT inside main campus. Internet
> connection is a fiber line running upstairs to a 6Mbit site2site
> wireless connection through PPPoE. Internet traffic is smashed by a
> packet shaper post routing by network appliance between firewall and
> internet router. Internet router does NAT.
>
> What kind of hardware does the firewall/router have to be in order to
> not be a bandwidth bottleneck?

Since your basic setup is in place, simply do some monitoring to see
how fast the packets are pounding the _interfaces_. Monitor Linux
resource use under load. Ram is cheap if you need more.

It's not clear what nics/bandwidth you have attached to the Linux box,
but doubt that's a problem with _properly_ functioning hardware. GigE
cards _can_ be troublesome though.

Linux processing will never be as "fast" as a level 3/4/5 switch with
_port_ asic processors (think Cisco Catalyst and LightStream). Latency
will be greater under load and and you will never achieve "wire speed"
throughput.

That said, if the upstream connection to ISP/provider is 6Mbit and
internally bound local traffic is not routing through the Linux box,
even 10Mb ethernet would likely work. The uplink is the bottleneck.
Ideally what you want is a "steady" stream of packets flowing at near
"bottleneck" speed. Achieve that, then you can think of "optimizing"
apparent throughput to make the system more responsive.

And note that the _upload_ bandwidth is probably much lower than the
download and frequently degrades download performance
disproportionately. This can get to be a real hassle with peer2peer
apps like bittorent

> I was going to provide diagram but google groups bugged it up really
> good...no point.

It's a Google thing. Use a text editor with a column width less than,
say 74, and a monospaced font (spaces - no tabs). Copy-n-paste into
the Google edit contol and let 'er rip. On receipt, the (Google)
reader can copy-n-paste into a text editor. I have to do it all the
time ;-)

hth,
prg
email above disabled

prg wrote:

> Linux processing will never be as "fast" as a level 3/4/5 switch with
> _port_ asic processors (think Cisco Catalyst and LightStream). Latency
> will be greater under load and and you will never achieve "wire speed"
> throughput.
>
> That said, if the upstream connection to ISP/provider is 6Mbit and
> internally bound local traffic is not routing through the Linux box,

I think that is my greatest worry, it is. The main purpose of the linux
box is to protect the main network from those on the other side.

Here is that diagram I wanted to post (now that I am at home I can):

VLAN A <---no route----> VLAN D
Main Campus----Linux FW----Packet shaper->ISP
| <- 801.2Q Tagged
L2 SW
/ \
(VLAN B) Dorms Semi-public 802.11 (VLAN C)

If the Vlan tagging is an issue I can probably just add a 4th card...and
of course any load information I can get on that stuff is also needed.

Dorms contain about 200 computers, 99% being windows xp and 98% run by
total invalids when it comes to computer maintenence (and sometimes it
is worse than that). The 802 network is going to be places like the
cafeteria and such where students can do school work on laptops. We
cannot trust ANY of these computers but we need to let them onto our
network so that they can access certain resources they need for homework
and such.

Currently the main campus and the dorms are directly connected through a
basic router. There is minimal protection. We use NetReg to register
dorm computers and theoretically keep them from harming the main
network. But that is based on a lot of assumptions such as the basic
stupidity of the average virus author, that assumes that malicious
traffic will not pass a gateway or try ip addresses outside the
computer's configured network. I really hate that idea...if I was a
virus author I would pass that gateway...

NetReg uses a fake DHCP and DNS to trick computers into going to a
registration site when they try to go to google or something. It is
pretty easily bypassed by anyone with half a brain...which isn't /much/
of a worry here but still.

A website is used to register computers so that the DHCP will give a
"valid" ip and directions to the gateway. This website contains a
program that must run on the local computer that checks for SP2 and
norton or mcafee.

So I am using a different idea to develop an in-between solution that is
a little harder to bypass (not much mind you, but more than even
semi-literate people know how to do). In speaking on the NetReg list it
was brought up that it doesn't scale well. So I am trying to figure out
just what the barriers are.

An overall description of what I am doing is now available online at
http://sourceforge.net/projects/fwreg and is in the documentation file
download. I just finished it at work today.

You could use NetReg and a 'hardware' firewall but I think there are
limitations there as well. The firewall rather needs to know a lot more
than the average box solution does I think. What I am doing now is
rather basic and could be done with one but further down the line I
doubt it. I just don't like the blackhole dns idea, it seems to much
like security through obscurity to me.

Noah Roberts wrote:
> prg wrote:
>
> > Linux processing will never be as "fast" as a level 3/4/5 switch
with
> > _port_ asic processors (think Cisco Catalyst and LightStream).
Latency
> > will be greater under load and and you will never achieve "wire
speed"
> > throughput.
> >
> > That said, if the upstream connection to ISP/provider is 6Mbit and
> > internally bound local traffic is not routing through the Linux
box,
>
> I think that is my greatest worry, it is. The main purpose of the
linux
> box is to protect the main network from those on the other side.
>
> Here is that diagram I wanted to post (now that I am at home I can):
>
> VLAN A <---no route----> VLAN D
> Main Campus----Linux FW----Packet shaper->ISP
> | <- 801.2Q Tagged
> L2 SW
> / \
> (VLAN B) Dorms Semi-public 802.11 (VLAN C)

My personal inclination is always to have the firewall do just its one
task of keeping the bad guys from getting in and allowing legit traffic
out. It's tough enough without burdening it with other duties -- it's
also tougher on me to maintain multiple funcitons on one
machine/firewall. Thus the classic and still quite sensible layout of
3 nics -- (a) to ISP, (b) to DMZ, and (c) to internal network. Why not
place an "inside" router/firewall just downstream of the Linux FW for
your internal routing needs -- it could be Linux as well.

That way you can keep the lan internal traffic -- which may get pretty
heavy in terms of bandwidth use simply because of the greater
host-to-host link speed. That is, try to segregate this internal
traffic flow from the Linux FW's duties of managing external packets.
I think it would be more flexible also as you would not have to worry
so much that "fixing" one function (say, internal routing) would
"break" the other (FW) function.

> If the Vlan tagging is an issue I can probably just add a 4th
card...and
> of course any load information I can get on that stuff is also
needed.

A separate internal router would help a lot with this, as you may find
that you have to make more changes here than anywhere else. Would also
allow you some internal routing control -- policy routing or queueing
-- as internal needs/demands change. Especially so with wifi access --
as people get used to it they will want more of it

> Dorms contain about 200 computers, 99% being windows xp and 98% run
by
> total invalids when it comes to computer maintenence (and sometimes
it
> is worse than that). The 802 network is going to be places like the
> cafeteria and such where students can do school work on laptops. We
> cannot trust ANY of these computers but we need to let them onto our
> network so that they can access certain resources they need for
homework
> and such.

As needs develop, you could also place a Linux box to help just with
the wifi connections. Nice thing about using Linux as a router/fw is
that with older hardware (meaning less $) you can deploy them where you
need them, when you need them. Most networks don't require high port
density routers -- a box that will hold 2-4 cards (single or
multi-port) and accept 64-512 MB ram can handle just about anything.
Good quality -- ie., server class -- nics will help a lot with GigE
connections.

> Currently the main campus and the dorms are directly connected
through a
> basic router. There is minimal protection. We use NetReg to
register
> dorm computers and theoretically keep them from harming the main
> network. But that is based on a lot of assumptions such as the basic

> stupidity of the average virus author, that assumes that malicious
> traffic will not pass a gateway or try ip addresses outside the
> computer's configured network. I really hate that idea...if I was a
> virus author I would pass that gateway...
>
> NetReg uses a fake DHCP and DNS to trick computers into going to a
> registration site when they try to go to google or something. It is
> pretty easily bypassed by anyone with half a brain...which isn't
/much/
> of a worry here but still.
>
> A website is used to register computers so that the DHCP will give a
> "valid" ip and directions to the gateway. This website contains a
> program that must run on the local computer that checks for SP2 and
> norton or mcafee.

Yes, I spent an hour reading up on NetReg before posting, and ...

> So I am using a different idea to develop an in-between solution that
is
> a little harder to bypass (not much mind you, but more than even
> semi-literate people know how to do). In speaking on the NetReg list
it
> was brought up that it doesn't scale well. So I am trying to figure
out
> just what the barriers are.
>
> An overall description of what I am doing is now available online at
> http://sourceforge.net/projects/fwreg and is in the documentation
file
> download. I just finished it at work today.

I'll check out your stuff. Like you, I was immediately struck by what
seemed rather lax security built into NetReg. So long as you don't try
to make dhcp perform functions it was not built for and understand its
limitations though, I can see where the "automatic" registration --
tightened up a bit -- could be a great aid in those hectic "new student
influx" periods.

> You could use NetReg and a 'hardware' firewall but I think there are
> limitations there as well.

The main limitation being that MAC addresses don't migrate along with
the normal packet flow -- just IPs and the next hop MAC.

> The firewall rather needs to know a lot more
> than the average box solution does I think. What I am doing now is
> rather basic and could be done with one but further down the line I
> doubt it. I just don't like the blackhole dns idea, it seems to much

> like security through obscurity to me.

Actually, quite a few sites use DNS records and mail authentication as
a "quick-n-dirty" network authentication mechanism. I think it's
because the mail server is already in place and requires DNS for _its_
operation, so why not just "extend" it for other uses. Presto, DHCP
and MAC addresses are available, so lets cross reference the data and
have "network authentication". YUK

Of course, a _great_ deal of this lies with the clients being Windows
and either not multi-user capable at all or defaulting to Admin/root
user access without a password. Jimeney ;[grrrrrrr

I'll check with you after viewing your stuff.
good weekend ,
prg
email above disabled