Max Ethernet ports

Hello,

Is there an inherent limit to the number of Ethernet interfaces Linux will
support? I'm looking to build a switch like appliance with custom
software/protocols for up to 48 10/100 downstream ports, convert and upload
over two upstream ports 10/100/1Gbe. I realize the throughput will be no
where close to a real switch however I don't need much as each device uses
minimal bandwidth and they hardly talk amongst themselves. Ethernet is used
more of a ease of install than a bandwidth requirement.

Any ideas?

Thanks

> Is there an inherent limit to the number of Ethernet interfaces Linux will
> support?

Somewhere, long, long ago, an official-ish statement was made on that
subject, to the effect that there IS a limit - but that it would be all but
impossible to connect that many to a single machine anyway. If memory
serves, the number was in the thousands, or tens of thousands.

steve

Thanks so now the bigger question is any ideas short of building a custom
board as to how to get these many ports on a single linux box?

Mike

"Steve Wolfe" <(E-Mail Removed)> wrote in message
news:g_-(E-Mail Removed)...
>> Is there an inherent limit to the number of Ethernet interfaces Linux
>> will
>> support?
>
> Somewhere, long, long ago, an official-ish statement was made on that
> subject, to the effect that there IS a limit - but that it would be all
> but impossible to connect that many to a single machine anyway. If memory
> serves, the number was in the thousands, or tens of thousands.
>
> steve
>

On Sat, 07 Apr 2007 18:36:37 +0000, Mike rearranged some electrons to
form:

> Thanks so now the bigger question is any ideas short of building a custom
> board as to how to get these many ports on a single linux box?
>
> Mike

Ethernet switch...?



--
David M (dmacchiarolo)
http://home.triad.rr.com/redsled
T/S 53
sled351 Linux 2.4.18-14 has been up 5 days 22:49

On Fri, 06 Apr 2007, in the Usenet newsgroup comp.os.linux.networking, in
article <HkxRh.255749$(E-Mail Removed)>, Mike
wrote:

>Is there an inherent limit to the number of Ethernet interfaces Linux
>will support?

It's more a hardware limit than O/S specific. Go price out a D-Link
DFE-570TX, DFE-580TX, AEI P430TX, 3Com has one as well, but I can't think
of the number - four port Ethernet cards - PCI. Then look at your PCI
motherboard, and tell me how many sockets you see where you can plug in
such a card. Five? That gives you 20 interfaces, and they're not
going to be cheap.

>I'm looking to build a switch like appliance with custom software/
>protocols for up to 48 10/100 downstream ports, convert and upload
>over two upstream ports 10/100/1Gbe.

Not sure I follow you here - Ethernet is a pretty dumb protocol, and
while the 'type' block in the frame is 16 bits, there are only about 180
protocol numbers currently assigned. See
http://www.iana.org/assignments/ethernet-numbers

>I realize the throughput will be no where close to a real switch however
>I don't need much as each device uses minimal bandwidth and they hardly
>talk amongst themselves. Ethernet is used more of a ease of install
>than a bandwidth requirement.

I'm not sure what problem you are trying to solve, but using a PC as a
switch is heading in a completely different direction. Spamazon is showing
a SMC 8748MINT 48-PORT 10/100/1000 10G switch for US$1713, but you can
likely cascade 8-10 port SOHO grade switches for even less. What makes a
switch work (basically a 64 bit delay line per input, to allow the smarts
to look at the destination MAC and switch accordingly) is quite different
from a general purpose compute like a PC.

Old guy

On Sat, 7 Apr 2007, in the Usenet newsgroup comp.os.linux.networking, in article
<g_-(E-Mail Removed)>, Steve Wolfe wrote:

>> Is there an inherent limit to the number of Ethernet interfaces Linux
>> will support?
>
> Somewhere, long, long ago, an official-ish statement was made on
>that subject, to the effect that there IS a limit - but that it would be
>all but impossible to connect that many to a single machine anyway.

That's the real key - even assuming 5 PCI slots with Quad Ethernet cards
(and perhaps a built-in interface on the motherboard), you're a long way
from what a cheap SOHO Ethernet Switch can do. Then there's the fact
that the switch and a general purpose computer need quite different
designs.

>If memory serves, the number was in the thousands, or tens of thousands.

I'd find that a bit unusual. Other than the hardware problem, the most
likely problem I can think of is 'Neighbor Table Overflow' which can be
a function of the connection tracking limits in the firewall code, and
the size of the ARP cache - both of which can be tweaked while the
system is running. There might also be a limit in the size of the
routing table.

Old guy

Hmm. I was browsing through the bridge code in Linux and it seemed to what
I was needing. The protocol layer I'm converting runs on top of Ethernet &
TCP/IP (Level 7 data transfers).

Does anyone make a switch where an OEM or developer can embed an application
rides in between the uplink ports and the downstream switch ports

Thanks again for the assistance in this,
Mike

"Moe Trin" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> On Fri, 06 Apr 2007, in the Usenet newsgroup comp.os.linux.networking, in
> article <HkxRh.255749$(E-Mail Removed)>, Mike
> wrote:
>
>>Is there an inherent limit to the number of Ethernet interfaces Linux
>>will support?
>
> It's more a hardware limit than O/S specific. Go price out a D-Link
> DFE-570TX, DFE-580TX, AEI P430TX, 3Com has one as well, but I can't think
> of the number - four port Ethernet cards - PCI. Then look at your PCI
> motherboard, and tell me how many sockets you see where you can plug in
> such a card. Five? That gives you 20 interfaces, and they're not
> going to be cheap.
>
>>I'm looking to build a switch like appliance with custom software/
>>protocols for up to 48 10/100 downstream ports, convert and upload
>>over two upstream ports 10/100/1Gbe.
>
> Not sure I follow you here - Ethernet is a pretty dumb protocol, and
> while the 'type' block in the frame is 16 bits, there are only about 180
> protocol numbers currently assigned. See
> http://www.iana.org/assignments/ethernet-numbers
>
>>I realize the throughput will be no where close to a real switch however
>>I don't need much as each device uses minimal bandwidth and they hardly
>>talk amongst themselves. Ethernet is used more of a ease of install
>>than a bandwidth requirement.
>
> I'm not sure what problem you are trying to solve, but using a PC as a
> switch is heading in a completely different direction. Spamazon is showing
> a SMC 8748MINT 48-PORT 10/100/1000 10G switch for US$1713, but you can
> likely cascade 8-10 port SOHO grade switches for even less. What makes a
> switch work (basically a 64 bit delay line per input, to allow the smarts
> to look at the destination MAC and switch accordingly) is quite different
> from a general purpose compute like a PC.
>
> Old guy

Mike wrote:
> Hmm. I was browsing through the bridge code in Linux and it seemed to what
> I was needing. The protocol layer I'm converting runs on top of Ethernet &
> TCP/IP (Level 7 data transfers).

If the Linux box needs to handle a zillion TCP/IP connections, then I
expect that lack of resources for that will get you first. You don't
need to add the overhead for a herd of NIC's to the mix. Take Moe's
suggestion and use Ethernet hubs or switches to concentrate the traffic.
A 24-port 10/100 switch costs under $100 these days.

That's what I'm looking at however, I need to place my app (ok, now running
on a Linux SBC) which is connected between the 24 port side of the switch
and the 1 uplink port. I'd like this all in a 1U enclosure.
I haven't found a switch I can insert a SBC in between these two ports and
I'm trying to have this in a all in one 1U unit.

I realize this could be done with two independent 1U boxes, but I'm trying
to conserve some space.

Thanks
Mike

"Allen McIntosh" <(E-Mail Removed)> wrote in message
news:WlWRh.530$(E-Mail Removed)...
> Mike wrote:
>> Hmm. I was browsing through the bridge code in Linux and it seemed to
>> what I was needing. The protocol layer I'm converting runs on top of
>> Ethernet & TCP/IP (Level 7 data transfers).
>
> If the Linux box needs to handle a zillion TCP/IP connections, then I
> expect that lack of resources for that will get you first. You don't need
> to add the overhead for a herd of NIC's to the mix. Take Moe's suggestion
> and use Ethernet hubs or switches to concentrate the traffic. A 24-port
> 10/100 switch costs under $100 these days.

On Sat, 07 Apr 2007, in the Usenet newsgroup comp.os.linux.networking, in
article <9qVRh.24106$(E-Mail Removed)>, Mike wrote:

>Hmm. I was browsing through the bridge code in Linux and it seemed to
>what I was needing. The protocol layer I'm converting runs on top of
>Ethernet & TCP/IP (Level 7 data transfers).

I hate talking "levels" as people tend to disagree which is which and
what is happening at the level they're referring to.

"runs on top of Ethernet" - which is _normally_ were a switch tends
to reside, shuffling packets based on the MAC address. The whole
concept of a switch is to isolate the links as rapidly as possible.
I mention the concept of a 64 bit delay line (often, a shift
register running at the line speed) with taps such that the start of
the frame can be recognized. At that point, the smarts reads the
first 48 bits of the frame (destination MAC address) and sets a
switch matrix so that the end of the shift register is fed into the
"gozoutta" hose appropriate for that MAC address. (If the port is
busy, the switch may dump the packet to short term memory so that it
can be kicked out the door as soon as the line is free.) The switch
doesn't have the time (read "bandwidth") to be doing anything else.
In fact, the difference between the 48 bits of the address and the
(example) 64 bits of the delay line is the amount of time that the
smarts has to read the MAC, figure out which port that may be on,
test to see if the port is free, and set the switch to route the
"comezoutta" from the delay line to that appropriate port. For 100
Megabit links, that might be 0.16 millionths of a second, or about
one clock cycle on an old ISA bus. Don't you be late now!

As for running above TCP/IP, that's three levels higher up the stack,
and that likely means lots of CPU processing. "Switches switch" at the
lowest protocol level for bandwidth. If you have to feed data to the
CPU, you are running a bandwidth limited process. On a PCI based
system, you _can_ move eight bytes (64 bits) at a chunk, but the
top speed of the PCI bus is 66 MHz. and it takes more than one
clock cycle to "inhale" and "exhale". Routers and similar can
move bits rapidly, but only through a small number (compared to a
switch) of interfaces.

>Does anyone make a switch where an OEM or developer can embed an
>application rides in between the uplink ports and the downstream
>switch ports.

I don't know of any, because you are leaving the realm of a switch
(a device meant to route packets with a minimal delay and no data
processing) and getting up into the realm where you are going to
open the packet and dink with the contents. That's a completely
incompatible process. Thats not a networking problem so much as a
data processing application.

In your original post, you spoke of "each device uses minimal
bandwidth and they hardly talk amongst themselves". Still not
enough information, but I'd suggest using cheap SOHO switches
to concentrate your 48 devices into one/few Ethernet connections
on the computing box, and do the processing serially, rather than
try to parallel things.

Old guy