Setting up a linux fileserver

Hello. I'm trying to find a solution that would work for me. I need a
file sever that would work for about 40 different nodes. Those nodes
might need large amounts of data at the same time (like 30 computers
trying go retrieve 500 MB of data each, that data is scattered in coule
hundreds of files), those peaks don't happen very often but there is a
general tendency of having new connections happening simultaneously.
Currently I have linux box serving samba to windows nodes but I can't
find any solution go give any load balancing for my connections. I use
gigabit ethernet and at some times it chokes my file server (not it's
disks, not it's cpu but it's network). I was wondering on some solutions
that would use multiple network cards from one fileserver. It would be
possible for me to change other nodes from windows to linux (but no
solaris please) so linux to linux solutions are welcome (linux to
windows would be perfect). I found some infomations about NFS v4.1 but
it might be an overkill for my situation (having multiple servers
working as one).

I hope that you can give me some advice in my situation.

Kuba

On Mon, 27 Apr 2009 12:40:14 +0200, groblus wrote:

> I was wondering on some solutions
> that would use multiple network cards from one fileserver.

Why not just break the network up into subnetworks on separate LANs?


If it is the NIC in the server that is saturating, depending on your
router, you could install multiple NICs and use your (managed) router to
isolate different boxen into different subnets

Alternatively, you could just have different routers connected to
different subnets and different groups of boxen.

About 15 years ago, I did install dual NICs into
pC boxen, one went to WAN gateway and Novell server) and tother went to
*nix boxen for data, printing, backups, xterm, etc.

terryc pisze:
> On Mon, 27 Apr 2009 12:40:14 +0200, groblus wrote:
>
>> I was wondering on some solutions
>> that would use multiple network cards from one fileserver.
>
> Why not just break the network up into subnetworks on separate LANs?
>
>
> If it is the NIC in the server that is saturating, depending on your
> router, you could install multiple NICs and use your (managed) router to
> isolate different boxen into different subnets
>
> Alternatively, you could just have different routers connected to
> different subnets and different groups of boxen.
>
> About 15 years ago, I did install dual NICs into
> pC boxen, one went to WAN gateway and Novell server) and tother went to
> *nix boxen for data, printing, backups, xterm, etc.
>
this might be a nice solution especially for now, but is there some nice
and elegant solution having one network (without subnetworks) and load
balance samba shares ? or if not by using samba perhaps use of nfs ?
or I could use some form of hardware link aggregation ? Perhaps thera
are some double port network card that allow "adapter teaming" under
linux ? There are number of possibilities and I don't know which to
explore. Whis will be easiest to implement ?

On Mon, 27 Apr 2009 16:21:54 +0200, groblus wrote:

> terryc pisze:
>> On Mon, 27 Apr 2009 12:40:14 +0200, groblus wrote:
>>
>>> I was wondering on some solutions
>>> that would use multiple network cards from one fileserver.
>>
>> Why not just break the network up into subnetworks on separate LANs?
>>
>>
>> If it is the NIC in the server that is saturating, depending on your
>> router, you could install multiple NICs and use your (managed) router
>> to isolate different boxen into different subnets
>>
>> Alternatively, you could just have different routers connected to
>> different subnets and different groups of boxen.

This is the way it is usually done.


> this might be a nice solution especially for now, but is there some nice
> and elegant solution having one network (without subnetworks) and load
> balance samba shares ?

AFAIK, not for samba, and not for any other service. The load balancers I
know off are to balance/share loads between multiple servers. then you
have the problem of separate data stores unless you have a modern version
of dual ported scsi arrays (aka DEC Storage works).

Can you go FULL DUPLEX on the server NIC.
Have you also checked for network errors?
In the really old days, sometimes puting the server onto a token ring
solved NIC bottlenecks as token ring gave 16MB/sec compared to practical
8MB/sec for ethenet. Sadly, it never made a similar step up.

Bottom line is something will continue to cause "the bottleneck"

or if not by using samba perhaps use of nfs ? or
> I could use some form of hardware link aggregation ? Perhaps thera are
> some double port network card that allow "adapter teaming" under linux ?
> There are number of possibilities and I don't know which to explore.
> Whis will be easiest to implement ?

Naah. I stand to be corrected but unless you are talking point to point,
hardware link aggregtion doesn't work AFAIK.

Put a second NIC in the server, get a second LAN router/switch and move
half the load to a new IP range and the new router/switch network.

terryc <newssevenspam-(E-Mail Removed)> wrote:
> Why not just break the network up into subnetworks on separate LANs?

Also, if you are using NFS, it produces less network traffic than samba, and
the kernel mode NFS server is relatively efficient.

Mark.

--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/

terryc pisze:
> On Mon, 27 Apr 2009 16:21:54 +0200, groblus wrote:
>(...)
>
> Naah. I stand to be corrected but unless you are talking point to point,
> hardware link aggregtion doesn't work AFAIK.
>
> Put a second NIC in the server, get a second LAN router/switch and move
> half the load to a new IP range and the new router/switch network.
>

how about ?
http://www.howtoforge.com/nic_bonding

will samba benefit from that solution ?

Moe Trin <(E-Mail Removed)> wrote:
> 30 computers - 500 Megs each = 15000 total. With those numbers,
> there isn't that much that you can do, as that is beyond the
> capability of the network card, and probably the disks and CPU.

It would be better to cluster the computers together and loadshare with this
amount of data. What sort of data is this? Maybe it would be worth a look at
how this data is generated and used. There may be a more efficient way of
handling this.

Mark.

--
Mark Hobley
Linux User: #370818 http://markhobley.yi.org/

On Mon, 27 Apr 2009 19:04:50 +0200, groblus wrote:


> how about ?
> http://www.howtoforge.com/nic_bonding

If you have the skills (kernel rebuild, mod loading, conditions, etc) and
the hardware, give it a try.

Seems to be a hardware fail over solution, rather than a throughput
solution. note the mention of round robin, aka alternate, rather then
consequtively.

I'm not a guru on NIC stacks, etc. I'm sure there are better qualified
people. I can only suggest hardware solutions based on what I've fiddled
with and practically and economically, that is splitting the LAN.

Now, if the company has lots of money, look at a router with a Fibre port
module and a fibre NIC in the server. Just make sure that the backpane of
the router can service the Fibre Optic speeds and isn't stuck at the 1GB
NIC speed. (if such beasties exist.



--
Once again, our prime minister Kevin Rudd brings stability to the nation
by reassurring the nation that one law still exists for the rich
and another for the poor. After a personal visit;
http://www.abc.net.au/news/stories/2...27/2553855.htm

On Mon, 27 Apr 2009 17:23:21 +0200, Wojtek Pietruszewski wrote:

>> I use
>> gigabit ethernet and at some times it chokes my file server (not it's
>> disks, not it's cpu but it's network).

> google iSCSI, maybe that's what you are looking for.

Could you explan better how you think iSCSI is going to solve his problem?
He already has 1gb NIC and networking, (say 700Mb/sec throughput).






--
Once again, our prime minister Kevin Rudd brings stability to the nation
by reassurring the nation that one law still exists for the rich
and another for the poor. After a personal visit;
http://www.abc.net.au/news/stories/2...27/2553855.htm

On Tue, 28 Apr 2009 00:07:29 +0200, David Brown wrote:


> One idea might be to have multiple NICs on the server

That part is good, economical and safe for your career.

> connected to a serious-sized network switch.

that part is expensive, dangerous and highly likely to fail.

Because a "serious switch" usually costs a serious amount of dollars, you
are are effectively going to have to insert gonads into one of those
incremental squashing device usually found on work benches to get
management to stump up the cach,

And the critical question is THROUGHPUT. I've seen wonderous switches
that did nothing for throughput as there backpane ran at exactly the same
speed as all the legs/NICS, so they were effectively all one segment for
throughput, which means same problem is going to occurr.

> Let each NIC have a different IP address,
> and set up a dns server to round-robin these IP addresses for the
> server's name, using very short timeouts.

AFAIK, round robin =/= throughput increase. See the link he posted, the
kernel still sees ONE NIC, not more.


What we have not yet been told is what investigations as to the cause of
the bottleneck. Ethernet does not fail gracefully, but crashes with
traffic jams. There might be other services causing the problem/
bottleneck that could be moved to another server and solve the problem.




--
Once again, our prime minister Kevin Rudd brings stability to the nation
by reassurring the nation that one law still exists for the rich
and another for the poor. After a personal visit;
http://www.abc.net.au/news/stories/2...27/2553855.htm