queuing up processes spawned by xinetd

So I have a daemon which is spawned by xinetd. It serves
files to other machines on the network... think of a
specialized version of rcp.

Due to the nature of our compute farm, occasionally a lot
of machines (several hundred) will request a large file
from a single linux box.

This will in turn spawn several hundred copies of my
program, which will make them all very slow since they
are sharing the network bandwidth to server the file.

So, I would like to to have the spawned programs queue
up the request, so the first client program can receive
an entire copy of the file and begin work, followed by
the second, etc.

Suggestions please? Is there some slick xinetd trick
to do this? If not, what's the best approach? I would
prefer to have something which does not have a lot of
state hanging around so that a crashed server process
blocks the machine from serving other clients.

Many TIA!
Mark

--
Mark Harrison
Pixar Animation Studios

Mark Harrison <(E-Mail Removed)> writes:

> So I have a daemon which is spawned by xinetd. It serves files to
> other machines on the network... think of a specialized version of
> rcp.
>
> Due to the nature of our compute farm, occasionally a lot of
> machines (several hundred) will request a large file from a single
> linux box.
>
> This will in turn spawn several hundred copies of my program, which
> will make them all very slow since they are sharing the network
> bandwidth to server the file.
>
> So, I would like to to have the spawned programs queue up the
> request, so the first client program can receive an entire copy of
> the file and begin work, followed by the second, etc.

With normal inetd you could just select "wait" instead of "nowait" as
the run mode. As long as your process does in fact exit each time,
the kernel and inetd will effectively queue arriving TCP connections
and run one process at a time to service them.

I would expect that xinetd also has a wait/nowait setting, but I
haven't used it in years so you'll have to check the manpage.

Depending on file size, LAN latency, etc, you may well find that
overall throughput is significantly less when only running one peer's
download at once. Hopefully your files are largeish and total
throughput is not the most important requirement.

You may also find that the potential for hosure is high; if a task
gets stuck, the entire world will stop.


FWIW it should be no trouble at all to serve 100s of clients at once
on any recent computer. However the usual fork-per-client model won't
do it, and you don't seem keen to recode your daemon.

--
Grant Taylor
Embedded Linux Consultant
http://www.picante.com/