Network Distributed File System

Hello,

I am trying to find a solution for distributing the storage of large
amounts of data in our network.

We operate a file distribution system based on bittorrent and we host
all the files on the network which require quite a lot of storage. Our
file serving network is comprised of several old computers (low
performance but high storage space) which run apache and use webseeding
to send the files. We do not plan to use a high performance central
storage solution because of costs and because we want to keep our
network as flexible as possible by seamlessly adding and removing
machines.

So I am looking for some kind of system (maybe a filesystem, or some
clustering software) which allows multiple directories on the network
(I'm guessing access will be through NFS) to be available as a single
directory. In this 'virtual directory' files added will be
automatically placed by the system onto a machine with the most
available drive space, and, of course, files could be deleted. Also
files could be accessed by any machine in the network. Another feature
would be adding and removing machines easily, just by changing a config
file, possibly on the fly (without restarting any deamons)

I imagine a possible configuration to be something like this:

- List of network directories making up the 'virtual dir':
/mnt/machine1, /mnt/machine2, ... where these directories are mounted
through nfs.
- Virtual dir: /mnt/storage
- Apache running and serving files from /mnt/storage
- Dns load balancing forwarding requests to one of the machines so
calls to download storage.mydomain.com/myfile.dat could be downloaded
from an outside machine.

Am I looking for trouble and using a central storage system would be
more efficient? If so, why?

Andrei Stoian

(E-Mail Removed) wrote:

> Am I looking for trouble and using a central storage system would be
> more efficient? If so, why?

What amount of storage space do you require?
You can do exactly the same with a single computer, several large SATA
drives and, if required, multiple NICs to increase network performance.
It would be a low cost solution, and by using LVM you can add more hard
drives later if required.

--
Regards,
ToreS

Tore Skogly wrote:
> (E-Mail Removed) wrote:
>
> > Am I looking for trouble and using a central storage system would be
> > more efficient? If so, why?
>
> What amount of storage space do you require?
> You can do exactly the same with a single computer, several large SATA
> drives and, if required, multiple NICs to increase network performance.
> It would be a low cost solution, and by using LVM you can add more hard
> drives later if required.
>
> --
> Regards,
> ToreS

I'm guessing we would grow to about 5-7TB in quite a short time so I
want a solution that scales well.