Pros and Cons of using ftp vs. rsync?

Discussion in 'Linux Networking' started by Goran Ivanic, May 3, 2008.

  1. Goran Ivanic

    Goran Ivanic Guest

    Assuem I want to transfer large amounts of stuff from one server to another (through Internet).

    Which method should I prefer:

    ftp or rsync ?

    What are the Pros and Cons?

    Which is faster?

    Which is more stable?

    Goran
     
    Goran Ivanic, May 3, 2008
    #1
    1. Advertisements

  2. Goran Ivanic

    Dave Guest

    It depends.

    If you need to send a lot, and none of it exists at the far end, then
    ftp will be faster. ftp just needs to transfer the files, with no
    overhead checking what versions exists at each end.

    My guess is ftp will be faster. It just moves the files, and does not
    care whether they exist on the other end or not.

    However, if a large number of files to be transfered already exist at
    the far end, then rsync will be faster as it needs to transfer less data.
     
    Dave, May 3, 2008
    #2
    1. Advertisements

  3. Ftp is not secure. I would not use it to transfer large amounts of stuff
    from one server to another, unless what you are doing is something like
    mirroring a ftp site. If you are planing to provide a mirror to a ftp
    site, you should talk to the sysadmin of the site you are mirroring.
    They probably have a rsync server in place for this purpose.

    Otherwise...

    Using rsync over ssh (rsunc -e ssh ...) or using tar over ssh (tar czvf -
    -C sourcedir files | ssh othermachine tar xzvf - -C destdir) are both
    good options. I'd use the tar/ssh route if this is a new / first time
    transfer. Using rsync is better if it is an update (some random subset
    of files need to be transfered).
     
    Robert Heller, May 3, 2008
    #3
  4. Goran Ivanic

    Unruh Guest

    On large files that is a trivial overhead. rsync can also checks if the
    files transfered are the same or not. ftp does not
    From man rsync
    Note that rsync always verifies that each transferred file was
    correctly reconstructed on the receiving side by checking its
    whole-file checksum,...
     
    Unruh, May 3, 2008
    #4
  5. I'd also take a look into 'unison', it is faster the rsync in
    certain situation and its GUI might make things easier for
    beginners, though you really want to use it from the shell to
    take most advantages.
     
    Michael Heiming, May 3, 2008
    #5
  6. http://stromberg.dnsalias.org/~strombrg/copy-lots-of-data.html

    rsync is good at picking up where a prior transfer left off. rsync
    defaults to ssh, but can use rsh or similar instead. ssh is too CPU-
    intensive to get decent performance on gigabit or better networks even
    with contemporary CPU's, but there are patches to openssh that take Some
    of the performance hit out of it.

    And as Robert said, ftp (and rsh and NFS) are not encrypted, so don't use
    them to transfer anything you need to keep private (unless you combine
    them with mcrypt or similar). These tools are fine for copying something
    you'd be putting on a public web site anyway, for example (even without
    mcrypt).

    NFS is actually a pretty good performer on gigabit and better networks,
    because it's able to make good use of jumbo frames. This despite NFS
    giving lackluster performance on 10BaseT and 100BaseT. NFS reads are
    quite a bit faster than NFS writes.

    ssh (including with rsync), ftp, rsh (including with rsync) and NFS are
    all pretty stable, though NFS is perhaps a little less so depending on
    the implementations involved.

    rsync gives OK progress information - not stellar. Modern ftp clients
    like tnftp (formerly lukemftp) give good progress information. ssh and
    rsh and NFS don't give progress info, but can give quite good progress
    information if you combine them with a tool like http://
    stromberg.dnsalias.org/~strombrg/reblock.html (there's a short list of
    similar programs at the bottom of the page).
     
    Dan Stromberg, May 4, 2008
    #6
  7. Here's a comparison of ssh, rsh, rsync, NFS, ftp and pnetcat for such a
    purpose:

    http://stromberg.dnsalias.org/~dstromberg/protocol-comparison.html
     
    Dan Stromberg, May 4, 2008
    #7
  8. Goran Ivanic

    Chris Davies Guest

    Either you do or you don't. It's hard enough understanding people's
    questions without having unnecessary assumptions thrown about.

    Your preference is entirely up to you. Personally, if it really was
    "large amounts of stuff", I'd consider sending a tape through the post.

    I think you probably ought to go and do your own homework, don't you?
    Chris
     
    Chris Davies, May 4, 2008
    #8
  9. Goran Ivanic

    Unruh Guest

    Using rsync on ssh, the site
    http://www.psc.edu/networking/projects/hpn-ssh/
    shows taht it depends on far away the other machine is. If it is on the
    same network where the roundtrip times are say 100usec, then even on Gb
    links the standard ssh buffer is fast enough. If the machine to which you
    are transfering stuff is much further away (many 10s-100s of msec) then ssh
    acts as a bottleneck. But the code on that page claims to fix that problem.

    So, for stability and for verification of the transfer, it is hard to beat
    rsync. ftp, nfs,... do not verify that the data received is the same as the
    data transfered. Of course you can put in an extra step to verify it.
     
    Unruh, May 4, 2008
    #9
  10. Goran Ivanic

    Nikhil Guest

    hey Michael,

    what else unison can offer in particular what rsync cannot at this point
    of time? What I understand is rsync is a one way transferr system
    whereas unison can do multi-way sync of file transferrs across like
    wansync/intellisync ... is that correct?
     
    Nikhil, May 4, 2008
    #10
  11. Goran Ivanic

    Moody Guest


    I would rather go for rsync, it gives you a lot more functionality
    over ftp,,,you may customize and do stuff like files to retain or
    not...
     
    Moody, May 5, 2008
    #11
  12. Assuming your FTP connection is secured (SSL/TLS), for first-time
    transfer, it may be better than Rsync, whereas Rsync is better for
    incremental updates, and also for first-time transfers using suitable
    command line switches. One further advantage of Rsync is that you can
    make use of your existing SSH PKI.

    Depends on many details. I don't know whether in FTP you can reuse the
    data channel, but if you can't, Rsync should be faster in almost all
    cases. FTP may be faster for a few large files. Rsync is almost
    certainly faster for lots of tiny files, and so on. In some cases, tar
    used cleverly through SSH may be faster than both, but usually Rsync
    will be just fine.

    FTPS and Rsync are both very stable.


    Regards,
    Ertugrul.
     
    Ertugrul Söylemez, May 5, 2008
    #12
  13. Goran Ivanic

    Todd H. Guest

    rsync over ssh is the way to go in most cases. The most compelling
    reasons against ftp are its cleartext method of credential exchange
    and braindeadness with respect to interrupted transfers. You'll be
    retransmitting everything if you have an issue half way through.
    With rsync, this issue is avoided.

    rsync -avz -e ssh /home/blah user@remotehost:/blah
     
    Todd H., May 5, 2008
    #13
  14. There's really no reason to prefer ftp over rsync, except maybe that most
    browser have ftp support but not always rsync.

    If you are going to have many simultaneous transfers, you may not want to do
    it over ssh as it will be too CPU intensive over a fat pipe.
     
    Guillaume Dargaud, May 5, 2008
    #14
  15. Indeed unison can work in both directions at the same time,
    though it's (iirc) also based on the rsync protocol it builds
    some database on the first run and will use this on subsequent
    runs. Then it outperforms rsync in order of magnitudes with very
    large filesystems you want to sync.
     
    Michael Heiming, May 6, 2008
    #15
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.