A good free backup progie like Rsync

Discussion in 'Linux Networking' started by Gabriel Knight, Oct 26, 2009.

  1. Hi all I need a free program to backup a ubuntu server for my school class,
    it has to be as good or better than Rdiff and Rsync the server will use SSH,
    MYSql and be a file and web server and do a couple of other things. I need
    it to be either a gui or text box program.

    Thanks
    GK
     
    Gabriel Knight, Oct 26, 2009
    #1
    1. Advertisements

  2. Gabriel Knight

    terryc Guest

    Bacula?
     
    terryc, Oct 26, 2009
    #2
    1. Advertisements

  3. Gabriel Knight

    Unruh Guest

    Why not use rsync? I can assure you that it is as good as rsync.
    And why not make it a cron job-- then you do not even need a gui or text
    box for it.
     
    Unruh, Oct 26, 2009
    #3
  4. Amanda? I have always been happy with it. For any backup utility that
    works on the file level the MySQL database will have to be dumped to a
    file prior to running the backup.

    Günther
     
    Günther Schwarz, Oct 26, 2009
    #4
  5. Gabriel Knight

    Joe Guest

    Which makes rsync or rsnapshot perfect for the job. Do the mysql
    backup first, then the rsync or rsnapshot, all from an automated cron
    job. No need for any UI at all.
     
    Joe, Oct 26, 2009
    #5
  6. Grsync.
     
    johnny bobby bee, Oct 27, 2009
    #6
  7. Gabriel Knight

    David Brown Guest

    It depends somewhat on your needs, your database usage, and your
    database server. I don't know about MySQL, but postgresql seems to be
    good at working with such straight file copy backups. If you don't
    write much to your database, your chances of a corrupt copy become
    smaller (and then you just use the previous backup instead). If you are
    using a system like rsnapshot with multiple backup copies that are
    hardlinked when the files are unchanged, you get very small incremental
    costs per backup. With monolithic dump files, even a single change to
    the database means that the whole file must be saved for each backup
    (though rsync will still minimise the traffic transferred in the copy).

    You can take it a stage further by putting your database files on an LVM
    volume and doing an LVM snapshot, which you then use for the backup.
    Then your files are consistent exactly as though the power was turned
    off when the snapshot is taken - and a database server worth the name
    should be able to recover from that.
    You get a fully consistent database snapshot in this way (and as I said,
    proper dumps like this are normally the best choice). But you have
    several downsides. First, you have extra processes to run and
    synchronise (not a problem with cron and a script, but it might be an
    issue for people wanting a simpler system, or if the backup is initiated
    from a different computer). Second, you are locking all the tables
    during the backup - that may or may not be an issue. Thirdly, as you
    say your backups may take more time, and they take more space since you
    can't take advantage of hard-linked copies (diff-based backups are
    horrible - I wouldn't recommend them).


    It's all a matter of balancing your needs. If you are already doing an
    rsync backup of everything else on the machine, including the database
    files is very simple. It may be good enough for you. Doing database
    dumps is the "right" way to do the backups, so that should be the method
    to use unless you have good reason not to. But wanting a simple and
    easy solution without having to learn about dumps may well count as a
    good enough reason.


    Whatever method you choose, do a practice restore to make sure you can
    recover your data!
     
    David Brown, Oct 27, 2009
    #7
  8. Gabriel Knight

    Grant Guest

    I'm using rsync and hard-linked backups run from a cron job each 30 min,
    with another daily cron job eating the backup tail -- I think at age 90
    days. Only problem I had was slocate filling the /var partition -- so I
    uninstalled slocate, I don't use it anyway.
    ....
    Yes, test the restore ability _before_ you need it, way back I discovered
    a booboo in a backup script -- it looked normal but was overwriting files
    with altered most recent, instead of saving the altered file -- easy to
    fix. But the backup was bust at the time I needed a file and found the
    buglet instead.

    Grant.
     
    Grant, Oct 27, 2009
    #8
  9. :Hi all I need a free program to backup a ubuntu server for my school class,
    :it has to be as good or better than Rdiff and Rsync the server will use SSH,
    :MYSql and be a file and web server and do a couple of other things. I need
    :it to be either a gui or text box program.

    If you want something like rdiff and rsync, then rdiff-backup might be a
    good match. It combines the features of a mirror and an incremental
    backup, so you can restore to any previous backup point. The increments
    are kept as a series of reverse diffs from the current mirror. Syntax
    is similar to rsync, and librsync is used to generate efficient reverse
    diffs. Biggest disadvantage is the difficulty of deleting that
    multi-gigabyte file that accidentally got included in your regular
    backup. Learning curve can be a bit steep at first.

    http://www.nongnu.org/rdiff-backup/
     
    Robert Nichols, Oct 28, 2009
    #9
  10. Gabriel Knight

    terryc Guest

    *sync isn't a abckup system. It is just a copy system.
     
    terryc, Oct 28, 2009
    #10
  11. Gabriel Knight

    Grant Guest

    And a backup is not a copy?

    Grant.
     
    Grant, Oct 28, 2009
    #11
  12. Gabriel Knight

    Joe Guest

    A copy *IS* a backup.
     
    Joe, Oct 28, 2009
    #12
  13. Gabriel Knight

    terryc Guest

    A backup system has multiple copies, with historical content.
     
    terryc, Oct 28, 2009
    #13
  14. Gabriel Knight

    jjg Guest

    Well, you _can_ do that with rsync. see e.g. Rubel's article, in
    http://www.mikerubel.org/computers/rsync_snapshots/

    But, strictly you are right; rsync is not, by itself, a backup system.
     
    jjg, Oct 28, 2009
    #14
  15. Gabriel Knight

    Joe Guest

    So does rsync, if you use it that way. Rsnapshot is rsync, and I can
    give you a restore from my backup drive from any day in the last week,
    any week in the last month, and any month in the last year. GFS.
    Rsync, on it's own, can do the same, but takes a little more leg work.
     
    Joe, Oct 28, 2009
    #15
  16. Gabriel Knight

    David Brown Guest

    A backup system means having copies of your data along with a way to
    restore it. You are right that rsync on its own is not a backup system,
    but it can form the backbone of a backup system. If you use it
    appropriately (either with your own scripts or command line arguments,
    or using a ready-made tool such as rsnapshot or dirvish), you then have
    a backup system.
     
    David Brown, Oct 28, 2009
    #16
  17. Gabriel Knight

    terryc Guest

    The basic reason is that problems can happen with "the copy". No matter
    what your system, one copy is no copy at certain times.

    As others have pointed out, it is the system, not the tools (tar, cpio,
    rsync, etc).

    As you say, it is about what he really needs.

    My 2c is that depending on amount of data and frequency of changing, he
    could probably just dump his databases and just burn a DVD each night.
    $AUS50 for a dvd burner and AUS50c/DVD. Even cheap DVD should last a few
    years before they are toasters.
     
    terryc, Oct 28, 2009
    #17
  18. Gabriel Knight

    David Brown Guest

    For an example, consider what happened to a very large software company
    recently when trying to do a hardware upgrade of the storage system.
    There was only one copy of the data (an old copy at that). Before doing
    the hardware upgrade, they wanted to make a new copy - but there was not
    enough space on the backup system. So they deleted their only copy, and
    started making a new copy. Since it was going to take days to make the
    new copy, and they couldn't be bothered waiting, they cancelled the copy
    and did the hardware upgrade anyway. When that failed, everything was gone.

    As you say, having a single copy is not a backup system by itself.

    And even multiple copies on multiple devices at multiple locations is
    not a backup system - you need a tried and tested recovery procedure to
    make it a backup system.
    However, the tools form the central part of the backup system, and
    determine the main features you have available. Thus it does make sense
    to talk about an rsync backup system.
     
    David Brown, Oct 28, 2009
    #18
  19. Gabriel Knight

    terryc Guest

    I actually worked, briefly, for a company that would not buy new backup
    tapes. After weeks of requesting new tapes, I eventually worked into the
    EDP mangers office and dumped a backup report that took up half a box of
    paper, normally 50 pages, and said "It is my belief that if we needed to
    recover from a backup, the company could not do it". Then I held up a
    sample of the 2nd hand tapes they were so proud of obtaining and left
    tape coating over the managers desk. {:).
    First thing you check when you become responsible for the company
    backups. I've also come across a couple of those in my time. This isn't
    too bad if you have a good GFS system and the paper trail (or transaction
    logs) still exists.
    Between scripts running tar, cpio and diff, there hasn't been much else
    in many "branded" backup systems I've used. The part that makes them easy
    to use is the database, query and file selection facilities.

    The only other trick you need to know is that many put multiple jobs onto
    one tape so you need to use /dev/nst0 (no rewind) and move pass the end
    of tape(?) marker to be able to manually recover the contents of some of
    these tapes.
     
    terryc, Oct 28, 2009
    #19
  20. Gabriel Knight

    David Brown Guest

    In case anyone didn't spot the reference:
    I've heard of that kind of thing many times - people often make the
    mistake of equating "tape" with "good backup system", regardless of the
    procedures used for the backup. And since backup tapes cost money, it's
    easy to think that you can save money by buying fewer tapes without
    thinking through the consequences.
    You don't necessarily need something like a GFS. It's often okay for a
    recovery procedure to be somewhat slow and inconvenient, as long as it
    works reliably, and within the required timeframe.

    But while /you/ may check the recovery procedure, a great many people do
    not - that's why I emphasis it so much each time the topic comes up.
    Again, it's often a fault with tape-based backups. People think they've
    got a great system with incremental and differential backups, but when
    disaster strikes they have huge troubles. Maybe they find that they
    need to feed in a week's worth of differential backup tapes just to
    restore a single file. Or they find that when the hardware failure /
    fire / breakin put their server and backup machine out of commission,
    they can't read the tapes on a new machine. Or perhaps their backup
    software won't let them restore part of their data - so they can't
    restore part of yesterday's backup without destroying today's good data.
    There are many ways for restores to fail even when the data is
    technically safe, but people often don't think through and test their
    recovery procedures.
    I find rsync particularly useful for doing offsite backups over the
    Internet, since it minimises the traffic.
    My experience with tape-based backups is that it involved too much
    manual work (changing tapes), recovery was slow and awkward, it was very
    difficult to be sure of the reliability of the system, and we had no
    practical (or cheap) way to test recovery via another machine - if the
    server and the backup machine with the tape drive died, we would have
    great difficulty getting the data back. For many years now, I've used
    rsync to two separate machines in two locations, with hardlinked copies
    to give snapshots without taking more space than necessary. Of course,
    the type of backup strategy you need depends entirely on the situation
    and the quantity of data.
     
    David Brown, Oct 28, 2009
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.