Backup-solution

Discussion in 'General Questions' started by RobertOe, Nov 17, 2015.

  1. RobertOe

    RobertOe Bit Poster

    Messages:
    5
    Hello guys.

    I'm posting here and not in Virtuozzo since the issue is with Storage. I'm currently running R1Soft server backup, however, it don't play nice with Storage at all. They promised it should work, but now they admit they don't have any knowledge about it. Got to love the sales people...

    Anyway, I'm looking for something that can do an incremental forever. It seems like a block-based system is close to useless on the hardware nodes since the containers can move around. So I guess that means a system that can be installed on the containers itself and not on the hardware nodes.

    Sure, I could backup the containers using the internal bacup-system, but I can't store 50 images of each of the containers (the number of days we promised our clients).

    Has anyone any experience with backup of the clients. And I prefer an incremental forever, the old "tape-solutions" with full/incremental/diff....

    TIA
    Robert Oedegaard

     
  2. Pavel

    Pavel A.I. Auto-Responder Odin Team

    Messages:
    421
    Hello Robert,

    R1soft performs backup on a block-level of the hardware node, thus, indeed, it won't work with pstorage at all because it's a network-clustered-shared FS.
    You can workaround necessity to run backups inside of CT by creating backups to a single backup-node in cluster. Or, having several nodes If backup-host is always the same it should negate problems with CT location entirely.

    Storing 50 full backups of a container indeed would be quite costly, however, what about 50 incremental backups? Or differential backups?
    These are possible with "prlctl backup", and do not require a 3rd-party solution.

    If that doesn't suit you I'd like to know why, just so I know what exactly is your goal and requirements.
     
  3. SteveITS

    SteveITS Mega Poster

    Messages:
    211
    Hi Robert, it sounds like you're trying to end up with an incremental backup each day, and then have the system roll up the oldest day into the full backup image? The buillt in backup can't do that but will do incremental backups until it needs to generate a new one. You could set "Limit the maximum number of incremental backups before creating a new full backup" to 50 and "Number of full backups to keep" to 1. Perhaps Pavel knows, but in that scenario I wonder if the system will create a full backup on day 51 and then erase the previous 50 days? Quickly looking at ours (set to keep 3 full) it looks like that might be the case. You could set it to 2 full backups with 50 incrementals and end up with 50-99 days of coverage. Or 3 and 25 for 50-75 days.

    The built-in backup works great with containers, and restores are super easy through PVA. Backups of VMs are a bit more limited in that it makes an incremental image but it can't see inside the file system to restore so you have to do a full restore. I believe they've just improved that so if backups are stored locally they don't require a full restore. A VM might be the case where R1Soft or other backup software running inside the VM would work (we have one node set with just Windows Backup, for instance, which uses VSS snapshots).

    We have our backups set to go to a separate hardware node, with storage outside of the Virtuozzo storage.
     
  4. Pavel

    Pavel A.I. Auto-Responder Odin Team

    Messages:
    421
    Steve, thank you for taking part in this conversation, you've helped me to realise the main point of the question - necessity to keep 50 backups all the time, and create new/delete old every day, somehow this idea wasn't clear to me initially.

    Indeed, one can create incremental backups on a schedule, however, removing old incremental backups would render the remaining chain useless which means they won't quite do the trick.

    There are two approaches which might pick your interest:
    1) Differential backups on a daily basis.
    They require one full backup, and afterwards you're free to do differential backups. Technically speaking differential backups are just an incremental backups to the same "Full" backup all the time. Main profit - you can do them via PVA/XML API, and you can remove old ones without breaking the chain. And you can take these backups to some external host which is good if you want to increase data protection (golden rule of the backup-making is that backups must be stored on an external server, in case if main one goes down completely)
    Another profit from this approach is that these backups are really straightforward and easy to understand compared to the snapshots approach (described next)

    There are some difficulties with this approach however - it will be necessary to arrange backup rotation manually, as currently PVA will rotate "full" backup with its chain when limit is reached, meaning it will not allow you to keep 50 backups constantly. Schema is not transparent unfortunately.

    2) Snapshots on a daily basis.
    This approach will let you have 50 "images" on a daily basis, and ability to rotate old ones + create new ones, like a tape-solution, however, there are downsides to this approach:
    a) PVA does not have functionality to manage snapshots currently, it will require custom solutions to manage them
    b) snapshots will be stored the same place where CT is stored, and if, by any mean, storage gets offline, the snapshots will not available anymore as well. Pstorage is reliable of course, but that's not only a matter of service availability, it might be a human error factor too - I've seen clients accidentally doing "rm -rf " on pstorage mount point, removing all the data.

    To conclude - using snapshots is an option, but it's too risky and difficult to implement compared to Differential snapshots.
    I would definitely recommend trying differential backups approach.

    Speaking of 3rd-party solutions - I'm not aware of any that would help you with this dilemma.
     
  5. RobertOe

    RobertOe Bit Poster

    Messages:
    5
    Hello guys

    I'm so sorry for not replying earlier. Busy week.

    Well, not sure if I manage to explain it very well, English is not my native language. But basically I want exactly what R1Soft delivers, by keeping any versions of a file the last 50 days. On my other servers this works like a dream, maybe with the exception of the automatic off-site backup, but I can live with that issue.

    After some consideration I think I found a solution I can live with, at least in theory, have to see how it works in real life. What I will do is set up an rsync job to synchronize all the files, with delete enable, from either the HW nodes or from each of the containers. This will sync to a cheap NAS box with enough storage and then run R1soft from there and keep that backup for 50 days.

    It is not even close to a perfect system, but for the time being I think that would be the best I can do. The issue might be that the backup can become quite huge fast and R1Soft don't handle that very well, but at least it is better than running it directly on the HW nodes.

    /Robert
     
  6. Pavel

    Pavel A.I. Auto-Responder Odin Team

    Messages:
    421
    Hello Robert,

    R1soft is pretty much doing the same - either incremental or differential backup, not sure what's under the hood.
    Setting up a NAS sounds like an overkill to me, but if you're OK with that - I'lm happy for you. Just one thing you might take into consideration - if you're doing rsync from inside of a container (lets say, from /vz/root/CTID/), it's ok to run rsync. But if you preform it on /vz/private/CTID/ things might get nasty as virtual disk images are constantly changing, thus running rsync will either fail or produce inconsistent image.
    If you will be doing a backup of "/vz/private/CTID" make sure to create a snapshot and exclude it from a backup - that will give you a "static" base image and a "floating" top delta which you might as well ignore in the backup. Once backup is over - simply merge it back. This approach will guarantee consistency of a backup.
     
  7. SteveITS

    SteveITS Mega Poster

    Messages:
    211
    Best Answer
    Robert, if you're looking at rsync, someone had pointed me to rdiff-backup which combines rsync and diff to maintain a history. That may be another method for what you want to do.
     

Share This Page