Backups

Backups

[Gandalf Corvotempesta]

Hi,
let's assume a huge ceph clusters, with 300TB full of datas (rgw, rbd
and the future cephfs)
How will you manage backups of this cluster?

Will snapshot be enough? For example, one snapshot per day should be
allow me to revert or restore a single file from the previous day.

Re: Backups

[Michael Grosser]

Hey,

I thought about some ideas, but I didn't really get any response.
You can take a look at some concepts:
http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/12061

The easiest is probably to have a backup ceph system and use this to
store backups and then for the different backups:
for cephfs for each fs do a snapshot and upload it to the backup ceph system.
for rgw loop through all containers and loop through all files and
backup them to the backup ceph system.
for rbd snapshot blockdevices and backup them to the backup ceph system.

With this approach you have the ability to have versioned backups if
you use some sort of versioning when you put all monthly ones into
their own pool for example or so.

The downside so is that it really strains the system to make backups
of the whole data and not partially the changed data (I imagine) and
second the bandwidth needed for storing these backups would be for
each backup the whole storage size of the production ceph cluster so
bandwidth saving with only storing changed stuff could be hard.

Just a few hints.

Cheers Michael

On Sat, Feb 16, 2013 at 5:15 PM, Gandalf Corvotempesta
<gandalf.corvotempesta@gmail.com> wrote:
> Hi,
> let's assume a huge ceph clusters, with 300TB full of datas (rgw, rbd
> and the future cephfs)
> How will you manage backups of this cluster?
>
> Will snapshot be enough? For example, one snapshot per day should be
> allow me to revert or restore a single file from the previous day.
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ceph-users] Backups

[Stefan Priebe]

Hi,

I think the bigges problem is that an incremental backup is missing in 
ceph for the rados object. So you can just copy / export the changes 
rados objects from an rbd volume instead the whole volume again and 
every day.

Greets,
Stefan
Am 16.02.2013 17:15, schrieb Gandalf Corvotempesta:
> Hi,
> let's assume a huge ceph clusters, with 300TB full of datas (rgw, rbd
> and the future cephfs)
> How will you manage backups of this cluster?
>
> Will snapshot be enough? For example, one snapshot per day should be
> allow me to revert or restore a single file from the previous day.
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>

Re: [ceph-users] Backups

[Gandalf Corvotempesta]

2013/2/16 Stefan Priebe <s.priebe@profihost.ag>:
> I think the bigges problem is that an incremental backup is missing in ceph
> for the rados object. So you can just copy / export the changes rados
> objects from an rbd volume instead the whole volume again and every day.

Could you explain by an example?