copying partition to partition, sector by sector, live

copying partition to partition, sector by sector, live

[Phil Howard]

Under ext2, I could copy a live mounted partition to another partition
of the same size sector by sector, followed by an e2fsck of the target
partition, and have complete all files that are otherwise complete on
the source partition.  Sometimes I might even get files copied this way
that would not have survived a power-reset event because such copying
would usually be done from cache.

Under reiserfs, the picture changes due to the changing roles of the
tools for managing and repairing the filesystem.  What I would like to
find out is what steps would be appropriate to bring a sector copied
reiserfs filesystem back into consistency on the target partition.
Surely I could run the maximal set of recovery tools, but this might
be overkill.  Or maybe not.

A power-reset event would leave a filesystem in the state as stored in
its entirety at the instant of the event.  It would be like freezing
the whole machine and taking a sector by sector snapshot.  One problem
is that doing a sector-by-sector copy of a partition does not get a
time-consistent snapshot.  Instead, you get a time skewed snapshot.
An update to the partition by the filesystem might write a group of
sectors crossing the sectors currently being read by the copying.
This would result in a potential out-of-order snapshot where sectors
written first are not present (written to locations already copied)
and sectors written later are present (written to locations not yet
copied).

How well can the various tools for reiserfs deal with this?  In ext2,
the time to do a raw sector copy is shorter than the time to mount the
target partition as a filesystem and scan both file trees for updates
(method 1) or to reformat the target filesystem and mount the target
and copy every file from the source filesystem (method 2).  This makes
sector copying attractive for backup purposes.  The e2fsck step would
not be routinely performed; it would be done if needed during recovery.

For reiserfs, the issue I'm concerned with is that the time skew may
completely disrupt the balanced tree structure in a way that the journal
cannot replay.  While tools may be able to recover this, it may involve
lengthy processing (rebuilding a tree, for example).  If this is too
lengthy, it may be counter productive.

A consideration exists for going back to backup by copying individual
files.  Before making such a decision, I would like to get an idea what
impact and effect this will have with regard to reiserfs, which I have
begun conversion to.  Since backups are done to guard against loss due
to catastrophic storage device failure, the fact that reiserfs does
journaling is unlikely to have much consequence for the choice; that
feature instead will be expected to aid in faster resumption of normal
operations in lesser failure modes, e.g. power-reset.

-- 
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
| phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/     |
-----------------------------------------------------------------

Re: copying partition to partition, sector by sector, live

[Matthew Toseland]

[-- Attachment #1: Type: text/plain, Size: 37 bytes --]

Ummm, LVM snapshots? (man lvcreate).

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: copying partition to partition, sector by sector, live

[Phil Howard]

On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:

| Ummm, LVM snapshots? (man lvcreate).

No.  Nothing to do with LVM.

-- 
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
| phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/     |
-----------------------------------------------------------------

Re: copying partition to partition, sector by sector, live

[Jonathan Briggs]

[-- Attachment #1: Type: text/plain, Size: 609 bytes --]

On Sun, 2002-04-21 at 19:39, Phil Howard wrote:
> On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:
> 
> | Ummm, LVM snapshots? (man lvcreate).
> 
> No.  Nothing to do with LVM.

In order to do what you are talking about (doing a raw copy of a file
system), you should use a LVM snapshot.  This will prevent the time skew
you were worried about.  The snapshot will not change over time as you
copy it.

When you mount the new copy of the filesystem, it should act as if the
power failed when you took the snapshot, and rerun the journal.
-- 
Jonathan Briggs
jbriggs@esoft.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 232 bytes --]

Re: copying partition to partition, sector by sector, live

[Chris Mason]

On Sun, 2002-04-21 at 21:39, Phil Howard wrote:
> On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:
> 
> | Ummm, LVM snapshots? (man lvcreate).
> 
> No.  Nothing to do with LVM.
>

Doing it safely will require something like lvm or evms snapshots.  You
could do the sector by sector copy and then run reiserfsck
--rebuild-tree.  The latest versions of reiserfsprogs are faster, the
speed relative to a search for updated files will depend on your data
set.

More importantly you just don't get a consistent copy, regardless of the
FS you choose.  I wouldn't consider a sector by sector copy on a mounted
FS a valid backup of any type of filesystem, especially not a tree based
one.

-chris

Re: copying partition to partition, sector by sector, live

[Matthew Toseland]

[-- Attachment #1: Type: text/plain, Size: 1609 bytes --]

On Sun, Apr 21, 2002 at 08:39:55PM -0500, Phil Howard wrote:
> On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:
> 
> | Ummm, LVM snapshots? (man lvcreate).
> 
> No.  Nothing to do with LVM.
I was suggesting a solution. Your problem is that reiserfs's metadata is
so dynamic that if you copy the partition while it is active, you end up
with metadata loss, which has to be fixed by reiserfsck. A possible
solution is to get a consistent snapshot. To do this, do:

lvcreate -n <snapshotname> -L 500M -v /dev/<vgname>/<partition name>
dd if=/dev/<vgname>/snapshotname of=/usr/local/temp/snapshot.img bs=1M
lvremove -f /dev/<vgname>/snapshotname

(1) 500M could be anything; it is the amount of space needed to log all
changes to the partition while the dd is going on; you can extend it later
but once it fills up completely, you're scr00d; presumably the dd will
return an error
(2) set dd parameters as usual
(3) unfortunately the original partition must be LVM, and the snapshot
will be allocated in the same group
(4) /dev/<vgname>/snapshotname is a read-only snapshot of the LV
(partition) at the time of the lvcreate. It will not change, so it gives
you a consistent snapshot. End of problem. No need to do risky fscks,
just take a consistent snapshot in the first place.
> 
> -- 
> -----------------------------------------------------------------
> | Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
> | phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/     |
> -----------------------------------------------------------------
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: copying partition to partition, sector by sector, live

[Phil Howard]

On Mon, Apr 22, 2002 at 11:47:55PM +0100, Matthew Toseland wrote:

| On Sun, Apr 21, 2002 at 08:39:55PM -0500, Phil Howard wrote:
| > On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:
| > 
| > | Ummm, LVM snapshots? (man lvcreate).
| > 
| > No.  Nothing to do with LVM.
| I was suggesting a solution. Your problem is that reiserfs's metadata is
| so dynamic that if you copy the partition while it is active, you end up
| with metadata loss, which has to be fixed by reiserfsck. A possible
| solution is to get a consistent snapshot. To do this, do:
| 
| lvcreate -n <snapshotname> -L 500M -v /dev/<vgname>/<partition name>
| dd if=/dev/<vgname>/snapshotname of=/usr/local/temp/snapshot.img bs=1M
| lvremove -f /dev/<vgname>/snapshotname
| 
| (1) 500M could be anything; it is the amount of space needed to log all
| changes to the partition while the dd is going on; you can extend it later
| but once it fills up completely, you're scr00d; presumably the dd will
| return an error

Sounds like journaling at the sector level.

How does all that change get replayed after the snapshot is done?

I'm starting to think it might be better to go back to using rsync on
mounted filesystems.

-- 
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
| phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/     |
-----------------------------------------------------------------

Re: copying partition to partition, sector by sector, live

[Phil Howard]

On Mon, Apr 22, 2002 at 02:58:34PM -0400, Chris Mason wrote:

| On Sun, 2002-04-21 at 21:39, Phil Howard wrote:
| > On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:
| > 
| > | Ummm, LVM snapshots? (man lvcreate).
| > 
| > No.  Nothing to do with LVM.
| >
| 
| Doing it safely will require something like lvm or evms snapshots.  You
| could do the sector by sector copy and then run reiserfsck
| --rebuild-tree.  The latest versions of reiserfsprogs are faster, the
| speed relative to a search for updated files will depend on your data
| set.
| 
| More importantly you just don't get a consistent copy, regardless of the
| FS you choose.  I wouldn't consider a sector by sector copy on a mounted
| FS a valid backup of any type of filesystem, especially not a tree based
| one.

I've never have a problem doing this with ext2 filesystems.  By comparison
I have had ext2 filesystems totally corrupted by just a power-reset.

But I'm starting to think that with reiserfs, I need to go back to rsync
as the backup mechanism.  Some memory leak and stalling problems with rsync
seem to be fixed, now.

-- 
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
| phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/     |
-----------------------------------------------------------------

Re: copying partition to partition, sector by sector, live

[Chris Mason]

On Mon, 2002-04-22 at 19:24, Phil Howard wrote:
> On Mon, Apr 22, 2002 at 11:47:55PM +0100, Matthew Toseland wrote:
> 
> | On Sun, Apr 21, 2002 at 08:39:55PM -0500, Phil Howard wrote:
> | > On Mon, Apr 22, 2002 at 02:07:33AM +0100, Matthew Toseland wrote:
> | > 
> | > | Ummm, LVM snapshots? (man lvcreate).
> | > 
> | > No.  Nothing to do with LVM.
> | I was suggesting a solution. Your problem is that reiserfs's metadata is
> | so dynamic that if you copy the partition while it is active, you end up
> | with metadata loss, which has to be fixed by reiserfsck. A possible
> | solution is to get a consistent snapshot. To do this, do:
> | 
> | lvcreate -n <snapshotname> -L 500M -v /dev/<vgname>/<partition name>
> | dd if=/dev/<vgname>/snapshotname of=/usr/local/temp/snapshot.img bs=1M
> | lvremove -f /dev/<vgname>/snapshotname
> | 
> | (1) 500M could be anything; it is the amount of space needed to log all
> | changes to the partition while the dd is going on; you can extend it later
> | but once it fills up completely, you're scr00d; presumably the dd will
> | return an error
> 
> Sounds like journaling at the sector level.
> 
> How does all that change get replayed after the snapshot is done?

The snapshots use a simple copy on write setup.  Before changing a block
on the source, the original is copied to the snapshot.  This allows for
very fast snapshot creation, and a moderate runtime cost doing the
copies.

When you're done with the backup, you can just delete the snapshot
without affecting the (now modified) original.

> 
> I'm starting to think it might be better to go back to using rsync on
> mounted filesystems.

It might.  Snapshots are very useful, but are best in databases and
other setups where you need to freeze the FS at a specific point in
time, and when you need an absolute minimum of application down time.

-chris

Re: copying partition to partition, sector by sector, live

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1334 bytes --]

On Mon, 22 Apr 2002 23:47:55 BST, Matthew Toseland said:

> lvcreate -n <snapshotname> -L 500M -v /dev/<vgname>/<partition name>

You missed the -s flag.  From 'man lvcreate' (LVM 1.0.3):

-s, --snapshot
  Create  a  snapshot  logical  volume  (or snapshot) for an existing, so
  called original  logical  volume  (or  origin).   Snapshots  provide  a
  'frozen image' of the contents of the origin while the origin can still
  be updated. They enable  consistent  backups  and  online  recovery  of
  removed/overwritten  data/files.  The  snapshot  does not need the same
  amount of storage the origin has. In a typical scenario,  15-20%  might
  be enough. In case the snapshot runs out of storage, use lvextend(8) to
  grow it. Shrinking a snapshot is supported by lvreduce(8) as well.  Run
  lvdisplay(8)  on  the snapshot in order to check how much data is allo­
  cated to it.

> dd if=/dev/<vgname>/snapshotname of=/usr/local/temp/snapshot.img bs=1M
> lvremove -f /dev/<vgname>/snapshotname

The rest of this looks OK (although I've *NOT* tried it myself).  I'm not
sure what the granularity of the snapshot is - whether you need to allocate
space for each block that's modified, or each 4M segment, or what...


-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: copying partition to partition, sector by sector, live

[The Amazing Dragon]

> From: Phil Howard <phil-reiserfs@ipal.net>
> On Mon, Apr 22, 2002 at 02:58:34PM -0400, Chris Mason wrote:
> | Doing it safely will require something like lvm or evms snapshots.  You
> | could do the sector by sector copy and then run reiserfsck
> | --rebuild-tree.  The latest versions of reiserfsprogs are faster, the
> | speed relative to a search for updated files will depend on your data
> | set.
> | 
> | More importantly you just don't get a consistent copy, regardless of the
> | FS you choose.  I wouldn't consider a sector by sector copy on a mounted
> | FS a valid backup of any type of filesystem, especially not a tree based
> | one.

> But I'm starting to think that with reiserfs, I need to go back to rsync
> as the backup mechanism.  Some memory leak and stalling problems with rsync
> seem to be fixed, now.

More significantly there was a security problem fixed in January.


--
|\__/|\__/|\______          --=> 8-) EHM <=--          ______/|\__/|\__/|
\    |    |       | EHeM@cs.pdx.edu      PGP 8881EF59 |       |    |    /
  \   \   | ______| -O #include <stddisclaimer.h> O-  |______ |   /   /
    \___\_|/82 04 A1 3C C7 B1 37 2A   E3 6E 84 DA 97 4C 40 E6\|_/___/

Re: copying partition to partition, sector by sector, live

[Matthew Toseland]

[-- Attachment #1: Type: text/plain, Size: 1593 bytes --]

On Mon, Apr 22, 2002 at 08:34:39PM -0400, Valdis.Kletnieks@vt.edu wrote:
> On Mon, 22 Apr 2002 23:47:55 BST, Matthew Toseland said:
> 
> > lvcreate -n <snapshotname> -L 500M -v /dev/<vgname>/<partition name>
> 
> You missed the -s flag.  From 'man lvcreate' (LVM 1.0.3):
Whoops. I tried it, then miscopied it, or something :)
> 
> -s, --snapshot
>   Create  a  snapshot  logical  volume  (or snapshot) for an existing, so
>   called original  logical  volume  (or  origin).   Snapshots  provide  a
>   'frozen image' of the contents of the origin while the origin can still
>   be updated. They enable  consistent  backups  and  online  recovery  of
>   removed/overwritten  data/files.  The  snapshot  does not need the same
>   amount of storage the origin has. In a typical scenario,  15-20%  might
>   be enough. In case the snapshot runs out of storage, use lvextend(8) to
>   grow it. Shrinking a snapshot is supported by lvreduce(8) as well.  Run
>   lvdisplay(8)  on  the snapshot in order to check how much data is allo?
>   cated to it.
> 
> > dd if=/dev/<vgname>/snapshotname of=/usr/local/temp/snapshot.img bs=1M
> > lvremove -f /dev/<vgname>/snapshotname
> 
> The rest of this looks OK (although I've *NOT* tried it myself).  I'm not
> sure what the granularity of the snapshot is - whether you need to allocate
> space for each block that's modified, or each 4M segment, or what...
Hmmmm. Presumably it's on the extent level, but I don't know.
> 
> 
> -- 
> 				Valdis Kletnieks
> 				Computer Systems Senior Engineer
> 				Virginia Tech
> 



[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]