[linux-lvm] resize of LV causing superblock corruption ( Bad magic number ). how to prevent/fix?

[linux-lvm] resize of LV causing superblock corruption ( Bad magic number ). how to prevent/fix?

[darx]

I have three LVs

	lvs | grep test
		test_boot   VGTEST -wi-a---- 200.00m                     
		test_root   VGTEST -wi-a----   4.00g                     
		test_swap   VGTEST -wi-a---- 512.00m           

They contain healthy/usable ext4 partitions, and are routinely,
successfully used as disks in launching a Xen guest

	cat test.cfg
		name = 'test'
		...
		disk = [ 'phy:/dev/VGTEST/test_boot,xvda,w',
		'phy:/dev/VGTEST/test_swap,xvdb,w',
		'phy:/dev/VGTEST/test_root,xvdc,w', ]
		...

	xl create test.cfg
	xl list
		Name          ID   Mem VCPUs      State   Time(s)
		Domain-0       0   985     4     r-----     101.2
		test           3  1024     2     -b----       7.6

	xl console test
		test login: darx
		Password: #######
	uname -a
		Linux test.lan 3.7.10-1.16-xen #1 SMP Fri May 31
		20:21:23 UTC 2013 (97c14ba) x86_64 x86_64 x86_64
		GNU/Linux

I need to grow one of the LVs, /dev/VGTEST/test_root by 8G.

To do that I 1st verify that it's healthy,

	kpartx -av /dev/VGTEST/test_root
	e2fsck -f /dev/mapper/VGTEST-test_root1
		e2fsck 1.42.6 (21-Sep-2012)
		Pass 1: Checking inodes, blocks, and sizes
		Pass 2: Checking directory structure
		Pass 3: Checking directory connectivity
		Pass 4: Checking reference counts
		Pass 5: Checking group summary information
		/dev/mapper/VGTEST-test_root1: 84121/262144 files (0.3%
		non-contiguous), 753868/1048320 blocks
	kpartx -dv /dev/VGTEST/test_root

then extend the partition,

	lvextend -L+8G /dev/VGTEST/test_root
	  Extending logical volume test_root to 12.00 GiB
	  Logical volume test_root successfully resized

	fdisk -l /dev/VGTEST/test_root | grep Linux
		/dev/VGTEST/test_root1            2048     8388607    
		4193280   83  Linux

delete the current partition, and re-add it as a new partition using
100% of the new/expanded available space

	cfdisk /dev/VGTEST/test_root/
		...
	fdisk -l /dev/VGTEST/test_root | grep Linux
		/dev/VGTEST/test_root1              32    25165823   
		12582896   83  Linux

then prepare to resize the FS on the grown partition

	kpartx -av /dev/VGTEST/test_root
		add map VGTEST-test_root1 (253:13): 0 25165792 linear
		/dev/VGTEST/test_root 32

and FSCK to be safe.  BUT, here it fails:

	e2fsck -f /dev/mapper/VGTEST-test_root1
		e2fsck -f /dev/mapper/VGTEST-test_root1
		e2fsck 1.42.6 (21-Sep-2012)
		ext2fs_open2: Bad magic number in super-block
		e2fsck: Superblock invalid, trying backup blocks...
		e2fsck: Bad magic number in super-block while trying to
		open /dev/mapper/VGTEST-test_root1

		The superblock could not be read or does not describe a
		correct ext2
		filesystem.  If the device is valid and it really
		contains an ext2
		filesystem (and not swap or ufs or something else), then
		the superblock
		is corrupt, and you might try running e2fsck with an
		alternate superblock:
		    e2fsck -b 8193 <device>

at this step I *used* to (a year or so ago, when I last did this) get a
good FSCK result, before then proceeding with

	resize2fs /dev/mapper/VGTEST-test_root1
	kpartx -dv /dev/mapper/VGTEST-test_root

to complete the resize.

What's likely causing this new/current superblock corruption, and how do
I prevent it?

Thanks.

darx

Re: [linux-lvm] resize of LV causing superblock corruption ( Bad magic number ). how to prevent/fix?

[Bryn M. Reeves]

This is an fdisk rather than LVM2 problem.

Compare:

On 07/21/2013 09:34 PM, darx@sent.com wrote:
> 	fdisk -l /dev/VGTEST/test_root | grep Linux
> 		/dev/VGTEST/test_root1            2048     8388607    
                                                  ^^^^
> 		4193280   83  Linux

vs.:


> 	fdisk -l /dev/VGTEST/test_root | grep Linux
> 		/dev/VGTEST/test_root1              32    25165823   
                                                    ^^
> 		12582896   83  Linux

I.e. your fdisk program moved the start of the partition. That'll pretty
much break any file system that has fixed superblock locations.

Either use an fdisk that matches the defaults used when the partition
was first created or ensure you are using sector display units (looks
like you are - 2048 == 1MiB which is the default used by modern fdisks
from util-linux) and carefully match the start location when
deleting/re-adding.

Btw this is fixable as long as you can still find the correct starting
sector.

Regards,
Bryn.

Re: [linux-lvm] resize of LV causing superblock corruption ( Bad magic number ). how to prevent/fix?

[darx]

Bryn,

On Mon, Jul 22, 2013, at 02:58 AM, Bryn M. Reeves wrote:
> I.e. your fdisk program moved the start of the partition. That'll pretty
> much break any file system that has fixed superblock locations.

I completely missed that, as I've never seen it happen before.  Thanks
for catching that.

I suspect my memory's of a procedure that used an older fdisk ... I'll
have to check exactly when the default/minimum became the 'modern' value
== 2048.

> Either use an fdisk that matches the defaults used when the partition
> was first created or ensure you are using sector display units (looks
> like you are - 2048 == 1MiB which is the default used by modern fdisks
> from util-linux) and carefully match the start location when
> deleting/re-adding.

Oddly, when I use 'my' fdisk to 1st create a new partition, it does
default to 2048 minimum sectors -- which, as you point out, is the value
used in my setup.

After the resizing, however, fdisk on the resized volume offers only a
minimum of 4096 sectors.  I.e., I'm unable to *match* the sector start
== 2048.

I've fixed the problem by cloning my source to a set of partitions that
start @4096.  Now, with fdisk on resize able to match the sector start,
my extended partition fsck's just find -- NO superblock issues.

I've no clue yet why fdisk changes its default from 2048->4096 before &
after the resizing.  But, for now I"m back in business.

Thanks for the help!

darx