[dm-crypt] Filesystem unreadabe after resizing LUKS

[dm-crypt] Filesystem unreadabe after resizing LUKS

[Pawel Chojnacki]

[-- Attachment #1: Type: text/plain, Size: 4195 bytes --]

Hello,

Two days ago I tried to shrink my LUKS-encrypted /dev/sdb5 partition from
119 to 110 GB according to
http://ubuntuforums.org/showthread.php?t=726724. Each step worked
properly, but in the end I'm not able to mount the
partition. I wanted to ask if you have any idea what may have caused that
and how to fix it:

# cryptsetup luksOpen /dev/sdb5 neuro

accepts the proper password, doesn't recognize any other, and fails to
mount the partition inside:

# mount /dev/mapper/neuro /mnt/
mount: unknown filesystem type 'LVM2_member'

# lvdisplay
  --- Logical volume ---
  LV Path                /dev/xubuntu-vg/root
  LV Name                root
  VG Name                xubuntu-vg
  LV UUID                WjTjmo-j1FO-vDej-QE5X-Qym6-DanB-B8Bnhx
  LV Write Access        read/write
  LV Creation host, time xubuntu, 2013-06-03 21:57:17 +0200
  LV Status              suspended
  # open                 0
  LV Size                110.00 GiB
  Current LE             28160
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:4

# pvdisplay
  --- Physical volume ---
  PV Name               /dev/mapper/neuro
  VG Name               xubuntu-vg
  PV Size               110.01 GiB / not usable 1.24 MiB
  Allocatable           NO
  PE Size               4.00 MiB
  Total PE              28162
  Free PE               2
  Allocated PE          28160
  PV UUID               3qmIAm-TFhv-ZzJQ-DLXR-SUPI-5CF8-V6JB6I

# fdisk -l /dev/sdb

Disk /dev/sdb: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders, total 625142448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d0d6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048      499711      248832   83  Linux
/dev/sdb2          499712   230709585   115104937    5  Extended
/dev/sdb5          501760   230709585   115103913   83  Linux

# fdisk -l /dev/mapper/neuro

Disk /dev/mapper/neuro: 117.9 GB, 117864309760 bytes
255 heads, 63 sectors/track, 14329 cylinders, total 230203730 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/neuro doesn't contain a valid partition table

As far as I understand, the physical volume is 110 GB large, and filesystem
is 117.9 GB (why?). When trying to read from superblocks:

Code:

# fsck.ext4 /dev/mapper/neuro
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext4: Superblock invalid, trying backup blocks...
fsck.ext4: Bad magic number in super-block while trying to open
/dev/mapper/neuro

# mke2fs -n /dev/mapper/neuro
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
7200768 inodes, 28775466 blocks
1438773 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
879 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872

When trying to replace any of the listed superblocks:

Code:

# fsck.ext4 -b 32768 /dev/mapper/neuro
e2fsck 1.42.9 (28-Dec-2013)
fsck.ext4: Bad magic number in super-block while trying to open
/dev/mapper/neuro

Testdisk recognizes the filesystems as FAT32/FAT12/NTFS and can't do much.
Photorec on the other hand can seamlessly recover all data as long as it's
told that the filesystem is ext4.

From what I understand, the first stem in the resizing process - resize2fs
-p /dev/mapper/neuro-root 9G - has gone wrong - even though all the tests
done in the meantime worked.

I have a backup from 3 days ago, yet it's a question of honor for me to
understand what has gone wrong and how to repair it. I will owe a beer to
whoever helps me.

Regards,

Pawel "Aleksander" Chojnacki

[-- Attachment #2: Type: text/html, Size: 4923 bytes --]

Re: [dm-crypt] Filesystem unreadabe after resizing LUKS

[Matthias Schniedermeyer]

On 30.01.2014 16:24, Pawel Chojnacki wrote:
> Hello,
> 
> Two days ago I tried to shrink my LUKS-encrypted /dev/sdb5 partition from
> 119 to 110 GB according to
> http://ubuntuforums.org/showthread.php?t=726724. Each step worked
> properly, but in the end I'm not able to mount the
> partition. I wanted to ask if you have any idea what may have caused that
> and how to fix it:

The actual commands you executed would be useful.

> As far as I understand, the physical volume is 110 GB large, and filesystem
> is 117.9 GB (why?). When trying to read from superblocks:

A filesystem has the information stored about it's size stored inside 
itself, it doesn't needs (or uses) the information about the outer 
container-size for that.

You have a problem when the outer size doesn't correspond with the 
stored filesystem size. Altough actually problematic is only the 
"filesystem is bigger" case, the other case is (potentially) just wasted 
space.

As for shrinking, this is an operation that is NOT guruanteed to be 
successful. There maybe files in the space you want to shrink away and 
AFAIK there aren't any (OSS) tools that relocate files (except the 
XFS-specific xfs_fsr, but more about XFS in the next paragraph). The 
resize2fs-manpage isn't really informative. You would have to check the 
file-layouts of EVERY file. "filefrag" can tell you where the physical 
blocks of a file are located, you would need to calcutate the offset 
that will ouside the shrinked filesystem and move any file that is 
beyond the offset (Just copying such a file away and back MIGHT be 
enough, but that depends a little on luck)
After that you would need to hope that there aren't any "other" things 
in "to be removed" space. Like for e.g. directory-data

For e.g. i can definitly say that you can't shrink an XFS, because there 
simply isn't any tool that can do that. It's technically possible to do 
that, but nobodoy bothered to create the necessary tool which would 
check & relocate files/directores... and then fix up the metadata of the 
filesystem. There are people that ask the question from time to time. 
But "Backup & Restore" is still the only "officially supported" way to 
shrink a XFS filesystem.

The only Linux filesystem (i know) that will definitly be able to 
"shrink" is BTRFS (Keyword: rebalancing), altough i'm unsure if it can 
do that today or at some point in the future.





-- 

Matthias

Re: [dm-crypt] Filesystem unreadabe after resizing LUKS

[Arno Wagner]

On Thu, Jan 30, 2014 at 16:24:27 CET, Pawel Chojnacki wrote:
> Hello,
> 
> Two days ago I tried to shrink my LUKS-encrypted /dev/sdb5 partition from
> 119 to 110 GB according to
> http://ubuntuforums.org/showthread.php?t=726724. Each step worked
> properly, but in the end I'm not able to mount the
> partition. I wanted to ask if you have any idea what may have caused that
> and how to fix it:
> 
> # cryptsetup luksOpen /dev/sdb5 neuro
> 
> accepts the proper password, doesn't recognize any other, and fails to
> mount the partition inside:
> 
> # mount /dev/mapper/neuro /mnt/
> mount: unknown filesystem type 'LVM2_member'

This means LUKS is fine, just the resize failed. 
I should pount out the clear "It should go without 
saying, resizing your crypt may result in data loss  
Be sure to BACK UP your data first." given in that posting.

I also have to say that this is one of the reasons I think that
lvm has no place in a normal installation: It just complicates
things.
 
[...]

> Testdisk recognizes the filesystems as FAT32/FAT12/NTFS and can't do much.
> Photorec on the other hand can seamlessly recover all data as long as it's
> told that the filesystem is ext4.

Is the data good? If so, this stronglyindicates that this is not
a nencryption problem.
 
> From what I understand, the first step in the resizing process - resize2fs
> -p /dev/mapper/neuro-root 9G - has gone wrong - even though all the tests
> done in the meantime worked.
> 
> I have a backup from 3 days ago, yet it's a question of honor for me to
> understand what has gone wrong and how to repair it. I will owe a beer to
> whoever helps me.

Ok, so at least you did not kill all data. The mystery. however seems 
to be in some other place than the encryption. You can try doing this
again and looking after each step what is on disk.

Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -  Plato

Re: [dm-crypt] Filesystem unreadabe after resizing LUKS

[Robert Nichols]

On 01/30/2014 03:01 PM, Arno Wagner wrote:
> On Thu, Jan 30, 2014 at 16:24:27 CET, Pawel Chojnacki wrote:
>> Hello,
>>
>> Two days ago I tried to shrink my LUKS-encrypted /dev/sdb5 partition from
>> 119 to 110 GB according to
>> http://ubuntuforums.org/showthread.php?t=726724. Each step worked
>> properly, but in the end I'm not able to mount the
>> partition. I wanted to ask if you have any idea what may have caused that
>> and how to fix it:
>>
>> # cryptsetup luksOpen /dev/sdb5 neuro
>>
>> accepts the proper password, doesn't recognize any other, and fails to
>> mount the partition inside:
>>
>> # mount /dev/mapper/neuro /mnt/
>> mount: unknown filesystem type 'LVM2_member'
>
> This means LUKS is fine, just the resize failed.
> I should pount out the clear "It should go without
> saying, resizing your crypt may result in data loss
> Be sure to BACK UP your data first." given in that posting.

The thing that bugs me about the procedure in that thread is that it
includes a totally unnecessary and complex ("I had to do this by trial
and error") step of "cryptsetup ... resize". Nowhere in the LUKS header
is there any field that holds the size of the container. Each time the
LUKS container is opened it takes on the size of whatever is holding it
(physical partition, LVM logical volume, dmsetup mapping, ... whatever).

The "resize" operation in cryptsetup is useful only in the fairly rare
circumstance that you actually want to _use_ an encrypted area _while_
it is smaller than whatever is holding it. That is similar to having
a filesystem that is smaller than its partition, except that the
filesystem _does_ have persistent knowledge of its size whereas a LUKS
container does _not_.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: [dm-crypt] Filesystem unreadabe after resizing LUKS

[Arno Wagner]

On Fri, Jan 31, 2014 at 02:36:33 CET, Robert Nichols wrote:
> On 01/30/2014 03:01 PM, Arno Wagner wrote:
> >On Thu, Jan 30, 2014 at 16:24:27 CET, Pawel Chojnacki wrote:
> >>Hello,
> >>
> >>Two days ago I tried to shrink my LUKS-encrypted /dev/sdb5 partition from
> >>119 to 110 GB according to
> >>http://ubuntuforums.org/showthread.php?t=726724. Each step worked
> >>properly, but in the end I'm not able to mount the
> >>partition. I wanted to ask if you have any idea what may have caused that
> >>and how to fix it:
> >>
> >># cryptsetup luksOpen /dev/sdb5 neuro
> >>
> >>accepts the proper password, doesn't recognize any other, and fails to
> >>mount the partition inside:
> >>
> >># mount /dev/mapper/neuro /mnt/
> >>mount: unknown filesystem type 'LVM2_member'
> >
> >This means LUKS is fine, just the resize failed.
> >I should pount out the clear "It should go without
> >saying, resizing your crypt may result in data loss
> >Be sure to BACK UP your data first." given in that posting.
> 
> The thing that bugs me about the procedure in that thread is that it
> includes a totally unnecessary and complex ("I had to do this by trial
> and error") step of "cryptsetup ... resize". Nowhere in the LUKS header
> is there any field that holds the size of the container. Each time the
> LUKS container is opened it takes on the size of whatever is holding it
> (physical partition, LVM logical volume, dmsetup mapping, ... whatever).
> 
> The "resize" operation in cryptsetup is useful only in the fairly rare
> circumstance that you actually want to _use_ an encrypted area _while_
> it is smaller than whatever is holding it. That is similar to having
> a filesystem that is smaller than its partition, except that the
> filesystem _does_ have persistent knowledge of its size whereas a LUKS
> container does _not_.

Indeed. I missed that. In fact, I was unaware  that cryptsetup
even had a resize operation (well, I must have been aware at 
some time, because I revised the man-page, but apparently 
forgot about it immediately), possibly because I know that there
is no size-field anywhere in the header and that it is completely
unnecessary to have one.

I guess there is some massive misunderstanding somewhere in that
set of instructions....

Arno

--- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -  Plato

Re: [dm-crypt] Filesystem unreadabe after resizing LUKS

[Pawel Chojnacki]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

> I guess there is some massive misunderstanding somewhere in that
> set of instructions....
>

Thank you all for your help. Sadly, I was forced to use the diskspace for
something else and had to remove the partitions, using the previous backup
instead.

I suggest writing to the tutorial's author or setting up your own, proper
tutorial on resizing LUKS partitions - it's really hard to come by, at
least in my experience.

Thank you again,

Pawel Chojnacki

[-- Attachment #2: Type: text/html, Size: 791 bytes --]

Re: [dm-crypt] Filesystem unreadabe after resizing LUKS

[Sven Eschenberg]

On Thu, January 30, 2014 22:01, Arno Wagner wrote:
> On Thu, Jan 30, 2014 at 16:24:27 CET, Pawel Chojnacki wrote:
>> Hello,
>>
>> Two days ago I tried to shrink my LUKS-encrypted /dev/sdb5 partition
>> from
>> 119 to 110 GB according to
>> http://ubuntuforums.org/showthread.php?t=726724. Each step worked
>> properly, but in the end I'm not able to mount the
>> partition. I wanted to ask if you have any idea what may have caused
>> that
>> and how to fix it:
>>
>> # cryptsetup luksOpen /dev/sdb5 neuro
>>
>> accepts the proper password, doesn't recognize any other, and fails to
>> mount the partition inside:
>>
>> # mount /dev/mapper/neuro /mnt/
>> mount: unknown filesystem type 'LVM2_member'
>
> This means LUKS is fine, just the resize failed.
> I should pount out the clear "It should go without
> saying, resizing your crypt may result in data loss
> Be sure to BACK UP your data first." given in that posting.
>
> I also have to say that this is one of the reasons I think that
> lvm has no place in a normal installation: It just complicates
> things.
>
> [...]

Actually mount complains that the crypto target holds an lvm which can not
be mounted (for obvious reasons). The question here is if there was a LVM
before taking any action and if not how the LVM signatures showed up.

I think the very first important information we need is: Setup before
anything was done, which steps were taken and were there any errors durign
transformation.

LVM makes it more complicated, esp if one is not familiar with the steps
to take.

>
>> Testdisk recognizes the filesystems as FAT32/FAT12/NTFS and can't do
>> much.
>> Photorec on the other hand can seamlessly recover all data as long as
>> it's
>> told that the filesystem is ext4.
>
> Is the data good? If so, this stronglyindicates that this is not
> a nencryption problem.

True. maybe the OP just forgot to activate the LVs. Hard to tell without
knowing anything about the system ;-).

>
>> From what I understand, the first step in the resizing process -
>> resize2fs
>> -p /dev/mapper/neuro-root 9G - has gone wrong - even though all the
>> tests
>> done in the meantime worked.
>>
>> I have a backup from 3 days ago, yet it's a question of honor for me to
>> understand what has gone wrong and how to repair it. I will owe a beer
>> to
>> whoever helps me.
>
> Ok, so at least you did not kill all data. The mystery. however seems
> to be in some other place than the encryption. You can try doing this
> again and looking after each step what is on disk.
>
> Arno
> --
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D
> 9718
> ----
> A good decision is based on knowledge and not on numbers. -  Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

@Pawel, you might want to do a blkid before opening the crypto container
and afterwards. Blkid is non-destructive and the output might help to
understand your layout (or whatever is left of it). Of course a blkid
output etc. of the before state would have been good.

Regards

-Sven