Re: Update FC22 - doesn't find root fs and doesn't boot

[Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

Am 27.05.2015 um 15:21 schrieb Gerhard Wiesinger:
> On 27.05.2015 15:07, Harald Hoyer wrote:
>> Am 27.05.2015 um 15:03 schrieb Gerhard Wiesinger:
>>> On 27.05.2015 14:43, Harald Hoyer wrote:
>>>> Am 27.05.2015 um 14:39 schrieb Gerhard Wiesinger:
>>>>> On 27.05.2015 14:25, Harald Hoyer wrote:
>>>>>> Am 27.05.2015 um 14:13 schrieb Andrei Borzenkov:
>>>>>>> On Wed, May 27, 2015 at 3:09 PM, Gerhard Wiesinger <lists-tcxHB8FAk+J54TAoqtyWWQ@public.gmane.org>
>>>>>>> wrote:
>>>>>>>> On 27.05.2015 13:56, Harald Hoyer wrote:
>>>>>>>>> Am 27.05.2015 um 13:28 schrieb Gerhard Wiesinger:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> After upgrading one machine it does't boot anymore. Booting the old
>>>>>>>>>> kernel
>>>>>>>>>> works well.
>>>>>>>>>>
>>>>>>>>>> It is a RAID, LUKS, LVM setup. Find relevant details below.
>>>>>>>>>>
>>>>>>>>>> So it looks RAID assembling doesn't work any more.
>>>>>>>>>>
>>>>>>>>>> Any ideas?
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>>
>>>>>>>>>> Ciao,
>>>>>>>>>> Gerhard
>>>>>>>>>>
>>>>>>>>>> blkid
>>>>>>>>>> /dev/sda1: UUID="56e9bde4-5cd1-630d-188c-22a54c1c8c37"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="5ec6792c-01"
>>>>>>>>>> /dev/sdb1: UUID="56e9bde4-5cd1-630d-188c-22a54c1c8c37"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="1c7cffaf-01"
>>>>>>>>>> /dev/sdc1: UUID="56e9bde4-5cd1-630d-188c-22a54c1c8c37"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="0079dfc3-01"
>>>>>>>>>> /dev/sdd1: UUID="eb295015-e20b-e133-15db-5c9130487300"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="fadeaffe-01"
>>>>>>>>>> /dev/sde1: UUID="eb295015-e20b-e133-15db-5c9130487300"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="539b5814-01"
>>>>>>>>>> /dev/sdf1: UUID="eb295015-e20b-e133-15db-5c9130487300"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="badeaffe-01"
>>>>>>>>>> /dev/sdh1: UUID="51e8e983-8786-b868-dbc3-87357e1a9534"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="15631562-01"
>>>>>>>>>> /dev/sdg1: UUID="51e8e983-8786-b868-dbc3-87357e1a9534"
>>>>>>>>>> TYPE="linux_raid_member"
>>>>>>>>>> PARTUUID="162f0d0b-01"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> blkid -o udev
>>>>>>>>>> ID_FS_UUID=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>>> ID_FS_UUID_ENC=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=5ec6792c-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>>> ID_FS_UUID_ENC=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=1c7cffaf-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>>> ID_FS_UUID_ENC=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=0079dfc3-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>>> ID_FS_UUID_ENC=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=fadeaffe-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>>> ID_FS_UUID_ENC=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=539b5814-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>>> ID_FS_UUID_ENC=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=badeaffe-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=51e8e983-8786-b868-dbc3-87357e1a9534
>>>>>>>>>> ID_FS_UUID_ENC=51e8e983-8786-b868-dbc3-87357e1a9534
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=15631562-01
>>>>>>>>>>
>>>>>>>>>> ID_FS_UUID=51e8e983-8786-b868-dbc3-87357e1a9534
>>>>>>>>>> ID_FS_UUID_ENC=51e8e983-8786-b868-dbc3-87357e1a9534
>>>>>>>>>> ID_FS_TYPE=linux_raid_member
>>>>>>>>>> ID_FS_PARTUUID=162f0d0b-01
>>>>>>>>>>
>>>>>>>>>> [    2.479836] myhost systemd[1]: Device
>>>>>>>>>> dev-disk-by\x2did-usb\x2dUSBDisk_RunDisk_ABCDEFGH\x2d0:0.device appeared
>>>>>>>>>> twice
>>>>>>>>>> with different sysfs paths /sys
>>>>>>>>>>
>>>>>>>>>> /devices/pci0000:00/0000:00:04.1/usb2/2-5/2-5:1.0/host9/target9:0:0/9:0:0:0/block/sdh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and /sys/devices/pci0000:00/0000:00:02.1/usb1/1-5/1-5:1.0/host8/target
>>>>>>>>>> 8:0:0/8:0:0:0/block/sdg
>>>>>>>>>> [    2.500826] myhost systemd[1]: Device
>>>>>>>>>> dev-disk-by\x2did-usb\x2dUSBDisk_RunDisk_ABCDEFGH\x2d0:0\x2dpart1.device
>>>>>>>>>> appeared twice with different sysfs p
>>>>>>>>>> aths
>>>>>>>>>>
>>>>>>>>>> /sys/devices/pci0000:00/0000:00:04.1/usb2/2-5/2-5:1.0/host9/target9:0:0/9:0:0:0/block/sdh/sdh1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and /sys/devices/pci0000:00/0000:00:02.1/usb1/1-5/1-5:1.
>>>>>>>>>> 0/host8/target8:0:0/8:0:0:0/block/sdg/sdg1
>>>>>>>>>> [    2.632754] myhost kernel: Switched to clocksource tsc
>>>>>>>>>> [   91.162986] myhost systemd[1]: Job dev-md0.device/start timed out.
>>>>>>>>>> [   91.163138] myhost systemd[1]: Timed out waiting for device
>>>>>>>>>> dev-md0.device.
>>>>>>>>>> [   91.163251] myhost systemd[1]: Dependency failed for Cryptography
>>>>>>>>>> Setup for
>>>>>>>>>> luks-md0.
>>>>>>>>>> [   91.163360] myhost systemd[1]: Dependency failed for
>>>>>>>>>> dev-mapper-luks\x2dmd0.device.
>>>>>>>>>> [   91.163474] myhost systemd[1]: Job
>>>>>>>>>> dev-mapper-luks\x2dmd0.device/start
>>>>>>>>>> failed with result 'dependency'.
>>>>>>>>>> [   91.163586] myhost systemd[1]: Dependency failed for Encrypted
>>>>>>>>>> Volumes.
>>>>>>>>>> [   91.163694] myhost systemd[1]: Job cryptsetup.target/start failed
>>>>>>>>>> with
>>>>>>>>>> result 'dependency'.
>>>>>>>>>> [   91.163801] myhost systemd[1]: Job
>>>>>>>>>> systemd-cryptsetup@luks\x2dmd0.service/start failed with result
>>>>>>>>>> 'dependency'.
>>>>>>>>>> [   91.163910] myhost systemd[1]: Job dev-md0.device/start failed with
>>>>>>>>>> result
>>>>>>>>>> 'timeout'.
>>>>>>>>>> [   91.164021] myhost systemd[1]: Reached target System Initialization.
>>>>>>>>>> [   91.164129] myhost systemd[1]: Starting System Initialization.
>>>>>>>>>> [   91.164236] myhost systemd[1]: Reached target Basic System.
>>>>>>>>>> [   91.164346] myhost systemd[1]: Starting Basic System.
>>>>>>>>>> [  185.740238] myhost dracut-initqueue[340]: Warning: Could not boot.
>>>>>>>>>> [  185.741266] myhost systemd[1]: Received SIGRTMIN+20 from PID 341
>>>>>>>>>> (plymouthd).
>>>>>>>>>> [  185.841257] myhost dracut-initqueue[340]: Warning:
>>>>>>>>>> /dev/VolGroup00/root does
>>>>>>>>>> not exist
>>>>>>>>>> [  185.841403] myhost systemd[1]: Starting Dracut Emergency Shell...
>>>>>>>>>> [  185.859366] myhost systemd[1]: Received SIGRTMIN+21 from PID 341
>>>>>>>>>> (plymouthd).
>>>>>>>>>>
>>>>>>>>>> dracut:/# cat "/run/initramfs/rdsosreport.txt"
>>>>>>>>>> + cat /lib/dracut/dracut-041-10.fc22.1
>>>>>>>>>> dracut-041-10.fc22.1
>>>>>>>>>> + cat /proc/cmdline
>>>>>>>>>> BOOT_IMAGE=/vmlinuz-4.0.4-301.fc22.x86_64 root=/dev/VolGroup00/root ro
>>>>>>>>>> quiet
>>>>>>>>>> console=ttyS0,115200n8 rd_NO_PLYMOUTH SYSFONT=latarcyrheb-sun16
>>>>>>>>>> LANG=en_US.UTF-
>>>>>>>>>> 8 KEYTABLE=de-latin1
>>>>>>>>> your kernel cmdline seems to be missing the raid configuration
>>>>>>>>>
>>>>>>>>> rd.md.uuid=56e9bde4-5cd1-630d-188c-22a54c1c8c37
>>>>>>>>> rd.md.uuid=eb295015-e20b-e133-15db-5c9130487300
>>>>>>>>> rd.md.uuid=51e8e983-8786-b868-dbc3-87357e1a9534
>>>>>>>>>
>>>>>>>>> ... whatever is needed to boot your system.
>>>>>>>>>
>>>>>>>>> # dracut --print-cmdline
>>>>>>>>>
>>>>>>>>> should tell you what kernel cmdline parameters are needed
>>>>>>>> Thank you for the fast reply.
>>>>>>>>
>>>>>>>> # comment: 3 separate lines!!!!! (maybe a problem)
>>>>>>>> dracut --print-cmdline
>>>>>>>>     rd.luks.uuid=luks-0c66fc77-a8a0-49e6-b96c-55e886a91f09
>>>>>>>> rd.lvm.lv=VolGroup00/root
>>>>>>>>     rd.lvm.lv=VolGroup00/swap
>>>>>>>>     rd.md.uuid=51e8e983:8786b868:dbc38735:7e1a9534
>>>>>>>> rd.md.uuid=56e9bde4:5cd1630d:188c22a5:4c1c8c37
>>>>>>>> resume=/dev/mapper/VolGroup00-swap root=/dev/mapper/VolGroup00-root
>>>>>>>> rootflags=rw,relatime,data=ordered rootfstype=ext4
>>>>>>>>
>>>>>>>> dracut --force 4.0.4-301.fc22.x86_64
>>>>>>>>
>>>>>>>> Still no correct entry in /etc/grub2.cfg
>>>>>>> I would be very surprised if dracut made any changes to grub.cfg. As I
>>>>>>> understand it, you either build host-specific dracut that will include
>>>>>>> this emulated command line or you have to add lines manually to your
>>>>>>> boot loader configuration.
>>>>>>>
>>>>>>>> linux16 /vmlinuz-4.0.4-301.fc22.x86_64 root=/dev/VolGroup00/root ro quiet
>>>>>>>> console=ttyS0,115200n8 rd_NO_PLYMOUTH SYSFONT=latarcyrheb-sun16
>>>>>>>> LANG=en_US.UTF-8 KEYTABLE=de-latin1
>>>>>>>>
>>>>>>>> Looks like a bug to me.
>>>>>> You have to add the options manually to the grub conf and all in the same
>>>>>> line.
>>>>>>
>>>>>> dracut does not modify your grub configuration.
>>>>> OK, I thought it was modifying it, at least in the past.
>>>>>
>>>>> Added it manually and it worked well.
>>>>>
>>>>> What's still strange:
>>>>>
>>>>> Last lines after startup, luks-md0 has already been started as the root fs is
>>>>> there ....
>>>>> Starting Cryptography Setup for luks-md0...
>>>>>            Starting Authorization Manager...
>>>>>
>>>>> Fedora release 22 (Twenty Two)
>>>>> Kernel 4.0.4-301.fc22.x86_64 on an x86_64 (ttyS0)
>>>>>
>>>>> # Rebooting, why is a passphrase necessary here?
>>>>> reboot
>>>>> Please enter passphrase for disk luks-md0!
>>>>>
>>>>> Possible bug here?
>>>>>
>>>>> Ciao,
>>>>> Gerhard
>>>>>
>>>> Did you add "  rd.luks.uuid=luks-0c66fc77-a8a0-49e6-b96c-55e886a91f09 " also?
>>> Yes, of course. :-)
>>>
>>>> What is your /etc/crypttab ?
>>> cat /etc/crypttab
>>> luks-md0                /dev/md0        none
>>> luks-md1                /dev/md1        /etc/crypto-key-md1
>>>
>>> BTW: Why did it work in previous versions without the long command line and
>>> predefined "hint" parameters?
>>>
>>> Ciao,
>>> Gerhard
>>>
>> Oh... please, please do not use kernel names like "md0" ... better use UUID=
> 
> 
> changed it, but didn't help. This machine was updated since around FC5. :-)
> 
> Booting:
> [FAILED] Failed to start Cryptography Setup for luks-md0.
> See "systemctl status 'systemd-cryptsetup@luks\x2dmd0.service'" for details.
> [DEPEND] Dependency failed for Encrypted Volumes.
> ...
> [FAILED] Failed to start Cryptography Setup for luks-md0.
> See "systemctl status 'systemd-cryptsetup@luks\x2dmd0.service'" for details.
> [DEPEND] Dependency failed for Encrypted Volumes.
> ...
>          Starting Cryptography Setup for luks-md0...
>          Starting Authorization Manager...
> 
> Fedora release 22 (Twenty Two)
> 
> cat /etc/crypttab
> luks-md0 UUID="0c66fc77-a8a0-49e6-b96c-55e886a91f09"        none
> luks-md1 UUID="aaa6ec1e-e54f-4864-a793-cc95d5fa59d2" /etc/crypto-key-md1
> 
> Ciao,
> Gerhard
> 

Does "dracut -f" help to get the update crypttab in the initramfs?