Stopping a bcache device during shutdown

Stopping a bcache device during shutdown

[Rolf Fokkens]

Hi,

My system has the following block device stack: 
RootFS->LVM->bcache(bcache0)->RAID5(md1)->(sda2, sdb2, sdc2)

During shutdown on Fedora I noticed the following on console:

[  137.036846] dracut Warning: Unmounted /oldroot.
[  137.103139] dracut: Disassembling device-mapper devices
[  137.147160] dracut: Waiting for mdraid devices to be clean.
[  137.158556] dracut: Disassembling mdraid devices.
mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
process, mounted filesystem or active volume group?
[  138.247044] bcache: bcache_reboot() Stopping all devices:
[  140.247073] bcache: bcache_reboot() Timeout waiting for devices to be 
closed
[  140.250295] Unregister pv shared memory for cpu 0
[  140.253139] kvm: exiting hardware virtualization

So apparently bcache is (b)locking /dev/md1 during shutdown. In an 
attempt to solve this, I added a "echo 1 > /sys/block/bcache0/bcache/stop":

[  137.036846] dracut Warning: Unmounted /oldroot.
[  137.076580] dracut: Stopping bcache devices
STOP /sys/block/bcache0/bcache/stop
[  137.103139] dracut: Disassembling device-mapper devices
[  137.147160] dracut: Waiting for mdraid devices to be clean.
[  137.158556] dracut: Disassembling mdraid devices.
mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
process, mounted filesystem or active volume group?
mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
process, mounted filesystem or active volume group?
mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
process, mounted filesystem or active volume group?
[  137.200086] dracut: Stopping bcache cache sets
STOP /sys/fs/bcache/408c0554-9752-4126-916d-5c5fe32037d2/stop
Rebooting.
[  138.247044] bcache: bcache_reboot() Stopping all devices:
[  140.247073] bcache: bcache_reboot() Timeout waiting for devices to be 
closed
[  140.250295] Unregister pv shared memory for cpu 0

The 1st STOP... line is the result of a debugging "echo .." command:

for stop in /sys/block/bcache[0-9]*/bcache/stop; do
     [ -f "$stop" ] || continue
     echo 1 > "$stop"
     echo STOP $stop
done


The second STOP does the same for the cache set, but that's not the 
issue now.

 From both the documentation and testing I understand that the first 
STOP should mark the bcache0 device for stop as soon as it's no longer 
used, in this case by LVM. So when LVM (device mapper devices) is 
disassembled, bcache should actually stop the bcache0 device. But his 
apparently doesn't happen. I did a test with almost the same config, but 
without bcache between LVM and RAID5, in that case all works fine. So it 
looks like bcache is the cause.

So I may be misunderstanding the way "echo 1 > 
/sys/block/bcache0/bcache/stop" works. Are there any suggestions for my 
where to look further?

Thanks,

Rolf

Re: Stopping a bcache device during shutdown

[Zachary Palmer]

Just to add a little bit to this discussion, I think I'm experiencing a 
similar issue.  I'm using bcache to improve my laptop's I/O 
performance.  My configuration is, using the syntax you used,

     RootFS->LVM->dm-crypt->bcache(bcache0)->(sda7)

I get pretty much the same sort of shutdown messages using the stock 
Linux 3.11.5 from kernel.org on my Debian Wheezy installation.

In my case, I'm getting this message with bcache immediately over a raw 
disk (with a partition on an SSD acting as the cache).  I don't have the 
spare hardware to test things out, but I'm guessing that this happens 
whenever LVM is anywhere on the stack above bcache.

It seems probable to me that this bug is preventing my laptop from 
hibernating.  :(  I can still put it to sleep.  The hibernation process 
will start correctly, but things don't come back correctly when I try to 
restore: I just get fscking on my root partition, which leads me to 
believe that things weren't closed up properly. (To be fair, this might 
also just be a kernel regression, since my Inspiron 17R SE laptop was 
running 3.10 from Debian beforehand and is running 3.11.5 from 
kernel.org now.)

Just wanted to throw in the extra data.  :)

Cheers,

Zach
> Hi,
>
> My system has the following block device stack: 
> RootFS->LVM->bcache(bcache0)->RAID5(md1)->(sda2, sdb2, sdc2)
>
> During shutdown on Fedora I noticed the following on console:
>
> [  137.036846] dracut Warning: Unmounted /oldroot.
> [  137.103139] dracut: Disassembling device-mapper devices
> [  137.147160] dracut: Waiting for mdraid devices to be clean.
> [  137.158556] dracut: Disassembling mdraid devices.
> mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
> process, mounted filesystem or active volume group?
> [  138.247044] bcache: bcache_reboot() Stopping all devices:
> [  140.247073] bcache: bcache_reboot() Timeout waiting for devices to 
> be closed
> [  140.250295] Unregister pv shared memory for cpu 0
> [  140.253139] kvm: exiting hardware virtualization
>
> So apparently bcache is (b)locking /dev/md1 during shutdown. In an 
> attempt to solve this, I added a "echo 1 > 
> /sys/block/bcache0/bcache/stop":
>
> [  137.036846] dracut Warning: Unmounted /oldroot.
> [  137.076580] dracut: Stopping bcache devices
> STOP /sys/block/bcache0/bcache/stop
> [  137.103139] dracut: Disassembling device-mapper devices
> [  137.147160] dracut: Waiting for mdraid devices to be clean.
> [  137.158556] dracut: Disassembling mdraid devices.
> mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
> process, mounted filesystem or active volume group?
> mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
> process, mounted filesystem or active volume group?
> mdadm: Cannot get exclusive access to /dev/md1:Perhaps a running 
> process, mounted filesystem or active volume group?
> [  137.200086] dracut: Stopping bcache cache sets
> STOP /sys/fs/bcache/408c0554-9752-4126-916d-5c5fe32037d2/stop
> Rebooting.
> [  138.247044] bcache: bcache_reboot() Stopping all devices:
> [  140.247073] bcache: bcache_reboot() Timeout waiting for devices to 
> be closed
> [  140.250295] Unregister pv shared memory for cpu 0
>
> The 1st STOP... line is the result of a debugging "echo .." command:
>
> for stop in /sys/block/bcache[0-9]*/bcache/stop; do
>     [ -f "$stop" ] || continue
>     echo 1 > "$stop"
>     echo STOP $stop
> done
>
>
> The second STOP does the same for the cache set, but that's not the 
> issue now.
>
> From both the documentation and testing I understand that the first 
> STOP should mark the bcache0 device for stop as soon as it's no longer 
> used, in this case by LVM. So when LVM (device mapper devices) is 
> disassembled, bcache should actually stop the bcache0 device. But his 
> apparently doesn't happen. I did a test with almost the same config, 
> but without bcache between LVM and RAID5, in that case all works fine. 
> So it looks like bcache is the cause.
>
> So I may be misunderstanding the way "echo 1 > 
> /sys/block/bcache0/bcache/stop" works. Are there any suggestions for 
> my where to look further?
>
> Thanks,
>
> Rolf
> -- 
> To unsubscribe from this list: send the line "unsubscribe 
> linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

bcache and laptop hibernation working under Linux 3.11.5

[Zachary Palmer]

All,

I previously had issues with hibernation when using bcache.  It seems 
that these issues were coincidental.  I am using Debian Wheezy and 
hibernation worked correctly with the 3.10 stock kernel; when I moved to 
3.11 kernels I had built myself, hibernation stopped working.  It would 
seem that, for reasons I do not currently know, it was necessary for me 
to create a file /etc/initramfs-tools/conf.d/resume containing a line 
"RESUME=UUID=..." (with the UUID of my swap device) and then run 
update-initramfs.  This is Debian Wheezy; depending on distribution, 
YMMV.  Normally, the .../conf.d/resume file is supposed to be 
autogenerated or the swap is supposed to be detected at startup. I'm not 
sure why this stopped working for me.  In any case, bcache doesn't seem 
to be causing me any hibernation problems on Linux 3.11.5 and I'm 
thrilled.  :)

Cheers,

Zach