* Weird behaviour with lsblk and freshly created loop device
@ 2014-09-01 15:39 Francis Moreau
2014-09-01 15:40 ` Francis Moreau
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Francis Moreau @ 2014-09-01 15:39 UTC (permalink / raw)
To: util-linux
Hello,
Right after associating a loop device with a disk image file, I'm using
lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw
--paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2
However adding a sleep or 'udevadm settle' after setting the loop device
the parttypes are correctly showed:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle
&& lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d
/dev/loop0
/dev/loop0
/dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f
/dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709
Could anybody tell me if that's expected ?
Thanks.
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-01 15:39 Weird behaviour with lsblk and freshly created loop device Francis Moreau
@ 2014-09-01 15:40 ` Francis Moreau
2014-09-01 17:56 ` Karel Zak
2014-09-01 19:41 ` Dale R. Worley
2 siblings, 0 replies; 10+ messages in thread
From: Francis Moreau @ 2014-09-01 15:40 UTC (permalink / raw)
To: util-linux
BTW, I'm using lsblk from util-linux 2.25 on Archlinux.
On 09/01/2014 05:39 PM, Francis Moreau wrote:
> Hello,
>
> Right after associating a loop device with a disk image file, I'm using
> lsblk to retrieve some info about the partitions.
>
> If I'm doing this as root:
>
> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw
> --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
>
> then the output is missing the parttypes:
>
> /dev/loop0
> /dev/loop0p1
> /dev/loop0p2
>
> However adding a sleep or 'udevadm settle' after setting the loop device
> the parttypes are correctly showed:
>
> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle
> && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d
> /dev/loop0
> /dev/loop0
> /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f
> /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709
>
> Could anybody tell me if that's expected ?
>
> Thanks.
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-01 15:39 Weird behaviour with lsblk and freshly created loop device Francis Moreau
2014-09-01 15:40 ` Francis Moreau
@ 2014-09-01 17:56 ` Karel Zak
2014-09-02 7:03 ` Francis Moreau
2014-09-01 19:41 ` Dale R. Worley
2 siblings, 1 reply; 10+ messages in thread
From: Karel Zak @ 2014-09-01 17:56 UTC (permalink / raw)
To: Francis Moreau; +Cc: util-linux
On Mon, Sep 01, 2014 at 05:39:48PM +0200, Francis Moreau wrote:
> Hello,
>
> Right after associating a loop device with a disk image file, I'm using
> lsblk to retrieve some info about the partitions.
>
> If I'm doing this as root:
>
> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw
> --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
>
> then the output is missing the parttypes:
>
> /dev/loop0
> /dev/loop0p1
> /dev/loop0p2
>
> However adding a sleep or 'udevadm settle' after setting the loop device
> the parttypes are correctly showed:
>
> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle
> && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d
> /dev/loop0
> /dev/loop0
> /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f
> /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709
>
> Could anybody tell me if that's expected ?
Sure, lsblk reads the information from udev db.
You can compile --without-udev, the it will use libblkid only.
Maybe we can add something like "lsblk --no-udev" to bypass udev
without recompilation, but is it really necessary?
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-01 17:56 ` Karel Zak
@ 2014-09-02 7:03 ` Francis Moreau
2014-09-02 7:30 ` Karel Zak
0 siblings, 1 reply; 10+ messages in thread
From: Francis Moreau @ 2014-09-02 7:03 UTC (permalink / raw)
To: Karel Zak; +Cc: util-linux
On 09/01/2014 07:56 PM, Karel Zak wrote:
> On Mon, Sep 01, 2014 at 05:39:48PM +0200, Francis Moreau wrote:
>> Hello,
>>
>> Right after associating a loop device with a disk image file, I'm using
>> lsblk to retrieve some info about the partitions.
>>
>> If I'm doing this as root:
>>
>> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw
>> --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
>>
>> then the output is missing the parttypes:
>>
>> /dev/loop0
>> /dev/loop0p1
>> /dev/loop0p2
>>
>> However adding a sleep or 'udevadm settle' after setting the loop device
>> the parttypes are correctly showed:
>>
>> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle
>> && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d
>> /dev/loop0
>> /dev/loop0
>> /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f
>> /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709
>>
>> Could anybody tell me if that's expected ?
>
> Sure, lsblk reads the information from udev db.
>
> You can compile --without-udev, the it will use libblkid only.
>
> Maybe we can add something like "lsblk --no-udev" to bypass udev
> without recompilation, but is it really necessary?
It seems that lsblk uses udev to get some block device metadata and asks
some others to the kernel. If so, it makes the whole process racy
because udev might not have handled the events sent by the kernel yet.
I'm not sure why udev is used by default in the first place, what are
the benefits ? Using libblkid, at least by default seems the right thing
to do.
Otherwise maybe lsblk should do the equivalent of 'udevadm settle' to
handle correctly freshly created devices ?
Thanks.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-02 7:03 ` Francis Moreau
@ 2014-09-02 7:30 ` Karel Zak
2014-09-02 16:45 ` Francis Moreau
0 siblings, 1 reply; 10+ messages in thread
From: Karel Zak @ 2014-09-02 7:30 UTC (permalink / raw)
To: Francis Moreau; +Cc: util-linux
On Tue, Sep 02, 2014 at 09:03:45AM +0200, Francis Moreau wrote:
> It seems that lsblk uses udev to get some block device metadata and asks
> some others to the kernel. If so, it makes the whole process racy
> because udev might not have handled the events sent by the kernel yet.
>
> I'm not sure why udev is used by default in the first place, what are
* info from udev is accessible for non-root users
* it's better to scan devices only once on one place only
* udev is able to gather information from more sources (for example
libblkid does not provide WWN)
> the benefits ? Using libblkid, at least by default seems the right thing
> to do.
>
> Otherwise maybe lsblk should do the equivalent of 'udevadm settle' to
> handle correctly freshly created devices ?
That's question, now (because it's not hardcoded to lsblk) everyone is
able to control this behavior, all you need is to add 'udevadm settle'
to your use-case.
The problem I see is that there is no any hint about this behavior in
lsblk man page.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-02 7:30 ` Karel Zak
@ 2014-09-02 16:45 ` Francis Moreau
2014-09-05 7:45 ` Karel Zak
0 siblings, 1 reply; 10+ messages in thread
From: Francis Moreau @ 2014-09-02 16:45 UTC (permalink / raw)
To: Karel Zak; +Cc: util-linux
On 09/02/2014 09:30 AM, Karel Zak wrote:
> On Tue, Sep 02, 2014 at 09:03:45AM +0200, Francis Moreau wrote:
>> It seems that lsblk uses udev to get some block device metadata and asks
>> some others to the kernel. If so, it makes the whole process racy
>> because udev might not have handled the events sent by the kernel yet.
>>
>> I'm not sure why udev is used by default in the first place, what are
>
> * info from udev is accessible for non-root users
> * it's better to scan devices only once on one place only
> * udev is able to gather information from more sources (for example
> libblkid does not provide WWN)
>
>> the benefits ? Using libblkid, at least by default seems the right thing
>> to do.
>>
>> Otherwise maybe lsblk should do the equivalent of 'udevadm settle' to
>> handle correctly freshly created devices ?
>
> That's question, now (because it's not hardcoded to lsblk) everyone is
> able to control this behavior, all you need is to add 'udevadm settle'
> to your use-case.
The question is more why let the user do that ?
If lsblk reports the partitions, shouldn't it do that only when it's
sure to fully have retrieve the partition's metadata ?
BTW, is the user supposed to know that lsblk relies on udev or is this
implementation detail ?
Thanks
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-02 16:45 ` Francis Moreau
@ 2014-09-05 7:45 ` Karel Zak
2014-09-05 8:13 ` Francis Moreau
0 siblings, 1 reply; 10+ messages in thread
From: Karel Zak @ 2014-09-05 7:45 UTC (permalink / raw)
To: Francis Moreau; +Cc: util-linux
On Tue, Sep 02, 2014 at 06:45:58PM +0200, Francis Moreau wrote:
> > That's question, now (because it's not hardcoded to lsblk) everyone is
> > able to control this behavior, all you need is to add 'udevadm settle'
> > to your use-case.
>
> The question is more why let the user do that ?
because we're talking about small, simple command line util, not
about complex high-level UI
> If lsblk reports the partitions, shouldn't it do that only when it's
> sure to fully have retrieve the partition's metadata ?
if you do not have exclusive access to the device then you cannot be
sure at all. All is asynchronous...
> BTW, is the user supposed to know that lsblk relies on udev or is this
> implementation detail ?
This is very generic system feature, for example if you want to mount
a device by /dev/disk/by-* symlinks then nowhere is guarantee that
mount(8) is not faster than udev+blkid, etc.
Yes, we can add "udevadm settle" functionality into all system utils,
but it will increase complexity and degrade performance. So from my
point of view it seems better to explain the problem in man page,
keep lsblk simple and stupid and assume that users who really care
will use "udevadm settle".
(The another disadvantage is that libudev is not able to wait for
specific device, it waits for all active event.)
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-05 7:45 ` Karel Zak
@ 2014-09-05 8:13 ` Francis Moreau
0 siblings, 0 replies; 10+ messages in thread
From: Francis Moreau @ 2014-09-05 8:13 UTC (permalink / raw)
To: Karel Zak; +Cc: util-linux
On 09/05/2014 09:45 AM, Karel Zak wrote:
> On Tue, Sep 02, 2014 at 06:45:58PM +0200, Francis Moreau wrote:
>>> That's question, now (because it's not hardcoded to lsblk) everyone is
>>> able to control this behavior, all you need is to add 'udevadm settle'
>>> to your use-case.
>>
>> The question is more why let the user do that ?
>
> because we're talking about small, simple command line util, not
> about complex high-level UI
>
>> If lsblk reports the partitions, shouldn't it do that only when it's
>> sure to fully have retrieve the partition's metadata ?
>
> if you do not have exclusive access to the device then you cannot be
> sure at all. All is asynchronous...
>
>> BTW, is the user supposed to know that lsblk relies on udev or is this
>> implementation detail ?
>
> This is very generic system feature, for example if you want to mount
> a device by /dev/disk/by-* symlinks then nowhere is guarantee that
> mount(8) is not faster than udev+blkid, etc.
>
> Yes, we can add "udevadm settle" functionality into all system utils,
> but it will increase complexity and degrade performance. So from my
> point of view it seems better to explain the problem in man page,
> keep lsblk simple and stupid and assume that users who really care
> will use "udevadm settle".
Also you might want to add a new option for forcing lsblk to wait for udev ?
Thanks for your answers.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-01 15:39 Weird behaviour with lsblk and freshly created loop device Francis Moreau
2014-09-01 15:40 ` Francis Moreau
2014-09-01 17:56 ` Karel Zak
@ 2014-09-01 19:41 ` Dale R. Worley
2014-09-02 6:54 ` Francis Moreau
2 siblings, 1 reply; 10+ messages in thread
From: Dale R. Worley @ 2014-09-01 19:41 UTC (permalink / raw)
To: Francis Moreau; +Cc: util-linux
> From: Francis Moreau <francis.moro@gmail.com>
> Right after associating a loop device with a disk image file, I'm using
> lsblk to retrieve some info about the partitions.
>
> If I'm doing this as root:
>
> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw
> --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
>
> then the output is missing the parttypes:
>
> /dev/loop0
> /dev/loop0p1
> /dev/loop0p2
>
> However adding a sleep or 'udevadm settle' after setting the loop device
> the parttypes are correctly showed:
It looks like the process of determining the partitions is
asynchronous, and is not assured of being completed when losetup
returns.
What happens when you add -P to losetup? The manual page (for Fedora
19) says "force kernel to scan partition table on newly created loop
device", which suggests that the partition scan will be completed by
the time losetup returns.
Dale
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Weird behaviour with lsblk and freshly created loop device
2014-09-01 19:41 ` Dale R. Worley
@ 2014-09-02 6:54 ` Francis Moreau
0 siblings, 0 replies; 10+ messages in thread
From: Francis Moreau @ 2014-09-02 6:54 UTC (permalink / raw)
To: Dale R. Worley; +Cc: util-linux
On 09/01/2014 09:41 PM, Dale R. Worley wrote:
>> From: Francis Moreau <francis.moro@gmail.com>
>
>> Right after associating a loop device with a disk image file, I'm using
>> lsblk to retrieve some info about the partitions.
>>
>> If I'm doing this as root:
>>
>> $ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw
>> --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
>>
>> then the output is missing the parttypes:
>>
>> /dev/loop0
>> /dev/loop0p1
>> /dev/loop0p2
>>
>> However adding a sleep or 'udevadm settle' after setting the loop device
>> the parttypes are correctly showed:
>
> It looks like the process of determining the partitions is
> asynchronous, and is not assured of being completed when losetup
> returns.
Still lsblk lists the partitions.
I'm not sure how lsblk works. It seems that it relies on udev to get
certain information, parttype is an example but doesn't for listing
partitions of block devices.
That seems to make the whole process racy.
If this is correct, I'm wondering why lsblk relies on udev to retrieve
block device metadata. Can't it find them out directly from the kernel ?
>
> What happens when you add -P to losetup? The manual page (for Fedora
> 19) says "force kernel to scan partition table on newly created loop
> device", which suggests that the partition scan will be completed by
> the time losetup returns.
Adding -P doesn't help. Actually the kernel correctly detected the
partitions embedded in the loop device. But udev hasn't handled the
associeted events yet when lsblk is called.
Thanks
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-09-05 8:13 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-01 15:39 Weird behaviour with lsblk and freshly created loop device Francis Moreau
2014-09-01 15:40 ` Francis Moreau
2014-09-01 17:56 ` Karel Zak
2014-09-02 7:03 ` Francis Moreau
2014-09-02 7:30 ` Karel Zak
2014-09-02 16:45 ` Francis Moreau
2014-09-05 7:45 ` Karel Zak
2014-09-05 8:13 ` Francis Moreau
2014-09-01 19:41 ` Dale R. Worley
2014-09-02 6:54 ` Francis Moreau
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).