/sys/block and /dev and partitions

/sys/block and /dev and partitions

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]

Hi,

Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. 

Any idea ?

Cheers
-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: /sys/block and /dev and partitions

[Sage Weil]

On Sat, 15 Aug 2015, Loic Dachary wrote:
> Hi,
> 
> Is there a portable and consistent way to figure out if a given /dev/XXX 
> path (for instance /dev/dm-1) is a partition of a whole device ? 
> Although checking /sys/block/dm-1/dm/name for a number at the end (like 
> mpatha1 or mpatha2) would probably work, it feels like a fragile hack. 
> Looking into /sys/block/dm-1/slaves will lead to 
> /sys/block/dm-1/slaves/dm-0 and we can check that 
> /sys/block/dm-*/subsystem is class/block. But that does not necessarily 
> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.

Take a look at is_partition in ceph-disk, whih is the best I came up with.  
Basically it checks if the device name appears as /sys/block/*/$foo...

sage

Re: /sys/block and /dev and partitions

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1289 bytes --]

Hi Sage,

On 15/08/2015 16:28, Sage Weil wrote:
> On Sat, 15 Aug 2015, Loic Dachary wrote:
>> Hi,
>>
>> Is there a portable and consistent way to figure out if a given /dev/XXX 
>> path (for instance /dev/dm-1) is a partition of a whole device ? 
>> Although checking /sys/block/dm-1/dm/name for a number at the end (like 
>> mpatha1 or mpatha2) would probably work, it feels like a fragile hack. 
>> Looking into /sys/block/dm-1/slaves will lead to 
>> /sys/block/dm-1/slaves/dm-0 and we can check that 
>> /sys/block/dm-*/subsystem is class/block. But that does not necessarily 
>> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.
> 
> Take a look at is_partition in ceph-disk, whih is the best I came up with.  
> Basically it checks if the device name appears as /sys/block/*/$foo...

That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found. 

Thanks !

> 
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: /sys/block and /dev and partitions

[Ilya Dryomov]

On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary <loic@dachary.org> wrote:
> Hi Sage,
>
> On 15/08/2015 16:28, Sage Weil wrote:
>> On Sat, 15 Aug 2015, Loic Dachary wrote:
>>> Hi,
>>>
>>> Is there a portable and consistent way to figure out if a given /dev/XXX
>>> path (for instance /dev/dm-1) is a partition of a whole device ?
>>> Although checking /sys/block/dm-1/dm/name for a number at the end (like
>>> mpatha1 or mpatha2) would probably work, it feels like a fragile hack.
>>> Looking into /sys/block/dm-1/slaves will lead to
>>> /sys/block/dm-1/slaves/dm-0 and we can check that
>>> /sys/block/dm-*/subsystem is class/block. But that does not necessarily
>>> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.
>>
>> Take a look at is_partition in ceph-disk, whih is the best I came up with.
>> Basically it checks if the device name appears as /sys/block/*/$foo...

For regular devices, you can access() /sys/dev/block/maj:min/partition.
If it's there, it's a partition - no need to iterate over /sys/block.

>
> That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found.

A better way might be to rely on the fact that a dm partition will
necessarily have its uuid prefixed by "part".  In that case, it should
be safe to assume that the thing in slaves is a whole disk - I think
that's what various util-linux tools do.  However, IIRC the dm uuid is
optional, so that won't work on a dm device without a uuid.

Thanks,

                Ilya

Re: /sys/block and /dev and partitions

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2554 bytes --]

Hi Ilya,

On 15/08/2015 19:42, Ilya Dryomov wrote:
> On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary <loic@dachary.org> wrote:
>> Hi Sage,
>>
>> On 15/08/2015 16:28, Sage Weil wrote:
>>> On Sat, 15 Aug 2015, Loic Dachary wrote:
>>>> Hi,
>>>>
>>>> Is there a portable and consistent way to figure out if a given /dev/XXX
>>>> path (for instance /dev/dm-1) is a partition of a whole device ?
>>>> Although checking /sys/block/dm-1/dm/name for a number at the end (like
>>>> mpatha1 or mpatha2) would probably work, it feels like a fragile hack.
>>>> Looking into /sys/block/dm-1/slaves will lead to
>>>> /sys/block/dm-1/slaves/dm-0 and we can check that
>>>> /sys/block/dm-*/subsystem is class/block. But that does not necessarily
>>>> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.
>>>
>>> Take a look at is_partition in ceph-disk, whih is the best I came up with.
>>> Basically it checks if the device name appears as /sys/block/*/$foo...
> 
> For regular devices, you can access() /sys/dev/block/maj:min/partition.
> If it's there, it's a partition - no need to iterate over /sys/block.

I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code.

> 
>>
>> That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found.
> 
> A better way might be to rely on the fact that a dm partition will
> necessarily have its uuid prefixed by "part".  In that case, it should
> be safe to assume that the thing in slaves is a whole disk - I think
> that's what various util-linux tools do.  However, IIRC the dm uuid is
> optional, so that won't work on a dm device without a uuid.

It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way.

Is it also safe to assume that if the uuid is:

$ cat /sys/dev/block/253:?/dm/uuid
mpath-353333330000007d0
part1-mpath-353333330000007d0
part2-mpath-353333330000007d0

it means these were created by multipath because of the mpath ? When asking dmsetup with:

$ sudo dmsetup info --columns --noheading --options subsystem
mpath
part2
part1

it looks like it assumes the first part of the uuid is always the subsystem. Which makes me wonder how reliable this is. What I'm really looking for is a way to assert that a give dm-* is owned by multipath.

Thanks a lot !

> 
> Thanks,
> 
>                 Ilya
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: /sys/block and /dev and partitions

[Ilya Dryomov]

On Sat, Aug 15, 2015 at 11:56 PM, Loic Dachary <loic@dachary.org> wrote:
> Hi Ilya,
>
> On 15/08/2015 19:42, Ilya Dryomov wrote:
>> On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary <loic@dachary.org> wrote:
>>> Hi Sage,
>>>
>>> On 15/08/2015 16:28, Sage Weil wrote:
>>>> On Sat, 15 Aug 2015, Loic Dachary wrote:
>>>>> Hi,
>>>>>
>>>>> Is there a portable and consistent way to figure out if a given /dev/XXX
>>>>> path (for instance /dev/dm-1) is a partition of a whole device ?
>>>>> Although checking /sys/block/dm-1/dm/name for a number at the end (like
>>>>> mpatha1 or mpatha2) would probably work, it feels like a fragile hack.
>>>>> Looking into /sys/block/dm-1/slaves will lead to
>>>>> /sys/block/dm-1/slaves/dm-0 and we can check that
>>>>> /sys/block/dm-*/subsystem is class/block. But that does not necessarily
>>>>> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.
>>>>
>>>> Take a look at is_partition in ceph-disk, whih is the best I came up with.
>>>> Basically it checks if the device name appears as /sys/block/*/$foo...
>>
>> For regular devices, you can access() /sys/dev/block/maj:min/partition.
>> If it's there, it's a partition - no need to iterate over /sys/block.
>
> I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code.
>
>>
>>>
>>> That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found.
>>
>> A better way might be to rely on the fact that a dm partition will
>> necessarily have its uuid prefixed by "part".  In that case, it should
>> be safe to assume that the thing in slaves is a whole disk - I think
>> that's what various util-linux tools do.  However, IIRC the dm uuid is
>> optional, so that won't work on a dm device without a uuid.
>
> It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way.
>
> Is it also safe to assume that if the uuid is:
>
> $ cat /sys/dev/block/253:?/dm/uuid
> mpath-353333330000007d0
> part1-mpath-353333330000007d0
> part2-mpath-353333330000007d0
>
> it means these were created by multipath because of the mpath ? When asking dmsetup with:

Yes, I think so.  I'm pretty sure these "part<id>-" and "mpath-"
prefixes were devised for exactly this purpose.

Thanks,

                Ilya

Re: /sys/block and /dev and partitions

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2600 bytes --]



On 16/08/2015 00:00, Ilya Dryomov wrote:
> On Sat, Aug 15, 2015 at 11:56 PM, Loic Dachary <loic@dachary.org> wrote:
>> Hi Ilya,
>>
>> On 15/08/2015 19:42, Ilya Dryomov wrote:
>>> On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary <loic@dachary.org> wrote:
>>>> Hi Sage,
>>>>
>>>> On 15/08/2015 16:28, Sage Weil wrote:
>>>>> On Sat, 15 Aug 2015, Loic Dachary wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Is there a portable and consistent way to figure out if a given /dev/XXX
>>>>>> path (for instance /dev/dm-1) is a partition of a whole device ?
>>>>>> Although checking /sys/block/dm-1/dm/name for a number at the end (like
>>>>>> mpatha1 or mpatha2) would probably work, it feels like a fragile hack.
>>>>>> Looking into /sys/block/dm-1/slaves will lead to
>>>>>> /sys/block/dm-1/slaves/dm-0 and we can check that
>>>>>> /sys/block/dm-*/subsystem is class/block. But that does not necessarily
>>>>>> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.
>>>>>
>>>>> Take a look at is_partition in ceph-disk, whih is the best I came up with.
>>>>> Basically it checks if the device name appears as /sys/block/*/$foo...
>>>
>>> For regular devices, you can access() /sys/dev/block/maj:min/partition.
>>> If it's there, it's a partition - no need to iterate over /sys/block.
>>
>> I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code.
>>
>>>
>>>>
>>>> That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found.
>>>
>>> A better way might be to rely on the fact that a dm partition will
>>> necessarily have its uuid prefixed by "part".  In that case, it should
>>> be safe to assume that the thing in slaves is a whole disk - I think
>>> that's what various util-linux tools do.  However, IIRC the dm uuid is
>>> optional, so that won't work on a dm device without a uuid.
>>
>> It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way.
>>
>> Is it also safe to assume that if the uuid is:
>>
>> $ cat /sys/dev/block/253:?/dm/uuid
>> mpath-353333330000007d0
>> part1-mpath-353333330000007d0
>> part2-mpath-353333330000007d0
>>
>> it means these were created by multipath because of the mpath ? When asking dmsetup with:
> 
> Yes, I think so.  I'm pretty sure these "part<id>-" and "mpath-"
> prefixes were devised for exactly this purpose.
> 

Excellent !

> Thanks,
> 
>                 Ilya
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: /sys/block and /dev and partitions

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2986 bytes --]

Hi Ilya,

For regular devices such as /dev/vdb2 or /dev/sda3, do you think it is safe to use /sys/dev/block/M:m/partition to figure out the partition number ? Or could it vary depending on the disk driver or the partition table layout ? Or the kernel version ?

Cheers

On 16/08/2015 00:10, Loic Dachary wrote:
> 
> 
> On 16/08/2015 00:00, Ilya Dryomov wrote:
>> On Sat, Aug 15, 2015 at 11:56 PM, Loic Dachary <loic@dachary.org> wrote:
>>> Hi Ilya,
>>>
>>> On 15/08/2015 19:42, Ilya Dryomov wrote:
>>>> On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary <loic@dachary.org> wrote:
>>>>> Hi Sage,
>>>>>
>>>>> On 15/08/2015 16:28, Sage Weil wrote:
>>>>>> On Sat, 15 Aug 2015, Loic Dachary wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a portable and consistent way to figure out if a given /dev/XXX
>>>>>>> path (for instance /dev/dm-1) is a partition of a whole device ?
>>>>>>> Although checking /sys/block/dm-1/dm/name for a number at the end (like
>>>>>>> mpatha1 or mpatha2) would probably work, it feels like a fragile hack.
>>>>>>> Looking into /sys/block/dm-1/slaves will lead to
>>>>>>> /sys/block/dm-1/slaves/dm-0 and we can check that
>>>>>>> /sys/block/dm-*/subsystem is class/block. But that does not necessarily
>>>>>>> mean dm-1 is a partition of dm-0, just that it's a slave of dm-0.
>>>>>>
>>>>>> Take a look at is_partition in ceph-disk, whih is the best I came up with.
>>>>>> Basically it checks if the device name appears as /sys/block/*/$foo...
>>>>
>>>> For regular devices, you can access() /sys/dev/block/maj:min/partition.
>>>> If it's there, it's a partition - no need to iterate over /sys/block.
>>>
>>> I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code.
>>>
>>>>
>>>>>
>>>>> That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found.
>>>>
>>>> A better way might be to rely on the fact that a dm partition will
>>>> necessarily have its uuid prefixed by "part".  In that case, it should
>>>> be safe to assume that the thing in slaves is a whole disk - I think
>>>> that's what various util-linux tools do.  However, IIRC the dm uuid is
>>>> optional, so that won't work on a dm device without a uuid.
>>>
>>> It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way.
>>>
>>> Is it also safe to assume that if the uuid is:
>>>
>>> $ cat /sys/dev/block/253:?/dm/uuid
>>> mpath-353333330000007d0
>>> part1-mpath-353333330000007d0
>>> part2-mpath-353333330000007d0
>>>
>>> it means these were created by multipath because of the mpath ? When asking dmsetup with:
>>
>> Yes, I think so.  I'm pretty sure these "part<id>-" and "mpath-"
>> prefixes were devised for exactly this purpose.
>>
> 
> Excellent !
> 
>> Thanks,
>>
>>                 Ilya
>>
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: /sys/block and /dev and partitions

[Ilya Dryomov]

On Tue, Aug 18, 2015 at 12:46 AM, Loic Dachary <loic@dachary.org> wrote:
> Hi Ilya,
>
> For regular devices such as /dev/vdb2 or /dev/sda3, do you think it is safe to use /sys/dev/block/M:m/partition to figure out the partition number ? Or could it vary depending on the disk driver or the partition table layout ? Or the kernel version ?

Yes, I think it is.  "partition" was added specifically for finding out
partition numbers after extended devt scheme was merged, because it
made it impossible to use arithmetic on minor numbers.  That happened
around 2.6.27 - 2.6.28, so it's a more recent one compared to others,
but I don't think you care about anything pre 2.6.32.

(You could choose not to rely on "partition" and access() "start" for
figuring out if the thing is a partition - it's been around forever,
but then, the only way to figure out the partition index would be to
parse the name, which you want to avoid.)

Thanks,

                Ilya

Re: /sys/block and /dev and partitions

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1172 bytes --]



On 18/08/2015 11:53, Ilya Dryomov wrote:
> On Tue, Aug 18, 2015 at 12:46 AM, Loic Dachary <loic@dachary.org> wrote:
>> Hi Ilya,
>>
>> For regular devices such as /dev/vdb2 or /dev/sda3, do you think it is safe to use /sys/dev/block/M:m/partition to figure out the partition number ? Or could it vary depending on the disk driver or the partition table layout ? Or the kernel version ?
> 
> Yes, I think it is.  "partition" was added specifically for finding out
> partition numbers after extended devt scheme was merged, because it
> made it impossible to use arithmetic on minor numbers.  That happened
> around 2.6.27 - 2.6.28, so it's a more recent one compared to others,
> but I don't think you care about anything pre 2.6.32.
> 
> (You could choose not to rely on "partition" and access() "start" for
> figuring out if the thing is a partition - it's been around forever,
> but then, the only way to figure out the partition index would be to
> parse the name, which you want to avoid.)

Exactly. Really happy to get rid of the name parsing logic :-)

> Thanks,
> 
>                 Ilya
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]