PV on disk without partitions not recognised as LVM2_member

PV on disk without partitions not recognised as LVM2_member

[Xen]

I am not getting a majordomo response (as usual, I guess, within a 
reasonable amount of time :p) to subscribe to this list, but....



Perchance,

Would someone be will to fix the issue that a Physical Volume from LVM2 
(PV) when placed directly on disk (no partitions or partition tables) 
will not be recognised as LVM2_member but rather as something else, such 
as "dos" (if nothing else is found) or e.g. some RAID device (if a RAID 
signature exists at the end of the device.

Ie. I had a disk that had a "promise fasttrack raid member" (from 
memory) signature at the end of the disk (last 1MB) and was recognised 
as such. When I wiped the signature, it was recognised as "dos":

/dev/sda: PTUUID="ef39c6e5" PTTYPE="dos"


pvck /dev/sda:

   Found label on /dev/sda, sector 1, type=LVM2 001
   Found text metadata area: offset=4096, size=1044480


The LVM2 udev rules for lvm-metad depend on blkid currently to report 
the nature of a block device such that they can know whether to activate 
it; as such a PV directly on disk will not get activated by udev rules.

Will also not get scanned, the whole pvscan --cache --activate ay 
command will not get executed.

It seems it would be best to solve it at the root of the issue rather 
than changing LVM's udev scripts.

Regards.

Re: PV on disk without partitions not recognised as LVM2_member

[Xen]

Xen schreef op 18-08-2016 22:39:

> Ie. I had a disk that had a "promise fasttrack raid member" (from
> memory) signature at the end of the disk (last 1MB) and was recognised
> as such. When I wiped the signature, it was recognised as "dos":
> 
> /dev/sda: PTUUID="ef39c6e5" PTTYPE="dos"

Oh, I mean... the PTTYPE would be "dos" but there would not be any 
FS_TYPE description (as per the udev name?). In any case.

Both regular "msdos" partition tables and PV signatures get an empty 
"TYPE" in blkid.

So I can match those in udev by just matching the empty string. Both are 
also given the same PTTYPE of "dos". So I cannot distinguish between 
msdos partition tables and PV disks.

Regards.

Re: PV on disk without partitions not recognised as LVM2_member

[Karel Zak]

On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
> Would someone be will to fix the issue that a Physical Volume from LVM2 (PV)
> when placed directly on disk (no partitions or partition tables) will not be

This is very unusual setup, but according to feedback from LVM guys
it's supported, so I will improve blkid to support it too.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: PV on disk without partitions not recognised as LVM2_member

[Xen]

Karel Zak schreef op 19-08-2016 13:14:
> On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
>> Would someone be will to fix the issue that a Physical Volume from 
>> LVM2 (PV)
>> when placed directly on disk (no partitions or partition tables) will 
>> not be
> 
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.

Thank you.

It is just extremely unusable because hardly anyone can do it yet.

The Grub support isn't there to boot from it (although I have a patch 
for that) and without being able to boot it no one would ordinarily use 
it. So it's just a chicken and the egg problem here.

As soon as they can get activated, people could start using them, but at 
this point it's almost impossible (it requires adjustments to systemd or 
the initrd that would be manual adjustments at this point).

Thanks :).

Regards.

Re: PV on disk without partitions not recognised as LVM2_member

[Xen]

Karel Zak schreef op 19-08-2016 13:14:
> On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
>> Would someone be will to fix the issue that a Physical Volume from 
>> LVM2 (PV)
>> when placed directly on disk (no partitions or partition tables) will 
>> not be
> 
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.

To provide a little feedback.

What is actually the case is that I am installing Grub2 on the PV which 
uses the LVM2 "--bootloaderareasize" option to pvcreate.

This creates a bootloader-area of say 1M with an offset like perhaps 
2048 sectors, I don't remember how to get that info, after which then 
the extents start.

Oh yes, it is this sort of thing:

physical_volumes {

     pv0 {
         id = "fEGbBn-tbIp-rL7y-m22b-1rQh-r9i5-Qwlqz7"
         device = "/dev/sda"     # Hint only

         status = ["ALLOCATABLE"]
         flags = []
         dev_size = 1258287104   # 599,998 Gigabytes
         pe_start = 4096
         pe_count = 153599       # 599,996 Gigabytes
         ba_start = 2048
         ba_size = 2048  # 1024 Kilobytes
     }
}

Grub would naturally create an MBR signature in the first sector, no 
partition table, but bootloader code (boot.img).

The result is this:

5182: libblkid: LOWPROBE: [16] LVM2_member:
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       magic sboff=536, kboff=0
5182: libblkid: LOWPROBE:       call probefunc()
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE: assigning UUID [superblocks]
5182: libblkid: LOWPROBE: wiper set to superblocks::LVM2_member off=0 
size=8192
5182: libblkid: LOWPROBE: assigning TYPE [superblocks]
5182: libblkid: LOWPROBE: <-- leaving probing loop (type=LVM2_member) 
[SUBLKS idx=16]
5182: libblkid: LOWPROBE: freeing values list
5182: libblkid: LOWPROBE: chain safeprobe topology DISABLED
5182: libblkid: LOWPROBE: chain safeprobe partitions ENABLED
5182: libblkid: LOWPROBE: reseting partitions values
5182: libblkid: LOWPROBE: --> starting probing loop [PARTS idx=-1]
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       magic sboff=510, kboff=0
5182: libblkid: LOWPROBE: dos: ---> call probefunc()
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       magic sboff=0, kboff=0
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x21e3160
5182: libblkid: LOWPROBE: previously wiped area modified  -- ignore 
previous results
5182: libblkid: LOWPROBE: zeroize wiper
5182: libblkid: LOWPROBE: reseting superblocks values
5182: libblkid: LOWPROBE:  free value UUID
5182: libblkid: LOWPROBE:  free value TYPE
5182: libblkid: LOWPROBE: assigning PTUUID [partitions]
5182: libblkid: LOWPROBE: dos: <--- (rc = 0)
5182: libblkid: LOWPROBE: assigning PTTYPE [partitions]
5182: libblkid: LOWPROBE: <-- leaving probing loop (type=dos) [PARTS 
idx=3]


And ...that's the real truth of it. I'm sorry, I hadn't realized it 
would be due to the bootsector.

When I wipe the bootsector actually it does find LVM2_member.

So I am sorry I didn't mention. The culprit is the Grub2 image.

5261: libblkid: LOWPROBE: [16] LVM2_member:
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       magic sboff=536, kboff=0
5261: libblkid: LOWPROBE:       call probefunc()
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: assigning UUID [superblocks]
5261: libblkid: LOWPROBE: wiper set to superblocks::LVM2_member off=0 
size=8192
5261: libblkid: LOWPROBE: assigning TYPE [superblocks]
5261: libblkid: LOWPROBE: <-- leaving probing loop (type=LVM2_member) 
[SUBLKS idx=16]
5261: libblkid: LOWPROBE: freeing values list
5261: libblkid: LOWPROBE: chain safeprobe topology DISABLED
5261: libblkid: LOWPROBE: chain safeprobe partitions ENABLED
5261: libblkid: LOWPROBE: reseting partitions values
5261: libblkid: LOWPROBE: --> starting probing loop [PARTS idx=-1]
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: gpt: ---> call probefunc()
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: gpt: <--- (rc = 1)
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: ultrix: ---> call probefunc()
5261: libblkid: LOWPROBE:       buffer read: off=15872 len=512 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: ultrix: <--- (rc = 1)
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       buffer read: off=28672 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE:       reuse buffer: off=0 len=1024 
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: <-- leaving probing loop (failed=1) [PARTS 
idx=11]
5261: libblkid: LOWPROBE: parts: start probing for partition entry
5261: libblkid: LOWPROBE: parts: end probing for partition entry 
[nothing]
5261: libblkid: LOWPROBE: partitions probe done [rc=1]
5261: libblkid: LOWPROBE: 0x1c4a0e0: end probe
5261: libblkid: LOWPROBE: zeroize wiper
5261: libblkid: LOWPROBE: returning UUID value
5261: libblkid:      TAG:     found cache tag head UUID
5261: libblkid: LOWPROBE: returning TYPE value
5261: libblkid:      TAG:     found cache tag head TYPE
5261: libblkid:    PROBE: /dev/sda: devno 0x0800, type LVM2_member

Pretty good debug output, by the way.

Anyway. Thanks to Zdenek Kabelac I found this issue. He knew what was 
going on.

Regards.

Re: PV on disk without partitions not recognised as LVM2_member

[Xen]

Karel Zak schreef op 19-08-2016 13:14:

> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.

Actually there are more issues. If you have any firmware RAID signature 
at the end of your disk, but you are not using it, blkid will also give 
that raid signature precendence over LVM2_member, but LVM depends on 
that blkid scan to know what to do; and will fail to scan (even though 
pvscan could handle it easily) when the required Udev rule doesn't 
match.

So when this part is fixed, anyone who has mistakenly perhaps created a 
raid array out of their firmware BIOS, will end up with the same 
situation because .... how are we going to let LVM2_member take 
precedence just because it uses blkid to decide when to call itself 
honey?

I mean I can see good reasons why a RAID signature would take 
precedence.

In this case it just disrupts the system LVM2 has set up, in which there 
is no redundancy and it depends on an external tool to do its scanning 
:-/.

If you replaced that entire udev rule file with just a single command 
(and a block device filter) it would just always work with no pain, but 
because they want to minimize the number of pvscan invocations (even if 
it is just 3 extra invocations that don't harm anyone at every boot) the 
system now fails to work. In its entirety. Because it depends on blkid 
to provide the right signature.

So basically they could simplify the system and do away with udev in 
that sense at the cost of a minimal amount of extra pvscan invocations 
that don't really bring any cost to the table. But would they do it?

You now have a problem that cannot be solved:

- blkid cannot be asked to give precedence to LMV2_member over raid 
signatures
- while lvm2 chooses to depend on blkid, it won't find all PVs. It 
limits its search, but you can't really blame blkid for that unless 
blkid started giving multiple results for every device. It would need to 
become a list, a set. You can't ask that of blkid just for that right. 
So now you have an issue that has no solution. Because they depend on 
blkid instead of their own internal code, which wasn't really all that 
necessary. I mean it is nice to be needed at times.

But the only solution that would really be painless is to just dump that 
lvm2 udev file and replace it with something extremely simple. :-/. ;-).

What can I say. It seems obvious.

I mean that doesn't take away from the current issue with blkid not 
recognising PV when Grub is present. But it does show the bigger problem 
in this... I think.

Anyway, regards, and sorry for the verbosity at times. Regards.

Re: PV on disk without partitions not recognised as LVM2_member

[Karel Zak]

On Sat, Aug 20, 2016 at 04:13:52PM +0200, Xen wrote:
> Karel Zak schreef op 19-08-2016 13:14:
> 
> > This is very unusual setup, but according to feedback from LVM guys
> > it's supported, so I will improve blkid to support it too.
> 
> Actually there are more issues. If you have any firmware RAID signature at
> the end of your disk, but you are not using it

You have to use "wipefs" to remove unwanted signatures. We do not
support random mess on disks. It's OS installer, mkfs-like, fdisk
tools and users responsibility to keep on-disk stuff in reliable
state.

It's not about RAIDs only, it's possible write many PT, filesystem and
raid signatures to the same disk.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: PV on disk without partitions not recognised as LVM2_member

[Xen]

Karel Zak schreef op 22-08-2016 10:40:
> On Sat, Aug 20, 2016 at 04:13:52PM +0200, Xen wrote:
>> Karel Zak schreef op 19-08-2016 13:14:
>> 
>> > This is very unusual setup, but according to feedback from LVM guys
>> > it's supported, so I will improve blkid to support it too.
>> 
>> Actually there are more issues. If you have any firmware RAID 
>> signature at
>> the end of your disk, but you are not using it
> 
> You have to use "wipefs" to remove unwanted signatures. We do not
> support random mess on disks. It's OS installer, mkfs-like, fdisk
> tools and users responsibility to keep on-disk stuff in reliable
> state.

Right. I didn't know wipefs would do that. I am approaching this just as 
a user from this perspective. It's just a practical issue because RAID 
(firmware) (what they call fakeraid) is hard to use on Linux, not that 
obvious, dm-raid package is required, it is broken on Ubuntu, you are 
not likely to use pmc-whatever-undecipherable-string so fast if it is 
just JBOD.

JBOD of single disk (for some controllers that is spare disk, kinda) is 
probably raw disk + 1 MB.

I know it is crap, some controllers won't allow decent passthough and 
require you to configure every disk as a raid disk, creating the 
problems here.

I'll take note of wipefs but just the next user will run into it also. 
No shortage of things to learn before you can use your system ;-).


My problem is more lack of redundancy; all tools work together, and they 
work together perfectly, but if one slightly fails, the whole thing 
collapses, because they are all weakest links.

Whole udev system is liability, a single systemd vgscan service (like 
real lvm2.service file, no /etc/init.d) would solve every udev problem 
there could ever be with loading devices at boot; vgscan -ay will 
activate every device in the book that would also ordinarily have been 
activated by udev rules regardless.

Dependability on flawed or fallable or fragile system == .....

pvscan/vgscan/vgscan (I mean vgchange service) has the tools to deal 
with duplicate devices resulting from raid-based activation and raw 
device (same UUID, it handles it) so system is already resilient at the 
core but their udev rules make it break.

Single vgchange -ay would solve all issues but it is contrary to the 
design philosophy of streamlining everything with events.

I'm in favour of barriers and completion of stages: activate all LVM you 
can, only then activate filesystems.

If you have crypttarget based on LVM activation, sure streamlining it 
after the device has come online is not bad, or rather, auto-activate 
cryptsetup result using udev is not bad (form of hotplug, you might say) 
but manual calls work better and are robust and are resilient.

That's just my opinion. Too many things break in the Linux system when 
you change the smallest thing and it needs to "stop" as they say ;-).

Just kidding that. People who think they are awesome people sitting in 
their chairs say that a lot "this needs to stop!". Sure. I'm sure it 
does. Will you do it? Go back to your kitchen then, and make me some 
pie.

Claims of impotence, those are.

Anyway.

> It's not about RAIDs only, it's possible write many PT, filesystem and
> raid signatures to the same disk.

I agree but... this just means that person creating firmware raid array 
might cause the system to fail booting only because of blkid interlink 
and its interdependence. When otherwise it wouldn't. Filesystem / device 
is deemed RAID signature but partition table still loads.

So partition table loading not dependent on blkid == resilient. 
Microsoft Windows is notorious for failing to boot when you put a disk 
on a different controller etc. Millions, millions of boot problems when 
you change things. Change system from IDE to RAID, system will not load. 
AHCI same. Loads different driver. Doesn't load all drivers by default. 
I think Windows 10 has that sorted now. Not want to talk about Windows, 
just explanation of where it goes wrong there. Their boot recovery is 
abysmal. It just stinks. It stinks worse than Linux ever was, but Linux 
is now also not so resilient anymore.

Don't want to crack down on Linux, I am a Linux user. Still, just 
saying.

I would have a lot less headaches (or footaches) if the system was more 
robust. Just saying.

Just add an invalid line for /var to /etc/fstab and you will have a 
failing boot system.

Users are always asked to edit that file.

nofail and noauto not heeded. Just saying. SystemD pulls it in, you 
can't mask it either.

Crypt device that you don't need but it is in /etc/cryttab will require 
to be unlocked at boot, there is no intelligence to determine whether it 
would actually halt or obstruct the system, also no way to unlock it 
after boot, there is no software for that (no LUKS software for that) 
that is moderately user friendly.

Post-boot systems perhaps possible with VeraCrypt these days, nothing 
else.

VeraCrypt designers cause too many iteration cycles over TrueCrypt, now 
unusable system. Not user friendly anymore. Takes too long to unlock 
something. Not usable anymore. But they know better than actual users. 
Always know better than users.

Sorry for the rant, was not my intent ;-).

Regards, and thanks for your help.

Kudos.

Re: PV on disk without partitions not recognised as LVM2_member

[Karel Zak]

On Fri, Aug 19, 2016 at 01:14:29PM +0200, Karel Zak wrote:
> On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
> > Would someone be will to fix the issue that a Physical Volume from LVM2 (PV)
> > when placed directly on disk (no partitions or partition tables) will not be
> 
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.

Fixed in the git tree (for the next v2.29). Thanks.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com