[linux-lvm] metadata copies

[linux-lvm] metadata copies

[Jewsco Pius Jacquez]

[-- Attachment #1: Type: text/plain, Size: 326 bytes --]

Hello,

If you have two metadata copies stored in one PV, how does the OS know which one is the legitimate copy?

Thanks,
Jewsco

This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
you may review at http://www.amdocs.com/email_disclaimer.asp

[-- Attachment #2: Type: text/html, Size: 4757 bytes --]

Re: [linux-lvm] metadata copies

[Milan Broz]

On 08/10/2011 06:30 PM, Jewsco Pius Jacquez wrote:
> If you have two metadata copies stored in one PV, how does the OS know which one is the legitimate copy?

Only one metadata copy is always active.

There is sequence ID (see seqno = X in metadata backup).
If there are several PVs and more versions of metadata present, the one with highest sequence ID
is used and other PVs are automatically updated to this version.

Every LVM operation which changes metadata also increases this sequence id.

Milan

Re: [linux-lvm] metadata copies

[Stuart D. Gathman]

On Wed, 10 Aug 2011, Milan Broz wrote:

> On 08/10/2011 06:30 PM, Jewsco Pius Jacquez wrote:
>> If you have two metadata copies stored in one PV, how does the OS know which
>> one is the legitimate copy?
>
> Only one metadata copy is always active.
>
> There is sequence ID (see seqno = X in metadata backup).
> If there are several PVs and more versions of metadata present, the one with
> highest sequence ID is used and other PVs are automatically updated to this
> version.
>
> Every LVM operation which changes metadata also increases this sequence id.

What happens if there are 2 or more metadata copies with the same sequence 
ID, but different contents?  Not just buggy/malicious code can cause this.
Imagine that a careless admin removes a PV, puts it on another system,
independently updates both systems for a while, then later adds the PV back to
the original system (and the seqnos match).

(Guesses - not authoritative information)

1) I believe the metadata includes a hash/checksum, so that an incomplete
copy of the metadata is easily detected.  (Another reason to have at
least 2 copies - in case of power loss during metadata update.)

2) I suspect there is no clever algorithm, and it probably uses the first
valid copy seen with the highest sequence.  AIX had a "quorum" system 
where the majority of metadata copies had to agree - or operator intervention
was required to bring the VG online.

--
 	      Stuart D. Gathman <stuart@bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] metadata copies

[Milan Broz]

On 08/10/2011 11:00 PM, Stuart D. Gathman wrote:

> What happens if there are 2 or more metadata copies with the same sequence 
> ID, but different contents?  Not just buggy/malicious code can cause this.
> Imagine that a careless admin removes a PV, puts it on another system,
> independently updates both systems for a while, then later adds the PV back to
> the original system (and the seqnos match).

I did not mention two other checks:
- there is a checksum of text metadata in PV header, so random
corruption is detected

- the whole update process is atomic, there is a round buffer, new
version of metadata is written and if done successfully, atomic
sector update changes just pointer (so either new or old version is active)

> 2) I suspect there is no clever algorithm, and it probably uses the first
> valid copy seen with the highest sequence.  AIX had a "quorum" system 
> where the majority of metadata copies had to agree - or operator intervention
> was required to bring the VG online.

Metadata should be updated on all PVs when the code detect inconsistency.

TBH I am not sure if this happens in every malicious situation
(correct metadata versions with different content but the same ID).

Maybe if anyone want to try it - report a bug if there is a problem
handling it. (IMHO this check should be somehow added to vgck.)

Anyway, there is date, lvm version, full command and system name written
to text metadata (from the system where it was written), so manual admin
interaction should be trivial - just vgcfgrestore proper backup file.)

Milan