From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx1.redhat.com (ext-mx14.extmail.prod.ext.phx2.redhat.com [10.5.110.19]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id p7AGUoK0015304 for ; Wed, 10 Aug 2011 12:30:50 -0400 Received: from cmiomail.amdocs.com (cmiomail.amdocs.com [68.77.114.71]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p7AGUmm0007297 for ; Wed, 10 Aug 2011 12:30:48 -0400 Received: from USCMIMAILFE2.corp.amdocs.com (unknown [10.120.38.116]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by cmiomail.amdocs.com (Postfix) with ESMTPS id D5C05D80CD for ; Wed, 10 Aug 2011 11:30:46 -0500 (CDT) From: Jewsco Pius Jacquez Date: Wed, 10 Aug 2011 11:30:46 -0500 Message-ID: <883D98939E388E429EE8D771168A6560027EE6AE1D@USCMIMAIL1.corp.amdocs.com> Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_883D98939E388E429EE8D771168A6560027EE6AE1DUSCMIMAIL1cor_" MIME-Version: 1.0 Subject: [linux-lvm] metadata copies Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: "linux-lvm@redhat.com" --_000_883D98939E388E429EE8D771168A6560027EE6AE1DUSCMIMAIL1cor_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello, If you have two metadata copies stored in one PV, how does the OS know wh= ich one is the legitimate copy? Thanks, Jewsco This message and the information contained herein is proprietary and conf= idential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp --_000_883D98939E388E429EE8D771168A6560027EE6AE1DUSCMIMAIL1cor_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable

Hello,<= /o:p>

 

I= f you have two metadata copies stored in one PV, how does the OS know whi= ch one is the legitimate copy?

&n= bsp;

Thanks,
Jewsco

= 
This message =
and the information contained herein is proprietary and confidential and =
subject to the Amdocs policy statement,
you may review at http://www.amdocs.com/email_disclaimer.asp
=
--_000_883D98939E388E429EE8D771168A6560027EE6AE1DUSCMIMAIL1cor_-- From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4E42E454.7000902@redhat.com> Date: Wed, 10 Aug 2011 22:04:36 +0200 From: Milan Broz MIME-Version: 1.0 References: <883D98939E388E429EE8D771168A6560027EE6AE1D@USCMIMAIL1.corp.amdocs.com> In-Reply-To: <883D98939E388E429EE8D771168A6560027EE6AE1D@USCMIMAIL1.corp.amdocs.com> Content-Transfer-Encoding: 7bit Subject: Re: [linux-lvm] metadata copies Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="us-ascii" To: LVM general discussion and development Cc: Jewsco Pius Jacquez On 08/10/2011 06:30 PM, Jewsco Pius Jacquez wrote: > If you have two metadata copies stored in one PV, how does the OS know which one is the legitimate copy? Only one metadata copy is always active. There is sequence ID (see seqno = X in metadata backup). If there are several PVs and more versions of metadata present, the one with highest sequence ID is used and other PVs are automatically updated to this version. Every LVM operation which changes metadata also increases this sequence id. Milan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx1.redhat.com (ext-mx11.extmail.prod.ext.phx2.redhat.com [10.5.110.16]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p7AL0U0X025013 for ; Wed, 10 Aug 2011 17:00:31 -0400 Received: from mail.bmsi.com (www.bmsi.com [24.248.44.156]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p7AL0T7n015302 for ; Wed, 10 Aug 2011 17:00:30 -0400 Date: Wed, 10 Aug 2011 17:00:11 -0400 (EDT) From: "Stuart D. Gathman" In-Reply-To: <4E42E454.7000902@redhat.com> Message-ID: References: <883D98939E388E429EE8D771168A6560027EE6AE1D@USCMIMAIL1.corp.amdocs.com> <4E42E454.7000902@redhat.com> MIME-Version: 1.0 Subject: Re: [linux-lvm] metadata copies Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: TEXT/PLAIN; charset="us-ascii"; format="flowed" Content-Transfer-Encoding: 7bit To: LVM general discussion and development Cc: Jewsco Pius Jacquez On Wed, 10 Aug 2011, Milan Broz wrote: > On 08/10/2011 06:30 PM, Jewsco Pius Jacquez wrote: >> If you have two metadata copies stored in one PV, how does the OS know which >> one is the legitimate copy? > > Only one metadata copy is always active. > > There is sequence ID (see seqno = X in metadata backup). > If there are several PVs and more versions of metadata present, the one with > highest sequence ID is used and other PVs are automatically updated to this > version. > > Every LVM operation which changes metadata also increases this sequence id. What happens if there are 2 or more metadata copies with the same sequence ID, but different contents? Not just buggy/malicious code can cause this. Imagine that a careless admin removes a PV, puts it on another system, independently updates both systems for a while, then later adds the PV back to the original system (and the seqnos match). (Guesses - not authoritative information) 1) I believe the metadata includes a hash/checksum, so that an incomplete copy of the metadata is easily detected. (Another reason to have at least 2 copies - in case of power loss during metadata update.) 2) I suspect there is no clever algorithm, and it probably uses the first valid copy seen with the highest sequence. AIX had a "quorum" system where the majority of metadata copies had to agree - or operator intervention was required to bring the VG online. -- Stuart D. Gathman Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4E439CD7.2010105@redhat.com> Date: Thu, 11 Aug 2011 11:11:51 +0200 From: Milan Broz MIME-Version: 1.0 References: <883D98939E388E429EE8D771168A6560027EE6AE1D@USCMIMAIL1.corp.amdocs.com> <4E42E454.7000902@redhat.com> In-Reply-To: Content-Transfer-Encoding: 7bit Subject: Re: [linux-lvm] metadata copies Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="us-ascii" To: LVM general discussion and development Cc: Jewsco Pius Jacquez On 08/10/2011 11:00 PM, Stuart D. Gathman wrote: > What happens if there are 2 or more metadata copies with the same sequence > ID, but different contents? Not just buggy/malicious code can cause this. > Imagine that a careless admin removes a PV, puts it on another system, > independently updates both systems for a while, then later adds the PV back to > the original system (and the seqnos match). I did not mention two other checks: - there is a checksum of text metadata in PV header, so random corruption is detected - the whole update process is atomic, there is a round buffer, new version of metadata is written and if done successfully, atomic sector update changes just pointer (so either new or old version is active) > 2) I suspect there is no clever algorithm, and it probably uses the first > valid copy seen with the highest sequence. AIX had a "quorum" system > where the majority of metadata copies had to agree - or operator intervention > was required to bring the VG online. Metadata should be updated on all PVs when the code detect inconsistency. TBH I am not sure if this happens in every malicious situation (correct metadata versions with different content but the same ID). Maybe if anyone want to try it - report a bug if there is a problem handling it. (IMHO this check should be somehow added to vgck.) Anyway, there is date, lvm version, full command and system name written to text metadata (from the system where it was written), so manual admin interaction should be trivial - just vgcfgrestore proper backup file.) Milan