From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B22CC433EF for ; Thu, 21 Oct 2021 06:07:33 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B2C1660ED5 for ; Thu, 21 Oct 2021 06:07:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B2C1660ED5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=redhat.com Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-199-xTaA3lvRM3WEFHtInpGmvA-1; Thu, 21 Oct 2021 02:07:27 -0400 X-MC-Unique: xTaA3lvRM3WEFHtInpGmvA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C18FB10A8E05; Thu, 21 Oct 2021 06:07:21 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8251060C5F; Thu, 21 Oct 2021 06:07:21 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 30BA54EA2A; Thu, 21 Oct 2021 06:07:21 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 19KI55VZ027972 for ; Wed, 20 Oct 2021 14:05:05 -0400 Received: by smtp.corp.redhat.com (Postfix) id 1A5DE11558B6; Wed, 20 Oct 2021 18:05:05 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast02.extmail.prod.ext.rdu2.redhat.com [10.11.55.18]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1558B11558B5 for ; Wed, 20 Oct 2021 18:04:58 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C61B6800B26 for ; Wed, 20 Oct 2021 18:04:58 +0000 (UTC) Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-41-2gS79wcCO9-HgLle1W__xQ-1; Wed, 20 Oct 2021 14:04:56 -0400 X-MC-Unique: 2gS79wcCO9-HgLle1W__xQ-1 Received: by mail-qv1-f54.google.com with SMTP id k19so2550456qvm.13 for ; Wed, 20 Oct 2021 11:04:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=btkMYmriKbvSr0gl0iuewEgK9TZi3VwUqn4A2Dm0/HY=; b=ezaLAcIdixlDpbyZtkT3d72CAKC1+iOor80Bd760EyD+5X+2GSk0C6+CUvNuNST52E BTvdne0HxeVonBqI6ADkNyW+QGu346Ql9k6L0VqZryhfVyzasI6Nj5T2V9yyXc9d+wL3 5d/uVK0sZ4gvPMmYp+QXhAf73UHt/wp9ZUYP1VbytZaqv3kZTnOMSdCPDkSTqFk51kPi +VA4vhLnowOyWLMSukC6va8WJzVBrjqqQUs27t5zqQKmRuxXV7bErxddoNBAJQ7WjlQp CBnbap4v5He4JMBx7OLMsI/SPGbkvn3oo98sKG6zjHVsSY7/W01hEjnqtGmiKzCOvSyu tNkA== X-Gm-Message-State: AOAM533CFd2uXe2DqltVVljAjedGzFK4f5RFlm+gI32IxxMTBFdFzgL0 go7mXtr4f2MjaeL0b68BdjTlYBDJ2vREhNwDHQXwKekx X-Google-Smtp-Source: ABdhPJxMPhaxfONBzPrewIXkTtb9es/4Kvnkb/8K4j7S7QMt8+DTNph89BqwCfVIC0kljERb2HkBVsETUjG8kBR5Vqs= X-Received: by 2002:a0c:f0c4:: with SMTP id d4mr215181qvl.38.1634753096070; Wed, 20 Oct 2021 11:04:56 -0700 (PDT) MIME-Version: 1.0 References: <20211018021719.GA16936@bdmcc-us.com> <20211018194508.GC28815@bdmcc-us.com> <20211020133828.GE9096@bdmcc-us.com> In-Reply-To: From: Roger Heflin Date: Wed, 20 Oct 2021 13:04:44 -0500 Message-ID: To: LVM general discussion and development X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-loop: linux-lvm@redhat.com X-Mailman-Approved-At: Thu, 21 Oct 2021 02:06:26 -0400 Subject: Re: [linux-lvm] Recovering "broken" disk ( 17th ) X-BeenThere: linux-lvm@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-lvm-bounces@redhat.com Errors-To: linux-lvm-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=linux-lvm-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/mixed; boundary="===============7977435902000083082==" --===============7977435902000083082== Content-Type: multipart/alternative; boundary="00000000000096389a05cecc9d56" --00000000000096389a05cecc9d56 Content-Type: text/plain; charset="UTF-8" replying to my last email. do the pvcreate -uuid and then do a pvs/lvs/vgs and see if the vg/lv's look like that are there. if so do a vgchange -ay and then test mounting the fs. And if with the fs either commented out and/or ,nofail the normal os boots up work from there as you should have backup files. On Wed, Oct 20, 2021 at 1:01 PM Roger Heflin wrote: > is the pv in the root device vg? if not changing fstab to not mount the > missing fs(es) should get it bootable. I have a practice of putting > ",nofail" on all non-root filesystems (ie defaults,nofail) since priority > #1 is getting the machine up and on the network after a reboot such that it > can be ssh'ed to and fixed as needed. > > If it is not the root device then on the root device there should be > several prior archive copy in /etc/lvm/archive/*, and maybe some > copies in /etc/lvm/backup. > > In the vg backup file there will be a bunch of uuids, you want the > specific pv uuid and not the vg/lv uuids. Each pv has a uuid and each lv > has a uuid and the vg has a uuid. > > On Wed, Oct 20, 2021 at 8:39 AM Brian McCullough > wrote: > >> On Tue, Oct 19, 2021 at 05:06:37AM -0500, Roger Heflin wrote: >> > I would edit the vgconfig you dd'ed with an editor and make sure it >> looks >> > reasonable for what you think you had. >> >> It turns out, comparing the information that I pulled off of the drive >> with what I find in /etc/lvm/backup, that the first part of the vgconfig >> information is missing. As I said in one of my messages, the >> information that I retrieved from the disk starts at 0x1200. I don't >> know whether that is correct or not. It does not appear to be a proper >> "backup" file, which I think it should be. >> >> I rebooted ( partially ) the machine and copied the vgconfig backup file >> from that, but am somewhat concerned, because I don't seem to be able to >> match the UUIDs. The one that I seem to see in the vgconfig data that I >> pulled off of the drive vs what I got out of /etc/lvm/backup. Maybe I >> am just mis-reading it. I will continue my research for a bit. >> >> >> >> >> > When you do the pvcreate --uuid it won't use anything except the uuid >> info >> > so the rest may not need to be exactly right, if you have to do a >> > vgcfgrestore to get it to read the rest of the info will be used. >> >> Oh, thank you. I did see that things got somewhat different on the >> target drive when I did "pvcreate --uuid --restorefile." I got paranoid >> when I saw that, and re-copied the ddrestore file back to the target >> drive before I did anything else. Should I do "pvcreate --uuid >> --norestorefile," instead? Then, once it is back in the machine, do the >> pvscan and vgcfgrestore, and expect good things? >> >> >> >> > I have seen some weird disk controller failures that appeared to zero >> out >> > the first bit of the disk (enough to get the partition table, grub, and >> the >> > pv header depending on where the first partition starts). >> >> I APPEAR to have a partition table, containing an NTFS partition, an LVM >> partiton ( the one that I am concentrating on ) and a Linux partion. I >> would have thought that it was all LVM, but my memory could easily be >> wrong. >> >> >> >> > You will need to reinstall grub if this was the bootable disk, since >> there >> > were 384 bytes of grub in the sector with the partition table that you >> know >> > are missing. >> >> Fortunately, this is all data, nothing to do with the boot sequence, >> except that the machine will not boot with the missing PV. >> >> >> >> Thank you, >> Brian >> >> _______________________________________________ >> linux-lvm mailing list >> linux-lvm@redhat.com >> https://listman.redhat.com/mailman/listinfo/linux-lvm >> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >> >> --00000000000096389a05cecc9d56 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
replying to my last email.

d= o the pvcreate -uuid and then do a pvs/lvs/vgs and see if the vg/lv's l= ook like that are there.=C2=A0=C2=A0 if so do a vgchange -ay <vgname>= and then test mounting the fs.

And if with the fs= either commented out and/or ,nofail the normal os boots up work from there= as you should have backup files.

On Wed, Oct 20, 2021 at 1:01 PM = Roger Heflin <rogerheflin@gmail= .com> wrote:
is the pv in the root device vg?=C2=A0 if not cha= nging fstab to not mount the missing=C2=A0 fs(es) should get it bootable.= =C2=A0=C2=A0 I have a practice of putting ",nofail" on all non-ro= ot filesystems (ie defaults,nofail) since priority #1 is getting the machin= e up and on the network after a reboot such that it can be ssh'ed to an= d fixed as needed.

If it is not the root devic= e then on the root device there should be several prior archive copy in /et= c/lvm/archive/<vgname>*, and maybe some copies in /etc/lvm/backup.

In the vg backup file there will be a bunch of u= uids, you want the specific pv uuid and not the vg/lv uuids.=C2=A0 Each pv = has a uuid and each lv has a uuid and the vg has a uuid.
On Wed, O= ct 20, 2021 at 8:39 AM Brian McCullough <bdmc@bdmcc-us.com> wrote:
On Tue, Oct 19, 2021 at 05:06:37AM= -0500, Roger Heflin wrote:
> I would edit the vgconfig you dd'ed with an editor and make sure i= t looks
> reasonable for what you think you had.

It turns out, comparing the information that I pulled off of the drive
with what I find in /etc/lvm/backup, that the first part of the vgconfig information is missing.=C2=A0 As I said in one of my messages, the
information that I retrieved from the disk starts at 0x1200.=C2=A0 I don= 9;t
know whether that is correct or not.=C2=A0 It does not appear to be a prope= r
"backup" file, which I think it should be.

I rebooted ( partially ) the machine and copied the vgconfig backup file from that, but am somewhat concerned, because I don't seem to be able t= o
match the UUIDs.=C2=A0 The one that I seem to see in the vgconfig data that= I
pulled off of the drive vs what I got out of /etc/lvm/backup.=C2=A0 Maybe I=
am just mis-reading it.=C2=A0 I will continue my research for a bit.




> When you do the pvcreate --uuid it won't use anything except the u= uid info
> so the rest may not need to be exactly right, if you have to do a
> vgcfgrestore to get it to read the rest of the info will be used.

Oh, thank you.=C2=A0 =C2=A0I did see that things got somewhat different on = the
target drive when I did "pvcreate --uuid --restorefile."=C2=A0 I = got paranoid
when I saw that, and re-copied the ddrestore file back to the target
drive before I did anything else.=C2=A0 =C2=A0Should I do "pvcreate --= uuid
--norestorefile," instead?=C2=A0 Then, once it is back in the machine,= do the
pvscan and vgcfgrestore, and expect good things?



> I have seen some weird disk controller failures that appeared to zero = out
> the first bit of the disk (enough to get the partition table, grub, an= d the
> pv header depending on where the first partition starts).

I APPEAR to have a partition table, containing an NTFS partition, an LVM partiton ( the one that I am concentrating on ) and a Linux partion.=C2=A0 = I
would have thought that it was all LVM, but my memory could easily be
wrong.



> You will need to reinstall grub if this was the bootable disk, since t= here
> were 384 bytes of grub in the sector with the partition table that you= know
> are missing.

Fortunately, this is all data, nothing to do with the boot sequence,
except that the machine will not boot with the missing PV.



Thank you,
Brian

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.= com
https://listman.redhat.com/mailman/listinfo/lin= ux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

--00000000000096389a05cecc9d56-- --===============7977435902000083082== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ --===============7977435902000083082==--