[linux-lvm] Unable to find device with UUID XXX.

[linux-lvm] Unable to find device with UUID XXX.

[Brian McCullough]

I have an issue that I hope that the group can save me from.

I have found the Thin discussions interesting, but I am still doing the
Thick thing.


OK, I had a drive that started to cause problems, would not boot without
fsck, etc., and decided to replace it using pvmove.

I made myself a CD with SystemRescueCD ( sp? ) and booted the system
with that.

I attached the new drive to the system before I booted.  It had no PVs
on it, but a set of partitions ( GPT table ) ready to be assigned.

( skipping a couple of steps )

I ran pvmove for about half of the PVs on the drive that was failing,
sending them to the new drive.  All seemed to be well.

THEN, the data cable on the old drive ( USB 3 ) was disturbed, and
pvmove stopped, complaining that it couldn't see the drive.  OK, I can
fix this.  I plugged the cable back in, but things didn't seem to be any
better.

I stopped the system, and rebooted.

I have rebooted several times, with not completely consistent results.

A couple of times it stopped when it said that it was loading the MD
drivers.

When I seem to have got the system booted properly, I do a pvs and it
complains about missing UUIDs.

I did a "pvscan -u" and thought that I was seeing the UUIDs in the
bottom part of the display, after it had complained about the missing
ones, but I think that I was just seeing what it wanted to see, not what
it was actually finding.

At this point, I seem to be dead in the water, and can't resume the
pvmove process.

Can you guide me back to the light, and help me finish what I started to
do, so that I can get this machine back to work?


Thank you,
Brian

Re: [linux-lvm] Unable to find device with UUID XXX.

[Marian Csontos]

On 05/18/2016 02:22 AM, Brian McCullough wrote:
> I have an issue that I hope that the group can save me from.
>
> I have found the Thin discussions interesting, but I am still doing the
> Thick thing.
>
>
> OK, I had a drive that started to cause problems, would not boot without
> fsck, etc., and decided to replace it using pvmove.
>
> I made myself a CD with SystemRescueCD ( sp? ) and booted the system
> with that.
>
> I attached the new drive to the system before I booted.  It had no PVs
> on it, but a set of partitions ( GPT table ) ready to be assigned.
>
> ( skipping a couple of steps )
>
> I ran pvmove for about half of the PVs on the drive that was failing,
> sending them to the new drive.  All seemed to be well.
>
> THEN, the data cable on the old drive ( USB 3 ) was disturbed, and
> pvmove stopped, complaining that it couldn't see the drive.  OK, I can
> fix this.  I plugged the cable back in, but things didn't seem to be any
> better.

BTW, pvmove is not a data rescue tool. It does not keep persistent log 
and it is necessary to resume synchronization between drives from 
scratch on reboot.

Use ddrescue to get data from disks with errors.

>
> I stopped the system, and rebooted.
>
> I have rebooted several times, with not completely consistent results.
>
> A couple of times it stopped when it said that it was loading the MD
> drivers.

We need to know more to help.

Are you using  md raid? If so is not using mdadm to replce and 
synchronize the drive better?

What distribution, version of LVM and kernel?

`lsblk` output would help us to see what's there.
`blkid` output would help us to see other bits and pieces.
Also `vgs`, `lvs -avo+devices`, `pvs` output, too, please.


>
> When I seem to have got the system booted properly, I do a pvs and it
> complains about missing UUIDs.

My guess is some of the PVs (those on the old drive) were marked missing 
and would have to be readded to the VG using `vgextend --restoremissing` 
before you can continue.

Another possibility is, is lvmetad enabled and running?
If so it may be possible, there are some duplicates there, and older 
versions of LVM did not always warn about that.

-- Martian

>
> I did a "pvscan -u" and thought that I was seeing the UUIDs in the
> bottom part of the display, after it had complained about the missing
> ones, but I think that I was just seeing what it wanted to see, not what
> it was actually finding.
>
> At this point, I seem to be dead in the water, and can't resume the
> pvmove process.
>
> Can you guide me back to the light, and help me finish what I started to
> do, so that I can get this machine back to work?
>
>
> Thank you,
> Brian
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] Unable to find device with UUID XXX.

[Brian McCullough]

On Wed, May 18, 2016 at 03:19:27PM +0200, Marian Csontos wrote:
> On 05/18/2016 02:22 AM, Brian McCullough wrote:
> >I have an issue that I hope that the group can save me from.
> >
> >I ran pvmove for about half of the PVs on the drive that was failing,
> >sending them to the new drive.  All seemed to be well.
> >
> >THEN, the data cable on the old drive ( USB 3 ) was disturbed, and
> >pvmove stopped, complaining that it couldn't see the drive.  OK, I can
> >fix this.  I plugged the cable back in, but things didn't seem to be any
> >better.
> 
> BTW, pvmove is not a data rescue tool. It does not keep persistent log and
> it is necessary to resume synchronization between drives from scratch on
> reboot.
> 
> Use ddrescue to get data from disks with errors.

Thank you.  I have seen recommendations for ddrescue to other people for
"dead drive" situations.  Didn't think that I was there yet.



> >I stopped the system, and rebooted.
> >
> >I have rebooted several times, with not completely consistent results.
> >
> >A couple of times it stopped when it said that it was loading the MD
> >drivers.
> 
> We need to know more to help.

Yes, I know.  That's why I asked, so that you could tell me what I need
to give you.


> Are you using  md raid? If so is not using mdadm to replce and synchronize
> the drive better?

No.  Not on this machine.  I have used LVM on several machines over the
years, with success.  This is the first time that I have tried some of
the "deeper" commands.


> What distribution, version of LVM and kernel?

On the main machine, or the Rescue CD?


> `lsblk` output would help us to see what's there.
> `blkid` output would help us to see other bits and pieces.
> Also `vgs`, `lvs -avo+devices`, `pvs` output, too, please.

OK, I will work on those.



> >When I seem to have got the system booted properly, I do a pvs and it
> >complains about missing UUIDs.
> 
> My guess is some of the PVs (those on the old drive) were marked missing and
> would have to be readded to the VG using `vgextend --restoremissing` before
> you can continue.

OK. I will get the output from the commands above, before I try this.



> Another possibility is, is lvmetad enabled and running?

No, or at least not as far as I know.  I have never intentionally done
so.


> -- Martian


Brian

Re: [linux-lvm] Unable to find device with UUID XXX.

[Brian McCullough]

On Wed, May 18, 2016 at 10:56:50AM -0400, Brian McCullough wrote:
> On Wed, May 18, 2016 at 03:19:27PM +0200, Marian Csontos wrote:
> > On 05/18/2016 02:22 AM, Brian McCullough wrote:
> > >I have an issue that I hope that the group can save me from.
> > >
> 
> > What distribution, version of LVM and kernel?
> 
> On the main machine, or the Rescue CD?

On the main, dead, machine, it is an Ubuntu 14.04.1 LTS machine with
version 3.13.0-86-generic x86_64 ( AMD 64 ) kernel.

I was able to get enough of the dead machine running to give the other
version numbers.

LVM: 2.02.98(2) (2012-10-15)
Lib: 1.02.77 (2012-10-15)
Driver: 4.22.0



Brian

Re: [linux-lvm] Unable to find device with UUID XXX.

[Brian McCullough]

Oops.  Wrong sender address.

On Wed, May 18, 2016 at 02:39:48PM -0400, Brian McCullough wrote:
> On Wed, May 18, 2016 at 10:56:50AM -0400, Brian McCullough wrote:
> > On Wed, May 18, 2016 at 03:19:27PM +0200, Marian Csontos wrote:
> > > On 05/18/2016 02:22 AM, Brian McCullough wrote:
> > > >I have an issue that I hope that the group can save me from.
> > > >
> > 
> > > What distribution, version of LVM and kernel?
> > 
> > On the main machine, or the Rescue CD?
> 
> On the main, dead, machine, it is an Ubuntu 14.04.1 LTS machine with
> version 3.13.0-86-generic x86_64 ( AMD 64 ) kernel.
> 
> I was able to get enough of the dead machine running to give the other
> version numbers.
> 
> LVM: 2.02.98(2) (2012-10-15)
> Lib: 1.02.77 (2012-10-15)
> Driver: 4.22.0
> 
> 
> 
> Brian
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Unable to find device with UUID XXX.

[Brian McCullough]

This is not looking good.

I have tried to re-attach the old ( probably failing ) drive to the test
machine, and I see this in dmesg:

[ 4566.743139] usb 1-5: new high-speed USB device number 27 using
ehci_hcd
[ 4566.861622] usb 1-5: New USB device found, idVendor=0bc2,
idProduct=3320
[ 4566.861625] usb 1-5: New USB device strings: Mfr=2, Product=3,
SerialNumber=1
[ 4566.861627] usb 1-5: Product: Expansion Desk
[ 4566.861628] usb 1-5: Manufacturer: Seagate
[ 4566.861629] usb 1-5: SerialNumber: 2HC015KJ
[ 4566.862054] scsi8 : usb-storage 1-5:1.0
[ 4567.865494] scsi 8:0:0:0: Direct-Access     Seagate  Expansion Desk
0711 PQ: 0 ANSI: 6
[ 4567.866036] sd 8:0:0:0: Attached scsi generic sg4 type 0
[ 4567.869533] sd 8:0:0:0: [sdd] Spinning up disk..............ready
[ 4578.911055] sd 8:0:0:0: [sdd] 732566645 4096-byte logical blocks:
(3.00 TB/2.72 TiB)
[ 4609.862126] usb 1-5: reset high-speed USB device number 27 using
ehci_hcd
[ 4624.964140] usb 1-5: device descriptor read/64, error -110
[ 4640.167241] usb 1-5: device descriptor read/64, error -110
[ 4640.370138] usb 1-5: reset high-speed USB device number 27 using
ehci_hcd
[ 4655.472298] usb 1-5: device descriptor read/64, error -110
[ 4665.690010] sd 8:0:0:0: [sdd] Test WP failed, assume Write Enabled
[ 4672.640209] sd 8:0:0:0: [sdd] Asking for cache data failed
[ 4672.640211] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[ 4672.643323] sd 8:0:0:0: [sdd] Spinning up
disk.......................................................................................................not
responding...
[ 4773.042814] sd 8:0:0:0: [sdd] 732566645 4096-byte logical blocks:
(3.00 TB/2.72 TiB)
[ 4779.990875] sd 8:0:0:0: [sdd] Test WP failed, assume Write Enabled
[ 4786.941084] sd 8:0:0:0: [sdd] Asking for cache data failed
[ 4786.941086] sd 8:0:0:0: [sdd] Assuming drive cache: write through


This is a Seagate ( as you can see ) packaged USB external drive.

With these errors, would it be worthwhile trying to extract the drive
from the case and try it either internally or in another case?


Thank you,
Brian

Re: [linux-lvm] Unable to find device with UUID XXX.

[Marian Csontos]

On 05/18/2016 09:44 PM, Brian McCullough wrote:
>
> This is not looking good.

No, it is not :-(

...

>
> This is a Seagate ( as you can see ) packaged USB external drive.
>
> With these errors, would it be worthwhile trying to extract the drive
> from the case and try it either internally or in another case?

I have absolutely no idea. In any case it depends on the value of data 
on the drive.

And it is also off topic for this list. Try googling the disk model. 
Then try to get help on a more appropriate place than here, e.g. on 
superuser, or a mailing list (like ddrescue's) where you are more likely 
to get answers from data rescue experts.

Good luck,

-- Martian

>
>
> Thank you,
> Brian
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] Unable to find device with UUID XXX.

[Brian McCullough]

On Mon, May 23, 2016 at 04:03:21PM +0200, Marian Csontos wrote:
> On 05/18/2016 09:44 PM, Brian McCullough wrote:
> >
> >This is not looking good.
> 
> No, it is not :-(
> 
> ...
> 
> And it is also off topic for this list. 

Agreed.  That is why I haven't bothered with any more questions here.
First I have to fix the hardware issue, and then get back to the pvmove,
which is how this started.


> Try googling the disk model. Then
> try to get help on a more appropriate place than here, e.g. on superuser, or
> a mailing list (like ddrescue's) where you are more likely to get answers
> from data rescue experts.

Yes, good suggestions.  I have local people who have had drive issues in
the past and had good success with ddrescue.

I am collecting tools and replacement drives, as well as exploring all
of these avenues.


Thanks again,
Brian



> Good luck,
> 
> -- Martian