raid10 drive replacement

raid10 drive replacement

[Vincent.]

Hi !

I have a faulty drive in my raid10 and want it to be replaced.
Working drive are xvd[bef] and replacement drive is xvdc.

When I mount my drive in RW:
#mount -odegraded /dev/xvdb /tank
#dmesg -c
[ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
devid 1 transid 551398 /dev/xvdb
[ 6207.327357] btrfs: allowing degraded mounts
[ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
[ 6219.785929] btrfs: open_ctree failed

When I mount my drive in RO:
#mount -odegraded,ro /dev/xvdb /tank
#btrfs filesystem show
Label: none  uuid: 728ef4d8-928c-435c-b707-f71c459e1520
Total devices 4 FS bytes used 4.70TiB
devid    1 size 2.73TiB used 2.73TiB path /dev/xvdb
devid    2 size 2.73TiB used 2.73TiB path
devid    3 size 2.73TiB used 2.73TiB path /dev/xvde
devid    4 size 2.73TiB used 2.73TiB path /dev/xvdf

Btrfs v3.12

Of course, because my mount is in RO, i can't add device and do a balance:
#btrfs device add /dev/xvdc /tank
ERROR: error adding the device '/dev/xvdc' - Read-only file system

Neither a "replace without disk":
#btrfs replace start -Br 2 /dev/xvdc /tank
ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
system, no error

FYI I'm on ubuntu 14.04 with btrfs 3.12

Any idea ?
Thanks !
--
fensoft

Re: raid10 drive replacement

[Suman Chakravartula]

On 2014-10-17 18:47, Vincent. wrote:
> Hi !
> 
> I have a faulty drive in my raid10 and want it to be replaced.
> Working drive are xvd[bef] and replacement drive is xvdc.
> 
This is something I ran into the other day. Key difference is that I was 
running 3.17.1 kernel and 3.16 btrfs-progs

> When I mount my drive in RW:
> #mount -odegraded /dev/xvdb /tank
> #dmesg -c
> [ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
> devid 1 transid 551398 /dev/xvdb
> [ 6207.327357] btrfs: allowing degraded mounts
> [ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
> 0, corrupt 0, gen 0
> [ 6219.703606] Btrfs: too many missing devices, writeable mount is not 
> allowed
> [ 6219.785929] btrfs: open_ctree failed
> 

In my case, I was able to rw mount. May be update btrfs-progs and retry?

> When I mount my drive in RO:
> #mount -odegraded,ro /dev/xvdb /tank
> #btrfs filesystem show
> Label: none  uuid: 728ef4d8-928c-435c-b707-f71c459e1520
> Total devices 4 FS bytes used 4.70TiB
> devid    1 size 2.73TiB used 2.73TiB path /dev/xvdb
> devid    2 size 2.73TiB used 2.73TiB path
> devid    3 size 2.73TiB used 2.73TiB path /dev/xvde
> devid    4 size 2.73TiB used 2.73TiB path /dev/xvdf
> 
> Btrfs v3.12
> 
> Of course, because my mount is in RO, i can't add device and do a 
> balance:
> #btrfs device add /dev/xvdc /tank
> ERROR: error adding the device '/dev/xvdc' - Read-only file system
> 
> Neither a "replace without disk":
> #btrfs replace start -Br 2 /dev/xvdc /tank
> ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
> system, no error
> 
Because my mount was rw, replace worked.

> FYI I'm on ubuntu 14.04 with btrfs 3.12
> 
> Any idea ?
> Thanks !
> --
> fensoft
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" 
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid10 drive replacement

[Anand Jain]

somethings aren't matching well. the issue is..
----
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not 
allowed
----
But per Vincent only xvdc is missing in a raid10 (both data and metadata 
are raid10 ?)


Anand


On 10/18/14 10:02, Suman Chakravartula wrote:
>
> On 2014-10-17 18:47, Vincent. wrote:
>> Hi !
>>
>> I have a faulty drive in my raid10 and want it to be replaced.
>> Working drive are xvd[bef] and replacement drive is xvdc.
>>
> This is something I ran into the other day. Key difference is that I was
> running 3.17.1 kernel and 3.16 btrfs-progs
>
>> When I mount my drive in RW:
>> #mount -odegraded /dev/xvdb /tank
>> #dmesg -c
>> [ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
>> devid 1 transid 551398 /dev/xvdb
>> [ 6207.327357] btrfs: allowing degraded mounts
>> [ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
>> 0, corrupt 0, gen 0
>> [ 6219.703606] Btrfs: too many missing devices, writeable mount is not
>> allowed
>> [ 6219.785929] btrfs: open_ctree failed
>>
>
> In my case, I was able to rw mount. May be update btrfs-progs and retry?
>
>> When I mount my drive in RO:
>> #mount -odegraded,ro /dev/xvdb /tank
>> #btrfs filesystem show
>> Label: none  uuid: 728ef4d8-928c-435c-b707-f71c459e1520
>> Total devices 4 FS bytes used 4.70TiB
>> devid    1 size 2.73TiB used 2.73TiB path /dev/xvdb
>> devid    2 size 2.73TiB used 2.73TiB path
>> devid    3 size 2.73TiB used 2.73TiB path /dev/xvde
>> devid    4 size 2.73TiB used 2.73TiB path /dev/xvdf
>>
>> Btrfs v3.12
>>
>> Of course, because my mount is in RO, i can't add device and do a
>> balance:
>> #btrfs device add /dev/xvdc /tank
>> ERROR: error adding the device '/dev/xvdc' - Read-only file system
>>
>> Neither a "replace without disk":
>> #btrfs replace start -Br 2 /dev/xvdc /tank
>> ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
>> system, no error
>>
> Because my mount was rw, replace worked.
>
>> FYI I'm on ubuntu 14.04 with btrfs 3.12
>>
>> Any idea ?
>> Thanks !
>> --
>> fensoft
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid10 drive replacement

[Vincent.]

i've upgraded to kernel 3.17.0 to update btrfs.ko and it's still not
mountable in -odegraded (-odegraded,ro works) with this error:
[  361.828752] BTRFS info (device xvde): allowing degraded mounts
[  361.984823] BTRFS: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[  374.253555] BTRFS: too many missing devices, writeable mount is not allowed
[  374.332474] BTRFS: open_ctree failed

and, yes, data + metadata are raid10
--
Vincent


On Sat, Oct 18, 2014 at 6:37 AM, Anand Jain <Anand.Jain@oracle.com> wrote:
>
> somethings aren't matching well. the issue is..
> ----
> [ 6219.703606] Btrfs: too many missing devices, writeable mount is not
> allowed
> ----
> But per Vincent only xvdc is missing in a raid10 (both data and metadata are
> raid10 ?)
>
>
> Anand
>
>
>
> On 10/18/14 10:02, Suman Chakravartula wrote:
>>
>>
>> On 2014-10-17 18:47, Vincent. wrote:
>>>
>>> Hi !
>>>
>>> I have a faulty drive in my raid10 and want it to be replaced.
>>> Working drive are xvd[bef] and replacement drive is xvdc.
>>>
>> This is something I ran into the other day. Key difference is that I was
>> running 3.17.1 kernel and 3.16 btrfs-progs
>>
>>> When I mount my drive in RW:
>>> #mount -odegraded /dev/xvdb /tank
>>> #dmesg -c
>>> [ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
>>> devid 1 transid 551398 /dev/xvdb
>>> [ 6207.327357] btrfs: allowing degraded mounts
>>> [ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
>>> 0, corrupt 0, gen 0
>>> [ 6219.703606] Btrfs: too many missing devices, writeable mount is not
>>> allowed
>>> [ 6219.785929] btrfs: open_ctree failed
>>>
>>
>> In my case, I was able to rw mount. May be update btrfs-progs and retry?
>>
>>> When I mount my drive in RO:
>>> #mount -odegraded,ro /dev/xvdb /tank
>>> #btrfs filesystem show
>>> Label: none  uuid: 728ef4d8-928c-435c-b707-f71c459e1520
>>> Total devices 4 FS bytes used 4.70TiB
>>> devid    1 size 2.73TiB used 2.73TiB path /dev/xvdb
>>> devid    2 size 2.73TiB used 2.73TiB path
>>> devid    3 size 2.73TiB used 2.73TiB path /dev/xvde
>>> devid    4 size 2.73TiB used 2.73TiB path /dev/xvdf
>>>
>>> Btrfs v3.12
>>>
>>> Of course, because my mount is in RO, i can't add device and do a
>>> balance:
>>> #btrfs device add /dev/xvdc /tank
>>> ERROR: error adding the device '/dev/xvdc' - Read-only file system
>>>
>>> Neither a "replace without disk":
>>> #btrfs replace start -Br 2 /dev/xvdc /tank
>>> ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
>>> system, no error
>>>
>> Because my mount was rw, replace worked.
>>
>>> FYI I'm on ubuntu 14.04 with btrfs 3.12
>>>
>>> Any idea ?
>>> Thanks !
>>> --
>>> fensoft
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid10 drive replacement

[Duncan]

Suman Chakravartula posted on Fri, 17 Oct 2014 19:02:49 -0700 as
excerpted:


> On 2014-10-17 18:47, Vincent. wrote:
>> Hi !
>> 
>> I have a faulty drive in my raid10 and want it to be replaced. Working
>> drive are xvd[bef] and replacement drive is xvdc.
>> 
> This is something I ran into the other day. Key difference is that I was
> running 3.17.1 kernel and 3.16 btrfs-progs
> 
>> When I mount my drive in RW:
>> #mount -odegraded /dev/xvdb /tank
>> [ 6219.703606] Btrfs: too many missing devices, writeable mount
>> is not allowed
>> 
>> 
> In my case, I was able to rw mount. May be update btrfs-progs and retry?
>> 
> Because my mount was rw, replace worked.
> 
>> FYI I'm on ubuntu 14.04 with btrfs 3.12

I'd suggest updating both btrfs-progs (as Suman suggested) and the kernel 
(the version of which you don't mention).

There was a period when btrfs was too strict in what it would allow to 
mount writable, thus not allowing a device replace to fix the problem.  
It would appear ubuntu 14.4's versions fell in that period.

For kernel, given some recent bugs, you want the latest stable 3.16, 
definitely > 3.16.2 as that contained a vital btrfs bugfix.  Current 3.17 
stable has a different bug that didn't affect 3.16, patch pending but not 
in 3.17, or in 3.18-rc yet, but 3.16.4 (I believe the latest) should be 
good.  You can also try the latest long-term-stable 3.14.x as it's pretty 
stable btrfs-wise, but 3.14 might have still been in the can't-mount-
writable period and I'm not sure they backported that fix to stable-
series.  But 3.16.4+ should be great.

Because btrfs is evolving so fast, in general you'll want to run a 
current kernel and keep up with the list to know about any current bugs 
(like the one currently affecting 3.17 and the 3.18-rcs, and the earlier 
one affecting the entire 3.15 series and 3.16 before 3.16.2).  Most 
distros run somewhat outdated kernels so sticking to a distro kernel 
doesn't always work so well.

btrfs-progs userspace isn't quite so vital to keep current as the kernel, 
but you still want to keep at least reasonably current.  3.12 is getting 
a bit long in the tooth, now, and I'd recommend at least 3.14.2, with 
3.16.1 preferred.  (A 3.17 is in rc ATM but not yet released.)

And (you may not need this but we get people all the time for whom it's a 
bit late so it's worth repeating) of course, because btrfs /is/ evolving 
so fast and isn't yet completely stable, the general rule that if you 
value it you have backups for it that applies even to mature and stable 
filesystems such as ext3 and ext4, applies even more to btrfs.   If 
something goes wrong and you don't have a backup, no matter HOW much you /
claim/ to value that data in hindsight, your actions say you simply 
didn't value it enough to ensure yourself against such possibilities by 
keeping backups, which means you by your actions you considered it throw-
away data if something /did/ go wrong.  Also, be aware that a backup 
that's not tested can't be considered a backup at all, because you don't 
know for sure that you can recover from it.  And RAID isn't a substitute 
for backups.

So if you value your data, have (tested) backups!  Not doing so simply 
means you don't value that data, no matter what you /claim/.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: raid10 drive replacement

[Anand Jain]

  still somethings aren't matching, it could be that
  underlying group profile might have changed without
  your notice (yes in some circumstance it could).
  can you show 'btrfs fi df <mnt>'

Anand


On 10/19/14 07:12, Vincent. wrote:
> i've upgraded to kernel 3.17.0 to update btrfs.ko and it's still not
> mountable in -odegraded (-odegraded,ro works) with this error:
> [  361.828752] BTRFS info (device xvde): allowing degraded mounts
> [  361.984823] BTRFS: bdev (null) errs: wr 15211054, rd 3038899, flush
> 0, corrupt 0, gen 0
> [  374.253555] BTRFS: too many missing devices, writeable mount is not allowed
> [  374.332474] BTRFS: open_ctree failed
>
> and, yes, data + metadata are raid10
> --
> Vincent
>
>
> On Sat, Oct 18, 2014 at 6:37 AM, Anand Jain <Anand.Jain@oracle.com> wrote:
>>
>> somethings aren't matching well. the issue is..
>> ----
>> [ 6219.703606] Btrfs: too many missing devices, writeable mount is not
>> allowed
>> ----
>> But per Vincent only xvdc is missing in a raid10 (both data and metadata are
>> raid10 ?)
>>
>>
>> Anand
>>
>>
>>
>> On 10/18/14 10:02, Suman Chakravartula wrote:
>>>
>>>
>>> On 2014-10-17 18:47, Vincent. wrote:
>>>>
>>>> Hi !
>>>>
>>>> I have a faulty drive in my raid10 and want it to be replaced.
>>>> Working drive are xvd[bef] and replacement drive is xvdc.
>>>>
>>> This is something I ran into the other day. Key difference is that I was
>>> running 3.17.1 kernel and 3.16 btrfs-progs
>>>
>>>> When I mount my drive in RW:
>>>> #mount -odegraded /dev/xvdb /tank
>>>> #dmesg -c
>>>> [ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
>>>> devid 1 transid 551398 /dev/xvdb
>>>> [ 6207.327357] btrfs: allowing degraded mounts
>>>> [ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
>>>> 0, corrupt 0, gen 0
>>>> [ 6219.703606] Btrfs: too many missing devices, writeable mount is not
>>>> allowed
>>>> [ 6219.785929] btrfs: open_ctree failed
>>>>
>>>
>>> In my case, I was able to rw mount. May be update btrfs-progs and retry?
>>>
>>>> When I mount my drive in RO:
>>>> #mount -odegraded,ro /dev/xvdb /tank
>>>> #btrfs filesystem show
>>>> Label: none  uuid: 728ef4d8-928c-435c-b707-f71c459e1520
>>>> Total devices 4 FS bytes used 4.70TiB
>>>> devid    1 size 2.73TiB used 2.73TiB path /dev/xvdb
>>>> devid    2 size 2.73TiB used 2.73TiB path
>>>> devid    3 size 2.73TiB used 2.73TiB path /dev/xvde
>>>> devid    4 size 2.73TiB used 2.73TiB path /dev/xvdf
>>>>
>>>> Btrfs v3.12
>>>>
>>>> Of course, because my mount is in RO, i can't add device and do a
>>>> balance:
>>>> #btrfs device add /dev/xvdc /tank
>>>> ERROR: error adding the device '/dev/xvdc' - Read-only file system
>>>>
>>>> Neither a "replace without disk":
>>>> #btrfs replace start -Br 2 /dev/xvdc /tank
>>>> ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
>>>> system, no error
>>>>
>>> Because my mount was rw, replace worked.
>>>
>>>> FYI I'm on ubuntu 14.04 with btrfs 3.12
>>>>
>>>> Any idea ?
>>>> Thanks !
>>>> --
>>>> fensoft
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>