From: Austin S Hemmelgarn <ahferroin7@gmail.com>
To: linux-btrfs <linux-btrfs@vger.kernel.org>
Subject: BTRFS raid6 unmountable after a couple of days of usage.
Date: Tue, 14 Jul 2015 07:49:33 -0400 [thread overview]
Message-ID: <55A4F74D.9000305@gmail.com> (raw)
[-- Attachment #1.1: Type: text/plain, Size: 1930 bytes --]
So, after experiencing this same issue multiple times (on almost a dozen different kernel versions since 4.0) and ruling out the possibility of it being caused by my hardware (or at least, the RAM, SATA controller and disk drives themselves), I've decided to report it here.
The general symptom is that raid6 profile filesystems that I have are working fine for multiple weeks, until I either reboot or otherwise try to remount them, at which point the system refuses to mount them.
I'm currently using btrfs-progs v4.1 with kernel 4.1.2, although I've been seeing this with versions of both since 4.0.
Output of 'btrfs fi show' for the most recent fs that I had this issue with:
Label: 'altroot' uuid: 86eef6b9-febe-4350-a316-4cb00c40bbc5
Total devices 4 FS bytes used 9.70GiB
devid 1 size 24.00GiB used 6.03GiB path /dev/mapper/vg-altroot.0
devid 2 size 24.00GiB used 6.01GiB path /dev/mapper/vg-altroot.1
devid 3 size 24.00GiB used 6.01GiB path /dev/mapper/vg-altroot.2
devid 4 size 24.00GiB used 6.01GiB path /dev/mapper/vg-altroot.3
btrfs-progs v4.1
Each of the individual LVS that are in the FS is just a flat chunk of space on a separate disk from the others.
The FS itself passes btrfs check just fine (no reported errors, exit value of 0), but the kernel refuses to mount it with the message 'open_ctree failed'.
I've run btrfs chunk recover and attached the output from that.
Here's a link to an image from 'btrfs image -c9 -w': https://www.dropbox.com/s/pl7gs305ej65u9q/altroot.btrfs.img?dl=0
(That link will expire in 30 days, let me know if you need access to it beyond that).
The filesystems in question all see relatively light but consistent usage as targets for receiving daily incremental snapshots for on-system backups (and because I know someone will mention it, yes, I do have other backups of the data, these are just my online backups).
[-- Attachment #1.2: chunk-recover-output.txt --]
[-- Type: text/plain, Size: 13773 bytes --]
All Devices:
Device: id = 4, name = /dev/mapper/vg-altroot.3
Device: id = 3, name = /dev/mapper/vg-altroot.2
Device: id = 2, name = /dev/mapper/vg-altroot.1
Device: id = 1, name = /dev/vg/altroot.0
DEVICE SCAN RESULT:
Filesystem Information:
sectorsize: 4096
leafsize: 16384
tree root generation: 26
chunk root generation: 11
All Devices:
Device: id = 4, name = /dev/mapper/vg-altroot.3
Device: id = 3, name = /dev/mapper/vg-altroot.2
Device: id = 2, name = /dev/mapper/vg-altroot.1
Device: id = 1, name = /dev/vg/altroot.0
All Block Groups:
Block Group: start = 0, len = 4194304, flag = 2
Block Group: start = 4194304, len = 8388608, flag = 4
Block Group: start = 12582912, len = 8388608, flag = 1
Block Group: start = 20971520, len = 16777216, flag = 102
Block Group: start = 37748736, len = 2147483648, flag = 104
Block Group: start = 2185232384, len = 2147483648, flag = 101
Block Group: start = 4332716032, len = 2147483648, flag = 101
Block Group: start = 6480199680, len = 2147483648, flag = 101
Block Group: start = 8627683328, len = 2147483648, flag = 101
Block Group: start = 10775166976, len = 2147483648, flag = 101
All Chunks:
Chunk: start = 0, len = 4194304, type = 2, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 1, offset = 0
Chunk: start = 4194304, len = 8388608, type = 4, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 1, offset = 4194304
Chunk: start = 12582912, len = 8388608, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 1, offset = 12582912
Chunk: start = 20971520, len = 16777216, type = 102, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 4, offset = 1048576
[ 1] Stripe: devid = 3, offset = 1048576
[ 2] Stripe: devid = 2, offset = 1048576
[ 3] Stripe: devid = 1, offset = 20971520
Chunk: start = 37748736, len = 2147483648, type = 104, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 4, offset = 9437184
[ 1] Stripe: devid = 3, offset = 9437184
[ 2] Stripe: devid = 2, offset = 9437184
[ 3] Stripe: devid = 1, offset = 29360128
Chunk: start = 2185232384, len = 2147483648, type = 101, num_stripes =
4
Stripes list:
[ 0] Stripe: devid = 4, offset = 1083179008
[ 1] Stripe: devid = 3, offset = 1083179008
[ 2] Stripe: devid = 2, offset = 1083179008
[ 3] Stripe: devid = 1, offset = 1103101952
Chunk: start = 4332716032, len = 2147483648, type = 101, num_stripes =
4
Stripes list:
[ 0] Stripe: devid = 2, offset = 2156920832
[ 1] Stripe: devid = 3, offset = 2156920832
[ 2] Stripe: devid = 4, offset = 2156920832
[ 3] Stripe: devid = 1, offset = 2176843776
Chunk: start = 6480199680, len = 2147483648, type = 101, num_stripes =
4
Stripes list:
[ 0] Stripe: devid = 2, offset = 3230662656
[ 1] Stripe: devid = 3, offset = 3230662656
[ 2] Stripe: devid = 4, offset = 3230662656
[ 3] Stripe: devid = 1, offset = 3250585600
Chunk: start = 8627683328, len = 2147483648, type = 101, num_stripes =
4
Stripes list:
[ 0] Stripe: devid = 2, offset = 4304404480
[ 1] Stripe: devid = 3, offset = 4304404480
[ 2] Stripe: devid = 4, offset = 4304404480
[ 3] Stripe: devid = 1, offset = 4324327424
Chunk: start = 10775166976, len = 2147483648, type = 101, num_stripes
= 4
Stripes list:
[ 0] Stripe: devid = 2, offset = 5378146304
[ 1] Stripe: devid = 3, offset = 5378146304
[ 2] Stripe: devid = 4, offset = 5378146304
[ 3] Stripe: devid = 1, offset = 5398069248
All Device Extents:
Device extent: devid = 1, start = 0, len = 4194304, chunk offset = 0
Device extent: devid = 1, start = 4194304, len = 8388608, chunk offset
= 4194304
Device extent: devid = 1, start = 12582912, len = 8388608, chunk
offset = 12582912
Device extent: devid = 1, start = 20971520, len = 8388608, chunk
offset = 20971520
Device extent: devid = 1, start = 29360128, len = 1073741824, chunk
offset = 37748736
Device extent: devid = 1, start = 1103101952, len = 1073741824, chunk
offset = 2185232384
Device extent: devid = 1, start = 2176843776, len = 1073741824, chunk
offset = 4332716032
Device extent: devid = 1, start = 3250585600, len = 1073741824, chunk
offset = 6480199680
Device extent: devid = 1, start = 4324327424, len = 1073741824, chunk
offset = 8627683328
Device extent: devid = 1, start = 5398069248, len = 1073741824, chunk
offset = 10775166976
Device extent: devid = 2, start = 1048576, len = 8388608, chunk offset
= 20971520
Device extent: devid = 2, start = 9437184, len = 1073741824, chunk
offset = 37748736
Device extent: devid = 2, start = 1083179008, len = 1073741824, chunk
offset = 2185232384
Device extent: devid = 2, start = 2156920832, len = 1073741824, chunk
offset = 4332716032
Device extent: devid = 2, start = 3230662656, len = 1073741824, chunk
offset = 6480199680
Device extent: devid = 2, start = 4304404480, len = 1073741824, chunk
offset = 8627683328
Device extent: devid = 2, start = 5378146304, len = 1073741824, chunk
offset = 10775166976
Device extent: devid = 3, start = 1048576, len = 8388608, chunk offset
= 20971520
Device extent: devid = 3, start = 9437184, len = 1073741824, chunk
offset = 37748736
Device extent: devid = 3, start = 1083179008, len = 1073741824, chunk
offset = 2185232384
Device extent: devid = 3, start = 2156920832, len = 1073741824, chunk
offset = 4332716032
Device extent: devid = 3, start = 3230662656, len = 1073741824, chunk
offset = 6480199680
Device extent: devid = 3, start = 4304404480, len = 1073741824, chunk
offset = 8627683328
Device extent: devid = 3, start = 5378146304, len = 1073741824, chunk
offset = 10775166976
Device extent: devid = 4, start = 1048576, len = 8388608, chunk offset
= 20971520
Device extent: devid = 4, start = 9437184, len = 1073741824, chunk
offset = 37748736
Device extent: devid = 4, start = 1083179008, len = 1073741824, chunk
offset = 2185232384
Device extent: devid = 4, start = 2156920832, len = 1073741824, chunk
offset = 4332716032
Device extent: devid = 4, start = 3230662656, len = 1073741824, chunk
offset = 6480199680
Device extent: devid = 4, start = 4304404480, len = 1073741824, chunk
offset = 8627683328
Device extent: devid = 4, start = 5378146304, len = 1073741824, chunk
offset = 10775166976
CHECK RESULT:
Recoverable Chunks:
Chunk: start = 0, len = 4194304, type = 2, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 1, offset = 0
Block Group: start = 0, len = 4194304, flag = 2
Device extent list:
[ 0]Device extent: devid = 1, start = 0, len = 4194304, chunk offset
= 0
Chunk: start = 4194304, len = 8388608, type = 4, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 1, offset = 4194304
Block Group: start = 4194304, len = 8388608, flag = 4
Device extent list:
[ 0]Device extent: devid = 1, start = 4194304, len = 8388608, chunk
offset = 4194304
Chunk: start = 12582912, len = 8388608, type = 1, num_stripes = 1
Stripes list:
[ 0] Stripe: devid = 1, offset = 12582912
Block Group: start = 12582912, len = 8388608, flag = 1
Device extent list:
[ 0]Device extent: devid = 1, start = 12582912, len = 8388608, chunk
offset = 12582912
Chunk: start = 20971520, len = 16777216, type = 102, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 4, offset = 1048576
[ 1] Stripe: devid = 3, offset = 1048576
[ 2] Stripe: devid = 2, offset = 1048576
[ 3] Stripe: devid = 1, offset = 20971520
Block Group: start = 20971520, len = 16777216, flag = 102
Device extent list:
[ 0]Device extent: devid = 1, start = 20971520, len = 8388608, chunk
offset = 20971520
[ 1]Device extent: devid = 2, start = 1048576, len = 8388608, chunk
offset = 20971520
[ 2]Device extent: devid = 3, start = 1048576, len = 8388608, chunk
offset = 20971520
[ 3]Device extent: devid = 4, start = 1048576, len = 8388608, chunk
offset = 20971520
Chunk: start = 37748736, len = 2147483648, type = 104, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 4, offset = 9437184
[ 1] Stripe: devid = 3, offset = 9437184
[ 2] Stripe: devid = 2, offset = 9437184
[ 3] Stripe: devid = 1, offset = 29360128
Block Group: start = 37748736, len = 2147483648, flag = 104
Device extent list:
[ 0]Device extent: devid = 1, start = 29360128, len = 1073741824,
chunk offset = 37748736
[ 1]Device extent: devid = 2, start = 9437184, len = 1073741824,
chunk offset = 37748736
[ 2]Device extent: devid = 3, start = 9437184, len = 1073741824,
chunk offset = 37748736
[ 3]Device extent: devid = 4, start = 9437184, len = 1073741824,
chunk offset = 37748736
Chunk: start = 2185232384, len = 2147483648, type = 101, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 4, offset = 1083179008
[ 1] Stripe: devid = 3, offset = 1083179008
[ 2] Stripe: devid = 2, offset = 1083179008
[ 3] Stripe: devid = 1, offset = 1103101952
Block Group: start = 2185232384, len = 2147483648, flag = 101
Device extent list:
[ 0]Device extent: devid = 1, start = 1103101952, len = 1073741824,
chunk offset = 2185232384
[ 1]Device extent: devid = 2, start = 1083179008, len = 1073741824,
chunk offset = 2185232384
[ 2]Device extent: devid = 3, start = 1083179008, len = 1073741824,
chunk offset = 2185232384
[ 3]Device extent: devid = 4, start = 1083179008, len = 1073741824,
chunk offset = 2185232384
Chunk: start = 4332716032, len = 2147483648, type = 101, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 2, offset = 2156920832
[ 1] Stripe: devid = 3, offset = 2156920832
[ 2] Stripe: devid = 4, offset = 2156920832
[ 3] Stripe: devid = 1, offset = 2176843776
Block Group: start = 4332716032, len = 2147483648, flag = 101
Device extent list:
[ 0]Device extent: devid = 1, start = 2176843776, len = 1073741824,
chunk offset = 4332716032
[ 1]Device extent: devid = 4, start = 2156920832, len = 1073741824,
chunk offset = 4332716032
[ 2]Device extent: devid = 3, start = 2156920832, len = 1073741824,
chunk offset = 4332716032
[ 3]Device extent: devid = 2, start = 2156920832, len = 1073741824,
chunk offset = 4332716032
Chunk: start = 6480199680, len = 2147483648, type = 101, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 2, offset = 3230662656
[ 1] Stripe: devid = 3, offset = 3230662656
[ 2] Stripe: devid = 4, offset = 3230662656
[ 3] Stripe: devid = 1, offset = 3250585600
Block Group: start = 6480199680, len = 2147483648, flag = 101
Device extent list:
[ 0]Device extent: devid = 1, start = 3250585600, len = 1073741824,
chunk offset = 6480199680
[ 1]Device extent: devid = 4, start = 3230662656, len = 1073741824,
chunk offset = 6480199680
[ 2]Device extent: devid = 3, start = 3230662656, len = 1073741824,
chunk offset = 6480199680
[ 3]Device extent: devid = 2, start = 3230662656, len = 1073741824,
chunk offset = 6480199680
Chunk: start = 8627683328, len = 2147483648, type = 101, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 2, offset = 4304404480
[ 1] Stripe: devid = 3, offset = 4304404480
[ 2] Stripe: devid = 4, offset = 4304404480
[ 3] Stripe: devid = 1, offset = 4324327424
Block Group: start = 8627683328, len = 2147483648, flag = 101
Device extent list:
[ 0]Device extent: devid = 1, start = 4324327424, len = 1073741824,
chunk offset = 8627683328
[ 1]Device extent: devid = 4, start = 4304404480, len = 1073741824,
chunk offset = 8627683328
[ 2]Device extent: devid = 3, start = 4304404480, len = 1073741824,
chunk offset = 8627683328
[ 3]Device extent: devid = 2, start = 4304404480, len = 1073741824,
chunk offset = 8627683328
Chunk: start = 10775166976, len = 2147483648, type = 101, num_stripes = 4
Stripes list:
[ 0] Stripe: devid = 2, offset = 5378146304
[ 1] Stripe: devid = 3, offset = 5378146304
[ 2] Stripe: devid = 4, offset = 5378146304
[ 3] Stripe: devid = 1, offset = 5398069248
Block Group: start = 10775166976, len = 2147483648, flag = 101
Device extent list:
[ 0]Device extent: devid = 1, start = 5398069248, len = 1073741824,
chunk offset = 10775166976
[ 1]Device extent: devid = 4, start = 5378146304, len = 1073741824,
chunk offset = 10775166976
[ 2]Device extent: devid = 3, start = 5378146304, len = 1073741824,
chunk offset = 10775166976
[ 3]Device extent: devid = 2, start = 5378146304, len = 1073741824,
chunk offset = 10775166976
Unrecoverable Chunks:
Total Chunks: 10
Recoverable: 10
Unrecoverable: 0
Orphan Block Groups:
Orphan Device Extents:
Check chunks successfully with no orphans
Recover the chunk tree successfully.
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]
next reply other threads:[~2015-07-14 11:49 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-14 11:49 Austin S Hemmelgarn [this message]
2015-07-14 13:25 ` BTRFS raid6 unmountable after a couple of days of usage Austin S Hemmelgarn
2015-07-14 23:20 ` Chris Murphy
2015-07-15 11:07 ` Austin S Hemmelgarn
2015-07-15 15:45 ` Chris Murphy
2015-07-15 16:15 ` Hugo Mills
2015-07-15 21:29 ` Chris Murphy
2015-07-16 11:41 ` Austin S Hemmelgarn
2015-08-25 18:12 ` Austin S Hemmelgarn
2015-07-16 11:49 ` Austin S Hemmelgarn
2015-08-25 18:09 ` Austin S Hemmelgarn
[not found] <55A5B13C.6060009@spotprint.com.au>
2015-07-15 6:53 ` Ryan Bourne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55A4F74D.9000305@gmail.com \
--to=ahferroin7@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).