* btrfs check --repair now runs in minutes instead of hours? aborting
@ 2017-09-05 1:05 Marc MERLIN
2017-09-05 1:21 ` Qu Wenruo
0 siblings, 1 reply; 12+ messages in thread
From: Marc MERLIN @ 2017-09-05 1:05 UTC (permalink / raw)
To: quwenruo, Lu Fengqi, Chris Mason; +Cc: Btrfs BTRFS, David Sterba
Ok, I don't want to sound like I'm complaining :) but I updated
btrfs-progs to top of tree in git, installed it, and ran it on an 8TiB
filesystem that used to take 12H or so to check.
It finished in maybe 10mn, just 10mn! :)
gargamel:/var/local/src/btrfs-progs# btrfs check --repair /dev/mapper/dshelf1
enabling repair mode
Checking filesystem on /dev/mapper/dshelf1
UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
checking extents
Fixed 0 roots.
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 11674263347200 bytes used, no error found
total csum bytes: 11384482936
total tree bytes: 13704495104
total fs tree bytes: 724729856
total extent tree bytes: 482639872
btree space waste bytes: 1167025205
file data blocks allocated: 12041456693248
referenced 12063146434560
This is great news, but can I trust that the program worked properly and
indeed that my filesystem is fully clean?
Or at this point if I'm not running --mode=lowmem, the regular mode is
really doesn't check much and only lowmem can do a proper check? (even
though it can't fix problems once it finds them)
Thanks
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 1:05 btrfs check --repair now runs in minutes instead of hours? aborting Marc MERLIN
@ 2017-09-05 1:21 ` Qu Wenruo
2017-09-05 2:55 ` Marc MERLIN
0 siblings, 1 reply; 12+ messages in thread
From: Qu Wenruo @ 2017-09-05 1:21 UTC (permalink / raw)
To: Marc MERLIN, quwenruo, Lu Fengqi, Chris Mason; +Cc: Btrfs BTRFS, David Sterba
On 2017年09月05日 09:05, Marc MERLIN wrote:
> Ok, I don't want to sound like I'm complaining :) but I updated
> btrfs-progs to top of tree in git, installed it, and ran it on an 8TiB
> filesystem that used to take 12H or so to check.
How much space allocated for that 8T fs?
If metadata is not that large, 10min is valid.
Here fi df output could help.
And, without --repair, how much time it takes to run?
>
> It finished in maybe 10mn, just 10mn! :)
> gargamel:/var/local/src/btrfs-progs# btrfs check --repair /dev/mapper/dshelf1
> enabling repair mode
> Checking filesystem on /dev/mapper/dshelf1
> UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
> checking extents
> Fixed 0 roots.
> checking free space cache
> cache and super generation don't match, space cache will be invalidated
> checking fs roots
> checking csums
> checking root refs
> found 11674263347200 bytes used, no error found
> total csum bytes: 11384482936
> total tree bytes: 13704495104
> total fs tree bytes: 724729856
> total extent tree bytes: 482639872
> btree space waste bytes: 1167025205
> file data blocks allocated: 12041456693248
> referenced 12063146434560
>
> This is great news, but can I trust that the program worked properly and
> indeed that my filesystem is fully clean?
Normally we rely on btrfs-progs selftest to make sure fsck works.
But maybe we could cross check original mode and lowmem mode.
> Or at this point if I'm not running --mode=lowmem, the regular mode is
> really doesn't check much and only lowmem can do a proper check? (even
> though it can't fix problems once it finds them)
No, original mode is still the core (although code is not that elegant),
please consider lowmem mode as a corner case for guys with small memory
(without swap) but want to check super large fs.
So if it turns out to be a bug, we must fix it.
Thanks,
Qu
>
> Thanks
> Marc
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 1:21 ` Qu Wenruo
@ 2017-09-05 2:55 ` Marc MERLIN
2017-09-05 4:06 ` Marc MERLIN
2017-09-05 8:05 ` Qu Wenruo
0 siblings, 2 replies; 12+ messages in thread
From: Marc MERLIN @ 2017-09-05 2:55 UTC (permalink / raw)
To: Qu Wenruo; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
On Tue, Sep 05, 2017 at 09:21:55AM +0800, Qu Wenruo wrote:
>
>
> On 2017年09月05日 09:05, Marc MERLIN wrote:
> >Ok, I don't want to sound like I'm complaining :) but I updated
> >btrfs-progs to top of tree in git, installed it, and ran it on an 8TiB
> >filesystem that used to take 12H or so to check.
>
> How much space allocated for that 8T fs?
> If metadata is not that large, 10min is valid.
>
> Here fi df output could help.
gargamel:~# btrfs fi df /mnt/btrfs_pool1
Data, single: total=10.60TiB, used=10.54TiB
System, DUP: total=32.00MiB, used=1.19MiB
Metadata, DUP: total=58.00GiB, used=12.69GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
> And, without --repair, how much time it takes to run?
Well, funny that you ask, it's now been running for hours, still waiting...
Just before, I ran lowmem, and it was pretty quick too (didn't time it,
but less than 1h):
gargamel:/var/local/src/btrfs-progs# btrfs check --mode=lowmem
/dev/mapper/dshelf1
Checking filesystem on /dev/mapper/dshelf1
UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 11674263330816 bytes used, no error found
total csum bytes: 11384482936
total tree bytes: 13738737664
total fs tree bytes: 758988800
total extent tree bytes: 482623488
btree space waste bytes: 1171475737
file data blocks allocated: 12888981110784
referenced 12930453286912
Now, this is good news for my filesystem being probably clean (previous
versions of lowmem before my git update found issues that were unclear, but
apparently errors in the code, and this version finds nothing)
But I'm not sure why --repair would be fast, and not --repair would be slow?
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 2:55 ` Marc MERLIN
@ 2017-09-05 4:06 ` Marc MERLIN
2017-09-05 8:05 ` Qu Wenruo
1 sibling, 0 replies; 12+ messages in thread
From: Marc MERLIN @ 2017-09-05 4:06 UTC (permalink / raw)
To: Qu Wenruo; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
Ok, not quite hours, but check takes 88mn, check --repair takes 11mn
gargamel:/var/local/src/btrfs-progs# time btrfs check /dev/mapper/dshelf1
Checking filesystem on /dev/mapper/dshelf1
UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 11674263330816 bytes used, no error found
total csum bytes: 11384482936
total tree bytes: 13704478720
total fs tree bytes: 724729856
total extent tree bytes: 482623488
btree space waste bytes: 1167009013
file data blocks allocated: 12041456693248
referenced 12063146434560
real 88m56.597s
user 2m13.985s
sys 2m7.880s
gargamel:/var/local/src/btrfs-progs# time btrfs check --repair
/dev/mapper/dshelf1
enabling repair mode
Checking filesystem on /dev/mapper/dshelf1
UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
checking extents
Fixed 0 roots.
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 11674263330816 bytes used, no error found
total csum bytes: 11384482936
total tree bytes: 13704478720
total fs tree bytes: 724729856
total extent tree bytes: 482623488
btree space waste bytes: 1167009013
file data blocks allocated: 12041456693248
referenced 12063146434560
real 11m10.499s
user 1m55.067s
sys 1m31.666s
And lowmem is 24mn:
gargamel:/var/local/src/btrfs-progs# time btrfs check --mode=lowmem
/dev/mapper/dshelf1
Checking filesystem on /dev/mapper/dshelf1
UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 11674263363584 bytes used, no error found
total csum bytes: 11384482936
total tree bytes: 13738770432
total fs tree bytes: 758988800
total extent tree bytes: 482656256
btree space waste bytes: 1171508121
file data blocks allocated: 12888981110784
referenced 12930453286912
real 24m20.493s
user 5m45.749s
sys 1m40.204s
Does this make any sense at all that check without --repair is so much
slower than with --repair or lowmem?
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 2:55 ` Marc MERLIN
2017-09-05 4:06 ` Marc MERLIN
@ 2017-09-05 8:05 ` Qu Wenruo
2017-09-05 8:54 ` Duncan
2017-09-05 14:45 ` Marc MERLIN
1 sibling, 2 replies; 12+ messages in thread
From: Qu Wenruo @ 2017-09-05 8:05 UTC (permalink / raw)
To: Marc MERLIN; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
On 2017年09月05日 10:55, Marc MERLIN wrote:
> On Tue, Sep 05, 2017 at 09:21:55AM +0800, Qu Wenruo wrote:
>>
>>
>> On 2017年09月05日 09:05, Marc MERLIN wrote:
>>> Ok, I don't want to sound like I'm complaining :) but I updated
>>> btrfs-progs to top of tree in git, installed it, and ran it on an 8TiB
>>> filesystem that used to take 12H or so to check.
>>
>> How much space allocated for that 8T fs?
>> If metadata is not that large, 10min is valid.
>>
>> Here fi df output could help.
>
> gargamel:~# btrfs fi df /mnt/btrfs_pool1
> Data, single: total\x10.60TiB, used\x10.54TiB
> System, DUP: total2.00MiB, used=1.19MiB
> Metadata, DUP: totalX.00GiB, used\x12.69GiB
Wait for a minute.
Is that .69GiB means 706 MiB? Or my email client/GMX screwed up the
format (again)?
This output format must be changed, at least to 0.69 GiB, or 706 MiB.
I'll fix this first.
> GlobalReserve, single: totalQ2.00MiB, used=0.00B
>
>> And, without --repair, how much time it takes to run?
>
> Well, funny that you ask, it's now been running for hours, still waiting...
>
> Just before, I ran lowmem, and it was pretty quick too (didn't time it,
> but less than 1h):
You mean lowmem is actually FASTER than original mode?
That's very surprising.
Is there any special operation done for that btrfs?
Like offline dedupe or tons of reflinks?
IIRC original mode did a quite slow check for tons of reflink, which may
be related.
BTW, how many subvolumes do you have in the fs?
> gargamel:/var/local/src/btrfs-progs# btrfs check --mode=wmem
> /dev/mapper/dshelf1
> Checking filesystem on /dev/mapper/dshelf1
> UUID: 36f5079e-ca6c-4855-8639-ccb82695c18d
> checking extents
> checking free space cache
> cache and super generation don't match, space cache will be invalidated
> checking fs roots
> checking csums
> checking root refs
> found 11674263330816 bytes used, no error found
> total csum bytes: 11384482936
> total tree bytes: 13738737664
> total fs tree bytes: 758988800
> total extent tree bytes: 482623488
> btree space waste bytes: 1171475737
> file data blocks allocated: 12888981110784
> referenced 12930453286912
>
> Now, this is good news for my filesystem being probably clean (previous
> versions of lowmem before my git update found issues that were unclear, but
> apparently errors in the code, and this version finds nothing)
>
> But I'm not sure why --repair would be fast, and not --repair would be slow?
This looks like a bug. My first guess is related to number of
subvolumes/reflinks, but I'm not sure since I don't have many real-world
btrfs.
I'll take sometime to look into it.
Thanks for the very interesting report,
Qu
>
> Marc
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 8:05 ` Qu Wenruo
@ 2017-09-05 8:54 ` Duncan
2017-09-05 9:06 ` Qu Wenruo
2017-09-05 14:45 ` Marc MERLIN
1 sibling, 1 reply; 12+ messages in thread
From: Duncan @ 2017-09-05 8:54 UTC (permalink / raw)
To: linux-btrfs
Qu Wenruo posted on Tue, 05 Sep 2017 16:05:04 +0800 as excerpted:
> On 2017年09月05日 10:55, Marc MERLIN wrote:
>>
>> gargamel:~# btrfs fi df /mnt/btrfs_pool1
>> Data, single: total\x10.60TiB, used\x10.54TiB
>> System, DUP: total2.00MiB, used=1.19MiB
>> Metadata, DUP: totalX.00GiB, used\x12.69GiB
>
> Wait for a minute.
>
> Is that .69GiB means 706 MiB? Or my email client/GMX screwed up the
> format (again)?
It appears to be your end. Based on the fact that I'm seeing a a bunch
of weird characters in your quote of the message that I didn't see in the
original, I'm guessing it's charset related, very possibly due to the
"equal" sign being an escape character for mime/quoted-printable (tho his
post was text/plain; charset equals utf-8, full 8-bit, so not quoted-
printable encoded at all) and I believe various i18n escapes as well,
with the latter being an issue if the client assumes local charset
despite the utf8 specified in the header.
See if these numbers, copied and reformatted from his post with spaces
inserted either side of the numbers and the equals signs deleted, arrive
any less garbled:
Data, single: total 10.60 TiB, used 10.54 TiB
System, DUP: total 32.00 MiB, used 1.19 MiB
Metadata, DUP: total 58.00 GiB, used 12.69 GiB
GlobalReserve, single: total 512.00 MiB, used 0.00 B
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 8:54 ` Duncan
@ 2017-09-05 9:06 ` Qu Wenruo
2017-09-05 9:35 ` Duncan
0 siblings, 1 reply; 12+ messages in thread
From: Qu Wenruo @ 2017-09-05 9:06 UTC (permalink / raw)
To: Duncan, linux-btrfs
On 2017年09月05日 16:54, Duncan wrote:
> Qu Wenruo posted on Tue, 05 Sep 2017 16:05:04 +0800 as excerpted:
>
>> On 2017年09月05日 10:55, Marc MERLIN wrote:
>>>
>>> gargamel:~# btrfs fi df /mnt/btrfs_pool1
>>> Data, single: total\x10.60TiB, used\x10.54TiB
>>> System, DUP: total2.00MiB, used=1.19MiB
>>> Metadata, DUP: totalX.00GiB, used\x12.69GiB
>>
>> Wait for a minute.
>>
>> Is that .69GiB means 706 MiB? Or my email client/GMX screwed up the
>> format (again)?
>
> It appears to be your end. Based on the fact that I'm seeing a a bunch
> of weird characters in your quote of the message that I didn't see in the
> original, I'm guessing it's charset related, very possibly due to the
> "equal" sign being an escape character for mime/quoted-printable (tho his
> post was text/plain; charset equals utf-8, full 8-bit, so not quoted-
> printable encoded at all) and I believe various i18n escapes as well,
> with the latter being an issue if the client assumes local charset
> despite the utf8 specified in the header.
>
> See if these numbers, copied and reformatted from his post with spaces
> inserted either side of the numbers and the equals signs deleted, arrive
> any less garbled:
>
> Data, single: total 10.60 TiB, used 10.54 TiB
> System, DUP: total 32.00 MiB, used 1.19 MiB
> Metadata, DUP: total 58.00 GiB, used 12.69 GiB
> GlobalReserve, single: total 512.00 MiB, used 0.00 B
>
Thanks a lot for this.
I'd better double check my client setup to avoid such embrassment.
Thanks,
Qu
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 9:06 ` Qu Wenruo
@ 2017-09-05 9:35 ` Duncan
0 siblings, 0 replies; 12+ messages in thread
From: Duncan @ 2017-09-05 9:35 UTC (permalink / raw)
To: linux-btrfs
Qu Wenruo posted on Tue, 05 Sep 2017 17:06:35 +0800 as excerpted:
>> See if these numbers, copied and reformatted from his post with spaces
>> inserted either side of the numbers and the equals signs deleted,
>> arrive any less garbled:
>>
>> Data, single: total 10.60 TiB, used 10.54 TiB System, DUP: total 32.00
>> MiB, used 1.19 MiB Metadata, DUP: total 58.00 GiB, used 12.69 GiB
>> GlobalReserve, single: total 512.00 MiB, used 0.00 B
>>
> Thanks a lot for this.
It worked. =:^) (But thinking about it now, that smiley with an equals
sign probably won't!)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 8:05 ` Qu Wenruo
2017-09-05 8:54 ` Duncan
@ 2017-09-05 14:45 ` Marc MERLIN
2017-09-09 17:44 ` Marc MERLIN
1 sibling, 1 reply; 12+ messages in thread
From: Marc MERLIN @ 2017-09-05 14:45 UTC (permalink / raw)
To: Qu Wenruo; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
On Tue, Sep 05, 2017 at 04:05:04PM +0800, Qu Wenruo wrote:
> > gargamel:~# btrfs fi df /mnt/btrfs_pool1
> > Data, single: total\x10.60TiB, used\x10.54TiB
> > System, DUP: total2.00MiB, used=1.19MiB
> > Metadata, DUP: totalX.00GiB, used\x12.69GiB
>
> Wait for a minute.
>
> Is that .69GiB means 706 MiB? Or my email client/GMX screwed up the format
> (again)?
> This output format must be changed, at least to 0.69 GiB, or 706 MiB.
Email client problem. I see control characters in what you quoted.
Let's try again
gargamel:~# btrfs fi df /mnt/btrfs_pool1
Data, single: total=10.66TiB, used=10.60TiB => 10TB
System, DUP: total=64.00MiB, used=1.20MiB => 1.2MB
Metadata, DUP: total=57.50GiB, used=12.76GiB => 13GB
GlobalReserve, single: total=512.00MiB, used=0.00B => 0
> You mean lowmem is actually FASTER than original mode?
> That's very surprising.
Correct, unless I add --repair and then original mode is 2x faster than
lowmem.
> Is there any special operation done for that btrfs?
> Like offline dedupe or tons of reflinks?
In this case, no.
Note that btrfs check used to take many hours overnight until I did a
git pull of btrfs progs and built the latest from TOT.
> BTW, how many subvolumes do you have in the fs?
gargamel:/mnt/btrfs_pool1# btrfs subvolume list . | wc -l
91
If I remove snapshots for btrfs send and historical 'backups':
gargamel:/mnt/btrfs_pool1# btrfs subvolume list . | grep -Ev '(hourly|daily|weekly|rw|ro)' | wc -l
5
> This looks like a bug. My first guess is related to number of
> subvolumes/reflinks, but I'm not sure since I don't have many real-world
> btrfs.
>
> I'll take sometime to look into it.
>
> Thanks for the very interesting report,
Thanks for having a look :)
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-05 14:45 ` Marc MERLIN
@ 2017-09-09 17:44 ` Marc MERLIN
2017-09-10 6:01 ` Qu Wenruo
0 siblings, 1 reply; 12+ messages in thread
From: Marc MERLIN @ 2017-09-09 17:44 UTC (permalink / raw)
To: Qu Wenruo; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
So, should I assume that btrfs progs git has some issue since there is
no plausible way that a check --repair should be faster than a regular
check?
Thanks,
Marc
On Tue, Sep 05, 2017 at 07:45:25AM -0700, Marc MERLIN wrote:
> On Tue, Sep 05, 2017 at 04:05:04PM +0800, Qu Wenruo wrote:
> > > gargamel:~# btrfs fi df /mnt/btrfs_pool1
> > > Data, single: total\x10.60TiB, used\x10.54TiB
> > > System, DUP: total2.00MiB, used=1.19MiB
> > > Metadata, DUP: totalX.00GiB, used\x12.69GiB
> >
> > Wait for a minute.
> >
> > Is that .69GiB means 706 MiB? Or my email client/GMX screwed up the format
> > (again)?
> > This output format must be changed, at least to 0.69 GiB, or 706 MiB.
>
> Email client problem. I see control characters in what you quoted.
>
> Let's try again
> gargamel:~# btrfs fi df /mnt/btrfs_pool1
> Data, single: total=10.66TiB, used=10.60TiB => 10TB
> System, DUP: total=64.00MiB, used=1.20MiB => 1.2MB
> Metadata, DUP: total=57.50GiB, used=12.76GiB => 13GB
> GlobalReserve, single: total=512.00MiB, used=0.00B => 0
>
> > You mean lowmem is actually FASTER than original mode?
> > That's very surprising.
>
> Correct, unless I add --repair and then original mode is 2x faster than
> lowmem.
>
> > Is there any special operation done for that btrfs?
> > Like offline dedupe or tons of reflinks?
>
> In this case, no.
> Note that btrfs check used to take many hours overnight until I did a
> git pull of btrfs progs and built the latest from TOT.
>
> > BTW, how many subvolumes do you have in the fs?
>
> gargamel:/mnt/btrfs_pool1# btrfs subvolume list . | wc -l
> 91
>
> If I remove snapshots for btrfs send and historical 'backups':
> gargamel:/mnt/btrfs_pool1# btrfs subvolume list . | grep -Ev '(hourly|daily|weekly|rw|ro)' | wc -l
> 5
>
> > This looks like a bug. My first guess is related to number of
> > subvolumes/reflinks, but I'm not sure since I don't have many real-world
> > btrfs.
> >
> > I'll take sometime to look into it.
> >
> > Thanks for the very interesting report,
>
> Thanks for having a look :)
>
> Marc
> --
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
> .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-09 17:44 ` Marc MERLIN
@ 2017-09-10 6:01 ` Qu Wenruo
2017-09-10 13:18 ` Marc MERLIN
0 siblings, 1 reply; 12+ messages in thread
From: Qu Wenruo @ 2017-09-10 6:01 UTC (permalink / raw)
To: Marc MERLIN; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
On 2017年09月10日 01:44, Marc MERLIN wrote:
> So, should I assume that btrfs progs git has some issue since there is
> no plausible way that a check --repair should be faster than a regular
> check?
Yes, the assumption that repair should be no faster than RO check is
correct.
Especially for clean fs, repair should just behave the same as RO check.
And I'll first submit a patch (or patches) to output the consumed time
for each tree, so we could have a clue what is going wrong.
(Digging the code is just a little too boring for me)
Thanks,
Qu
>
> Thanks,
> Marc
>
> On Tue, Sep 05, 2017 at 07:45:25AM -0700, Marc MERLIN wrote:
>> On Tue, Sep 05, 2017 at 04:05:04PM +0800, Qu Wenruo wrote:
>>>> gargamel:~# btrfs fi df /mnt/btrfs_pool1
>>>> Data, single: total\x10.60TiB, used\x10.54TiB
>>>> System, DUP: total2.00MiB, used=1.19MiB
>>>> Metadata, DUP: totalX.00GiB, used\x12.69GiB
>>>
>>> Wait for a minute.
>>>
>>> Is that .69GiB means 706 MiB? Or my email client/GMX screwed up the format
>>> (again)?
>>> This output format must be changed, at least to 0.69 GiB, or 706 MiB.
>>
>> Email client problem. I see control characters in what you quoted.
>>
>> Let's try again
>> gargamel:~# btrfs fi df /mnt/btrfs_pool1
>> Data, single: total=10.66TiB, used=10.60TiB => 10TB
>> System, DUP: total=64.00MiB, used=1.20MiB => 1.2MB
>> Metadata, DUP: total=57.50GiB, used=12.76GiB => 13GB
>> GlobalReserve, single: total=512.00MiB, used=0.00B => 0
>>
>>> You mean lowmem is actually FASTER than original mode?
>>> That's very surprising.
>>
>> Correct, unless I add --repair and then original mode is 2x faster than
>> lowmem.
>>
>>> Is there any special operation done for that btrfs?
>>> Like offline dedupe or tons of reflinks?
>>
>> In this case, no.
>> Note that btrfs check used to take many hours overnight until I did a
>> git pull of btrfs progs and built the latest from TOT.
>>
>>> BTW, how many subvolumes do you have in the fs?
>>
>> gargamel:/mnt/btrfs_pool1# btrfs subvolume list . | wc -l
>> 91
>>
>> If I remove snapshots for btrfs send and historical 'backups':
>> gargamel:/mnt/btrfs_pool1# btrfs subvolume list . | grep -Ev '(hourly|daily|weekly|rw|ro)' | wc -l
>> 5
>>
>>> This looks like a bug. My first guess is related to number of
>>> subvolumes/reflinks, but I'm not sure since I don't have many real-world
>>> btrfs.
>>>
>>> I'll take sometime to look into it.
>>>
>>> Thanks for the very interesting report,
>>
>> Thanks for having a look :)
>>
>> Marc
>> --
>> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
>> Microsoft is to operating systems ....
>> .... what McDonalds is to gourmet cooking
>> Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: btrfs check --repair now runs in minutes instead of hours? aborting
2017-09-10 6:01 ` Qu Wenruo
@ 2017-09-10 13:18 ` Marc MERLIN
0 siblings, 0 replies; 12+ messages in thread
From: Marc MERLIN @ 2017-09-10 13:18 UTC (permalink / raw)
To: Qu Wenruo; +Cc: quwenruo, Lu Fengqi, Chris Mason, Btrfs BTRFS, David Sterba
On Sun, Sep 10, 2017 at 02:01:58PM +0800, Qu Wenruo wrote:
>
>
> On 2017年09月10日 01:44, Marc MERLIN wrote:
> > So, should I assume that btrfs progs git has some issue since there is
> > no plausible way that a check --repair should be faster than a regular
> > check?
>
> Yes, the assumption that repair should be no faster than RO check is
> correct.
> Especially for clean fs, repair should just behave the same as RO check.
>
> And I'll first submit a patch (or patches) to output the consumed time for
> each tree, so we could have a clue what is going wrong.
> (Digging the code is just a little too boring for me)
Cool. Let me know when I should sync and re-try.
In the meantime, though, my check --repair went back to 170mn after
triggering an FS bug for Josef, so it seems back to normal.
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2017-09-10 13:18 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-09-05 1:05 btrfs check --repair now runs in minutes instead of hours? aborting Marc MERLIN
2017-09-05 1:21 ` Qu Wenruo
2017-09-05 2:55 ` Marc MERLIN
2017-09-05 4:06 ` Marc MERLIN
2017-09-05 8:05 ` Qu Wenruo
2017-09-05 8:54 ` Duncan
2017-09-05 9:06 ` Qu Wenruo
2017-09-05 9:35 ` Duncan
2017-09-05 14:45 ` Marc MERLIN
2017-09-09 17:44 ` Marc MERLIN
2017-09-10 6:01 ` Qu Wenruo
2017-09-10 13:18 ` Marc MERLIN
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.