From: Wang Yugui <wangyugui@e16-tech.com>
To: linux-btrfs@vger.kernel.org
Subject: Re: [PATCH PoC 1/9] btrfs: introduce BTRFS_IOC_SCRUB_FS family of ioctls
Date: Sat, 03 Sep 2022 17:49:14 +0800 [thread overview]
Message-ID: <20220903174913.BAEA.409509F4@e16-tech.com> (raw)
In-Reply-To: <410234ec-1c4a-f683-6913-1df9757685ff@gmx.com>
Hi,
> On 2022/9/3 17:25, Wang Yugui wrote:
> > Hi,
> >
> >> The new ioctls are to address the disadvantages of the existing
> >> btrfs_scrub_dev():
> >>
> >> a One thread per-device
> >> This can cause multiple block groups to be marked read-only for scrub,
> >> reducing available space temporarily.
> >>
> >> This also causes higher CPU/IO usage.
> >> For scrub, we should use the minimal amount of CPU and cause less
> >> IO when possible.
> >>
> >> b Extra IO for RAID56
> >> For data stripes, we will cause at least 2x IO if we run "btrfs scrub
> >> start <mnt>".
> >> 1x from scrubbing the device of data stripe.
> >> The other 1x from scrubbing the parity stripe.
> >>
> >> This duplicated IO should definitely be avoided
> >>
> >> c Bad progress report for RAID56
> >> We can not report any repaired P/Q bytes at all.
> >>
> >> The a and b will be addressed by the new one thread per-fs
> >> btrfs_scrub_fs ioctl.
> >
> > CRC check of scrub is CPU sensitive, so we still need multiple threads,
> > such as one thread per-fs but with additional threads pool based on
> > chunks?
>
> This depends.
>
> Scrub should be a background work, which can already be affected by
> scheduling, and I don't think users would bother 5% or 10% longer
> runtime for a several TB fs.
>
> Furthermore if checksum in a single thread is going to be a bottleneck,
> then I'd say your storage is already so fast that scrub duration is not
> your primary concern any more.
scrub is sequence I/O, HDD is very fast too.
HDD*10 with HW RAID6 is very fast for scrub, about 2GB/s or more.
> Yes, it can be possible to offload the csum verification into multiple
> threads, like one thread per mirror/device, but I don't want to
> sacrifice readability for very minor performance improvement.
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2022/09/03
next prev parent reply other threads:[~2022-09-03 9:49 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-03 8:19 [PATCH PoC 0/9] btrfs: scrub: introduce a new family of ioctl, scrub_fs Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 1/9] btrfs: introduce BTRFS_IOC_SCRUB_FS family of ioctls Qu Wenruo
2022-09-03 9:25 ` Wang Yugui
2022-09-03 9:37 ` Qu Wenruo
2022-09-03 9:49 ` Wang Yugui [this message]
2022-09-03 11:47 ` kernel test robot
2022-09-03 11:55 ` Qu Wenruo
2022-09-05 2:05 ` [kbuild-all] " Chen, Rong A
2022-09-09 4:17 ` Wang Yugui
2022-09-09 6:57 ` Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 2/9] btrfs: scrub: introduce place holder for btrfs_scrub_fs() Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 3/9] btrfs: scrub: introduce a place holder helper scrub_fs_iterate_bgs() Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 4/9] btrfs: scrub: introduce place holder helper scrub_fs_block_group() Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 5/9] btrfs: scrub: add helpers to fulfill csum/extent_generation Qu Wenruo
2022-09-03 12:19 ` kernel test robot
2022-09-03 8:19 ` [PATCH PoC 6/9] btrfs: scrub: submit and wait for the read of each copy Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 7/9] btrfs: scrub: implement metadata verification code for scrub_fs Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 8/9] btrfs: scrub: implement data " Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 9/9] btrfs: scrub: implement recoverable sectors report " Qu Wenruo
2022-09-03 11:22 ` kernel test robot
2022-09-03 11:33 ` kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220903174913.BAEA.409509F4@e16-tech.com \
--to=wangyugui@e16-tech.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox