From: Wang Yugui <wangyugui@e16-tech.com>
To: linux-btrfs@vger.kernel.org
Subject: Re: [PATCH PoC 1/9] btrfs: introduce BTRFS_IOC_SCRUB_FS family of ioctls
Date: Sat, 03 Sep 2022 17:49:14 +0800 [thread overview]
Message-ID: <20220903174913.BAEA.409509F4@e16-tech.com> (raw)
In-Reply-To: <410234ec-1c4a-f683-6913-1df9757685ff@gmx.com>
Hi,
> On 2022/9/3 17:25, Wang Yugui wrote:
> > Hi,
> >
> >> The new ioctls are to address the disadvantages of the existing
> >> btrfs_scrub_dev():
> >>
> >> a One thread per-device
> >> This can cause multiple block groups to be marked read-only for scrub,
> >> reducing available space temporarily.
> >>
> >> This also causes higher CPU/IO usage.
> >> For scrub, we should use the minimal amount of CPU and cause less
> >> IO when possible.
> >>
> >> b Extra IO for RAID56
> >> For data stripes, we will cause at least 2x IO if we run "btrfs scrub
> >> start <mnt>".
> >> 1x from scrubbing the device of data stripe.
> >> The other 1x from scrubbing the parity stripe.
> >>
> >> This duplicated IO should definitely be avoided
> >>
> >> c Bad progress report for RAID56
> >> We can not report any repaired P/Q bytes at all.
> >>
> >> The a and b will be addressed by the new one thread per-fs
> >> btrfs_scrub_fs ioctl.
> >
> > CRC check of scrub is CPU sensitive, so we still need multiple threads,
> > such as one thread per-fs but with additional threads pool based on
> > chunks?
>
> This depends.
>
> Scrub should be a background work, which can already be affected by
> scheduling, and I don't think users would bother 5% or 10% longer
> runtime for a several TB fs.
>
> Furthermore if checksum in a single thread is going to be a bottleneck,
> then I'd say your storage is already so fast that scrub duration is not
> your primary concern any more.
scrub is sequence I/O, HDD is very fast too.
HDD*10 with HW RAID6 is very fast for scrub, about 2GB/s or more.
> Yes, it can be possible to offload the csum verification into multiple
> threads, like one thread per mirror/device, but I don't want to
> sacrifice readability for very minor performance improvement.
Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2022/09/03
next prev parent reply other threads:[~2022-09-03 9:49 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-03 8:19 [PATCH PoC 0/9] btrfs: scrub: introduce a new family of ioctl, scrub_fs Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 1/9] btrfs: introduce BTRFS_IOC_SCRUB_FS family of ioctls Qu Wenruo
2022-09-03 9:25 ` Wang Yugui
2022-09-03 9:37 ` Qu Wenruo
2022-09-03 9:49 ` Wang Yugui [this message]
2022-09-03 11:47 ` kernel test robot
2022-09-03 11:55 ` Qu Wenruo
2022-09-03 11:55 ` Qu Wenruo
2022-09-05 2:05 ` [kbuild-all] " Chen, Rong A
2022-09-09 4:17 ` Wang Yugui
2022-09-09 6:57 ` Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 2/9] btrfs: scrub: introduce place holder for btrfs_scrub_fs() Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 3/9] btrfs: scrub: introduce a place holder helper scrub_fs_iterate_bgs() Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 4/9] btrfs: scrub: introduce place holder helper scrub_fs_block_group() Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 5/9] btrfs: scrub: add helpers to fulfill csum/extent_generation Qu Wenruo
2022-09-03 12:19 ` kernel test robot
2022-09-03 8:19 ` [PATCH PoC 6/9] btrfs: scrub: submit and wait for the read of each copy Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 7/9] btrfs: scrub: implement metadata verification code for scrub_fs Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 8/9] btrfs: scrub: implement data " Qu Wenruo
2022-09-03 8:19 ` [PATCH PoC 9/9] btrfs: scrub: implement recoverable sectors report " Qu Wenruo
2022-09-03 11:22 ` kernel test robot
2022-09-03 11:33 ` kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220903174913.BAEA.409509F4@e16-tech.com \
--to=wangyugui@e16-tech.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.