* Re: [PATCH 06/16] mmc: core: replace waitqueue with worker
From: Bartlomiej Zolnierkiewicz @ 2017-02-28 16:10 UTC (permalink / raw)
To: Linus Walleij
Cc: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente,
Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann
In-Reply-To: <20170209153403.9730-7-linus.walleij@linaro.org>
On Thursday, February 09, 2017 04:33:53 PM Linus Walleij wrote:
> The waitqueue in the host context is there to signal back from
> mmc_request_done() through mmc_wait_data_done() that the hardware
> is done with a command, and when the wait is over, the core
> will typically submit the next asynchronous request that is pending
> just waiting for the hardware to be available.
>
> This is in the way for letting the mmc_request_done() trigger the
> report up to the block layer that a block request is finished.
>
> Re-jig this as a first step, remvoving the waitqueue and introducing
> a work that will run after a completed asynchronous request,
> finalizing that request, including retransmissions, and eventually
> reporting back with a completion and a status code to the
> asynchronous issue method.
>
> This had the upside that we can remove the MMC_BLK_NEW_REQUEST
> status code and the "new_request" state in the request queue
> that is only there to make the state machine spin out
> the first time we send a request.
>
> Introduce a workqueue in the host for handling just this, and
> then a work and completion in the asynchronous request to deal
> with this mechanism.
>
> This is a central change that let us do many other changes since
> we have broken the submit and complete code paths in two, and we
> can potentially remove the NULL flushing of the asynchronous
> pipeline and report block requests as finished directly from
> the worker.
>
> Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung R&D Institute Poland
Samsung Electronics
^ permalink raw reply
* Re: [PATCH 06/13] bdi: Make wb->bdi a proper reference
From: Tejun Heo @ 2017-02-28 16:07 UTC (permalink / raw)
To: Jan Kara
Cc: Jens Axboe, linux-block, Christoph Hellwig, Dan Williams,
Thiago Jung Bauermann, Lekshmi Pillai, NeilBrown, Omar Sandoval
In-Reply-To: <20170221170958.21845-7-jack@suse.cz>
On Tue, Feb 21, 2017 at 06:09:51PM +0100, Jan Kara wrote:
> Make wb->bdi a proper refcounted reference to bdi for all bdi_writeback
> structures except for the one embedded inside struct backing_dev_info.
> That will allow us to simplify bdi unregistration.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
Acked-by: Tejun Heo <tj@kernel.org>
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH 05/13] bdi: Mark congested->bdi as internal
From: Tejun Heo @ 2017-02-28 16:06 UTC (permalink / raw)
To: Jan Kara
Cc: Jens Axboe, linux-block, Christoph Hellwig, Dan Williams,
Thiago Jung Bauermann, Lekshmi Pillai, NeilBrown, Omar Sandoval
In-Reply-To: <20170221170958.21845-6-jack@suse.cz>
On Tue, Feb 21, 2017 at 06:09:50PM +0100, Jan Kara wrote:
> congested->bdi pointer is used only to be able to remove congested
> structure from bdi->cgwb_congested_tree on structure release. Moreover
> the pointer can become NULL when we unregister the bdi. Rename the field
> to __bdi and add a comment to make it more explicit this is internal
> stuff of memcg writeback code and people should not use the field as
> such use will be likely race prone.
>
> We do not bother with converting congested->bdi to a proper refcounted
> reference. It will be slightly ugly to special-case bdi->wb.congested to
> avoid effectively a cyclic reference of bdi to itself and the reference
> gets cleared from bdi_unregister() making it impossible to reference
> a freed bdi.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
Acked-by: Tejun Heo <tj@kernel.org>
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH 0/3] Separate zone requests from medium access requests
From: Christoph Hellwig @ 2017-02-28 16:02 UTC (permalink / raw)
To: Damien Le Moal
Cc: Jens Axboe, linux-block, Martin K . Petersen, linux-scsi,
MPT-FusionLinux.pdl, Hannes Reinecke, Christoph Hellwig,
Bart Van Assche
In-Reply-To: <20170228102535.15966-1-damien.lemoal@wdc.com>
I don't really like this too much - this is too many SCSI specifics
for the block layer to care. Maybe using bios for the zone ops was a
mistake after all, and we should just have operations in struct block_device
instead..
^ permalink raw reply
* [PATCH v2 13/13] md: raid10: avoid direct access to bvec table in handle_reshape_read_error
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
The cost is 128bytes(8*16) stack space in kernel thread context, and
just use the bio helper to retrieve pages from bio.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid10.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 0b97631e3905..6ffb64ab45f8 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -4670,7 +4670,15 @@ static int handle_reshape_read_error(struct mddev *mddev,
struct r10bio *r10b = &on_stack.r10_bio;
int slot = 0;
int idx = 0;
- struct bio_vec *bvec = r10_bio->master_bio->bi_io_vec;
+ struct bio_vec *bvl;
+ struct page *pages[RESYNC_PAGES];
+
+ /*
+ * This bio is allocated in reshape_request(), and size
+ * is still RESYNC_PAGES
+ */
+ bio_for_each_segment_all(bvl, r10_bio->master_bio, idx)
+ pages[idx] = bvl->bv_page;
r10b->sector = r10_bio->sector;
__raid10_find_phys(&conf->prev, r10b);
@@ -4699,7 +4707,7 @@ static int handle_reshape_read_error(struct mddev *mddev,
success = sync_page_io(rdev,
addr,
s << 9,
- bvec[idx].bv_page,
+ pages[idx],
REQ_OP_READ, 0, false);
rdev_dec_pending(rdev, mddev);
rcu_read_lock();
--
2.7.4
^ permalink raw reply related
* [PATCH v2 12/13] md: raid10: retrieve page from preallocated resync page array
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Now one page array is allocated for each resync bio, and we can
retrieve page from this table directly.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid10.c | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index f887b21332e7..0b97631e3905 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -2065,6 +2065,7 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
int i, first;
struct bio *tbio, *fbio;
int vcnt;
+ struct page **tpages, **fpages;
atomic_set(&r10_bio->remaining, 1);
@@ -2080,6 +2081,7 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
fbio = r10_bio->devs[i].bio;
fbio->bi_iter.bi_size = r10_bio->sectors << 9;
fbio->bi_iter.bi_idx = 0;
+ fpages = get_resync_pages(fbio)->pages;
vcnt = (r10_bio->sectors + (PAGE_SIZE >> 9) - 1) >> (PAGE_SHIFT - 9);
/* now find blocks with errors */
@@ -2094,6 +2096,8 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
continue;
if (i == first)
continue;
+
+ tpages = get_resync_pages(tbio)->pages;
d = r10_bio->devs[i].devnum;
rdev = conf->mirrors[d].rdev;
if (!r10_bio->devs[i].bio->bi_error) {
@@ -2106,8 +2110,8 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
int len = PAGE_SIZE;
if (sectors < (len / 512))
len = sectors * 512;
- if (memcmp(page_address(fbio->bi_io_vec[j].bv_page),
- page_address(tbio->bi_io_vec[j].bv_page),
+ if (memcmp(page_address(fpages[j]),
+ page_address(tpages[j]),
len))
break;
sectors -= len/512;
@@ -2205,6 +2209,7 @@ static void fix_recovery_read_error(struct r10bio *r10_bio)
int idx = 0;
int dr = r10_bio->devs[0].devnum;
int dw = r10_bio->devs[1].devnum;
+ struct page **pages = get_resync_pages(bio)->pages;
while (sectors) {
int s = sectors;
@@ -2220,7 +2225,7 @@ static void fix_recovery_read_error(struct r10bio *r10_bio)
ok = sync_page_io(rdev,
addr,
s << 9,
- bio->bi_io_vec[idx].bv_page,
+ pages[idx],
REQ_OP_READ, 0, false);
if (ok) {
rdev = conf->mirrors[dw].rdev;
@@ -2228,7 +2233,7 @@ static void fix_recovery_read_error(struct r10bio *r10_bio)
ok = sync_page_io(rdev,
addr,
s << 9,
- bio->bi_io_vec[idx].bv_page,
+ pages[idx],
REQ_OP_WRITE, 0, false);
if (!ok) {
set_bit(WriteErrorSeen, &rdev->flags);
--
2.7.4
^ permalink raw reply related
* [PATCH v2 11/13] md: raid10: don't use bio's vec table to manage resync pages
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Now we allocate one page array for managing resync pages, instead
of using bio's vec table to do that, and the old way is very hacky
and won't work any more if multipage bvec is enabled.
The introduced cost is that we need to allocate (128 + 16) * copies
bytes per r10_bio, and it is fine because the inflight r10_bio for
resync shouldn't be much, as pointed by Shaohua.
Also bio_reset() in raid10_sync_request() and reshape_request()
are removed because all bios are freshly new now in these functions
and not necessary to reset any more.
This patch can be thought as cleanup too.
Suggested-by: Shaohua Li <shli@kernel.org>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid10.c | 125 ++++++++++++++++++++++++++++++----------------------
1 file changed, 73 insertions(+), 52 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index a9ddd4f14008..f887b21332e7 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -110,6 +110,16 @@ static void end_reshape(struct r10conf *conf);
#define raid10_log(md, fmt, args...) \
do { if ((md)->queue) blk_add_trace_msg((md)->queue, "raid10 " fmt, ##args); } while (0)
+static inline struct resync_pages *get_resync_pages(struct bio *bio)
+{
+ return bio->bi_private;
+}
+
+static inline struct r10bio *get_resync_r10bio(struct bio *bio)
+{
+ return get_resync_pages(bio)->raid_bio;
+}
+
static void * r10bio_pool_alloc(gfp_t gfp_flags, void *data)
{
struct r10conf *conf = data;
@@ -140,11 +150,11 @@ static void r10bio_pool_free(void *r10_bio, void *data)
static void * r10buf_pool_alloc(gfp_t gfp_flags, void *data)
{
struct r10conf *conf = data;
- struct page *page;
struct r10bio *r10_bio;
struct bio *bio;
- int i, j;
- int nalloc;
+ int j;
+ int nalloc, nalloc_rp;
+ struct resync_pages *rps;
r10_bio = r10bio_pool_alloc(gfp_flags, conf);
if (!r10_bio)
@@ -156,6 +166,15 @@ static void * r10buf_pool_alloc(gfp_t gfp_flags, void *data)
else
nalloc = 2; /* recovery */
+ /* allocate once for all bios */
+ if (!conf->have_replacement)
+ nalloc_rp = nalloc;
+ else
+ nalloc_rp = nalloc * 2;
+ rps = kmalloc(sizeof(struct resync_pages) * nalloc_rp, gfp_flags);
+ if (!rps)
+ goto out_free_r10bio;
+
/*
* Allocate bios.
*/
@@ -175,36 +194,40 @@ static void * r10buf_pool_alloc(gfp_t gfp_flags, void *data)
* Allocate RESYNC_PAGES data pages and attach them
* where needed.
*/
- for (j = 0 ; j < nalloc; j++) {
+ for (j = 0; j < nalloc; j++) {
struct bio *rbio = r10_bio->devs[j].repl_bio;
+ struct resync_pages *rp, *rp_repl;
+
+ rp = &rps[j];
+ if (rbio)
+ rp_repl = &rps[nalloc + j];
+
bio = r10_bio->devs[j].bio;
- for (i = 0; i < RESYNC_PAGES; i++) {
- if (j > 0 && !test_bit(MD_RECOVERY_SYNC,
- &conf->mddev->recovery)) {
- /* we can share bv_page's during recovery
- * and reshape */
- struct bio *rbio = r10_bio->devs[0].bio;
- page = rbio->bi_io_vec[i].bv_page;
- get_page(page);
- } else
- page = alloc_page(gfp_flags);
- if (unlikely(!page))
+
+ if (!j || test_bit(MD_RECOVERY_SYNC,
+ &conf->mddev->recovery)) {
+ if (resync_alloc_pages(rp, gfp_flags))
goto out_free_pages;
+ } else {
+ memcpy(rp, &rps[0], sizeof(*rp));
+ resync_get_all_pages(rp);
+ }
- bio->bi_io_vec[i].bv_page = page;
- if (rbio)
- rbio->bi_io_vec[i].bv_page = page;
+ rp->idx = 0;
+ rp->raid_bio = r10_bio;
+ bio->bi_private = rp;
+ if (rbio) {
+ memcpy(rp_repl, rp, sizeof(*rp));
+ rbio->bi_private = rp_repl;
}
}
return r10_bio;
out_free_pages:
- for ( ; i > 0 ; i--)
- safe_put_page(bio->bi_io_vec[i-1].bv_page);
- while (j--)
- for (i = 0; i < RESYNC_PAGES ; i++)
- safe_put_page(r10_bio->devs[j].bio->bi_io_vec[i].bv_page);
+ while (--j >= 0)
+ resync_free_pages(&rps[j * 2]);
+
j = 0;
out_free_bio:
for ( ; j < nalloc; j++) {
@@ -213,30 +236,34 @@ static void * r10buf_pool_alloc(gfp_t gfp_flags, void *data)
if (r10_bio->devs[j].repl_bio)
bio_put(r10_bio->devs[j].repl_bio);
}
+ kfree(rps);
+out_free_r10bio:
r10bio_pool_free(r10_bio, conf);
return NULL;
}
static void r10buf_pool_free(void *__r10_bio, void *data)
{
- int i;
struct r10conf *conf = data;
struct r10bio *r10bio = __r10_bio;
int j;
+ struct resync_pages *rp = NULL;
- for (j=0; j < conf->copies; j++) {
+ for (j = conf->copies; j--; ) {
struct bio *bio = r10bio->devs[j].bio;
- if (bio) {
- for (i = 0; i < RESYNC_PAGES; i++) {
- safe_put_page(bio->bi_io_vec[i].bv_page);
- bio->bi_io_vec[i].bv_page = NULL;
- }
- bio_put(bio);
- }
+
+ rp = get_resync_pages(bio);
+ resync_free_pages(rp);
+ bio_put(bio);
+
bio = r10bio->devs[j].repl_bio;
if (bio)
bio_put(bio);
}
+
+ /* resync pages array stored in the 1st bio's .bi_private */
+ kfree(rp);
+
r10bio_pool_free(r10bio, conf);
}
@@ -1935,7 +1962,7 @@ static void __end_sync_read(struct r10bio *r10_bio, struct bio *bio, int d)
static void end_sync_read(struct bio *bio)
{
- struct r10bio *r10_bio = bio->bi_private;
+ struct r10bio *r10_bio = get_resync_r10bio(bio);
struct r10conf *conf = r10_bio->mddev->private;
int d = find_bio_disk(conf, r10_bio, bio, NULL, NULL);
@@ -1944,6 +1971,7 @@ static void end_sync_read(struct bio *bio)
static void end_reshape_read(struct bio *bio)
{
+ /* reshape read bio isn't allocated from r10buf_pool */
struct r10bio *r10_bio = bio->bi_private;
__end_sync_read(r10_bio, bio, r10_bio->read_slot);
@@ -1978,7 +2006,7 @@ static void end_sync_request(struct r10bio *r10_bio)
static void end_sync_write(struct bio *bio)
{
- struct r10bio *r10_bio = bio->bi_private;
+ struct r10bio *r10_bio = get_resync_r10bio(bio);
struct mddev *mddev = r10_bio->mddev;
struct r10conf *conf = mddev->private;
int d;
@@ -2058,6 +2086,7 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
for (i=0 ; i < conf->copies ; i++) {
int j, d;
struct md_rdev *rdev;
+ struct resync_pages *rp;
tbio = r10_bio->devs[i].bio;
@@ -2099,11 +2128,13 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio)
* First we need to fixup bv_offset, bv_len and
* bi_vecs, as the read request might have corrupted these
*/
+ rp = get_resync_pages(tbio);
bio_reset(tbio);
tbio->bi_vcnt = vcnt;
tbio->bi_iter.bi_size = fbio->bi_iter.bi_size;
- tbio->bi_private = r10_bio;
+ rp->raid_bio = r10_bio;
+ tbio->bi_private = rp;
tbio->bi_iter.bi_sector = r10_bio->devs[i].addr;
tbio->bi_end_io = end_sync_write;
bio_set_op_attrs(tbio, REQ_OP_WRITE, 0);
@@ -3171,10 +3202,8 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
}
}
bio = r10_bio->devs[0].bio;
- bio_reset(bio);
bio->bi_next = biolist;
biolist = bio;
- bio->bi_private = r10_bio;
bio->bi_end_io = end_sync_read;
bio_set_op_attrs(bio, REQ_OP_READ, 0);
if (test_bit(FailFast, &rdev->flags))
@@ -3198,10 +3227,8 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
if (!test_bit(In_sync, &mrdev->flags)) {
bio = r10_bio->devs[1].bio;
- bio_reset(bio);
bio->bi_next = biolist;
biolist = bio;
- bio->bi_private = r10_bio;
bio->bi_end_io = end_sync_write;
bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
bio->bi_iter.bi_sector = to_addr
@@ -3226,10 +3253,8 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
if (mreplace == NULL || bio == NULL ||
test_bit(Faulty, &mreplace->flags))
break;
- bio_reset(bio);
bio->bi_next = biolist;
biolist = bio;
- bio->bi_private = r10_bio;
bio->bi_end_io = end_sync_write;
bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
bio->bi_iter.bi_sector = to_addr +
@@ -3351,7 +3376,6 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
r10_bio->devs[i].repl_bio->bi_end_io = NULL;
bio = r10_bio->devs[i].bio;
- bio_reset(bio);
bio->bi_error = -EIO;
rcu_read_lock();
rdev = rcu_dereference(conf->mirrors[d].rdev);
@@ -3376,7 +3400,6 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
atomic_inc(&r10_bio->remaining);
bio->bi_next = biolist;
biolist = bio;
- bio->bi_private = r10_bio;
bio->bi_end_io = end_sync_read;
bio_set_op_attrs(bio, REQ_OP_READ, 0);
if (test_bit(FailFast, &conf->mirrors[d].rdev->flags))
@@ -3395,13 +3418,11 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
/* Need to set up for writing to the replacement */
bio = r10_bio->devs[i].repl_bio;
- bio_reset(bio);
bio->bi_error = -EIO;
sector = r10_bio->devs[i].addr;
bio->bi_next = biolist;
biolist = bio;
- bio->bi_private = r10_bio;
bio->bi_end_io = end_sync_write;
bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
if (test_bit(FailFast, &conf->mirrors[d].rdev->flags))
@@ -3440,7 +3461,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
if (len == 0)
break;
for (bio= biolist ; bio ; bio=bio->bi_next) {
- page = bio->bi_io_vec[bio->bi_vcnt].bv_page;
+ page = resync_fetch_page(get_resync_pages(bio));
/*
* won't fail because the vec table is big enough
* to hold all these pages
@@ -3449,7 +3470,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
}
nr_sectors += len>>9;
sector_nr += len>>9;
- } while (biolist->bi_vcnt < RESYNC_PAGES);
+ } while (resync_page_available(get_resync_pages(biolist)));
r10_bio->sectors = nr_sectors;
while (biolist) {
@@ -3457,7 +3478,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
biolist = biolist->bi_next;
bio->bi_next = NULL;
- r10_bio = bio->bi_private;
+ r10_bio = get_resync_r10bio(bio);
r10_bio->sectors = nr_sectors;
if (bio->bi_end_io == end_sync_read) {
@@ -4352,6 +4373,7 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr,
struct bio *blist;
struct bio *bio, *read_bio;
int sectors_done = 0;
+ struct page **pages;
if (sector_nr == 0) {
/* If restarting in the middle, skip the initial sectors */
@@ -4502,11 +4524,9 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr,
if (!rdev2 || test_bit(Faulty, &rdev2->flags))
continue;
- bio_reset(b);
b->bi_bdev = rdev2->bdev;
b->bi_iter.bi_sector = r10_bio->devs[s/2].addr +
rdev2->new_data_offset;
- b->bi_private = r10_bio;
b->bi_end_io = end_reshape_write;
bio_set_op_attrs(b, REQ_OP_WRITE, 0);
b->bi_next = blist;
@@ -4516,8 +4536,9 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr,
/* Now add as many pages as possible to all of these bios. */
nr_sectors = 0;
+ pages = get_resync_pages(r10_bio->devs[0].bio)->pages;
for (s = 0 ; s < max_sectors; s += PAGE_SIZE >> 9) {
- struct page *page = r10_bio->devs[0].bio->bi_io_vec[s/(PAGE_SIZE>>9)].bv_page;
+ struct page *page = pages[s / (PAGE_SIZE >> 9)];
int len = (max_sectors - s) << 9;
if (len > PAGE_SIZE)
len = PAGE_SIZE;
@@ -4701,7 +4722,7 @@ static int handle_reshape_read_error(struct mddev *mddev,
static void end_reshape_write(struct bio *bio)
{
- struct r10bio *r10_bio = bio->bi_private;
+ struct r10bio *r10_bio = get_resync_r10bio(bio);
struct mddev *mddev = r10_bio->mddev;
struct r10conf *conf = mddev->private;
int d;
--
2.7.4
^ permalink raw reply related
* [PATCH v2 09/13] md: raid1: use bio_segments_all()
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Use this helper, instead of direct access to .bi_vcnt.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid1.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 316bd6dd6cc1..7396c99ff7b1 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1091,7 +1091,8 @@ static void alloc_behind_pages(struct bio *bio, struct r1bio *r1_bio)
{
int i;
struct bio_vec *bvec;
- struct bio_vec *bvecs = kzalloc(bio->bi_vcnt * sizeof(struct bio_vec),
+ unsigned vcnt = bio_segments_all(bio);
+ struct bio_vec *bvecs = kzalloc(vcnt * sizeof(struct bio_vec),
GFP_NOIO);
if (unlikely(!bvecs))
return;
@@ -1107,12 +1108,12 @@ static void alloc_behind_pages(struct bio *bio, struct r1bio *r1_bio)
kunmap(bvec->bv_page);
}
r1_bio->behind_bvecs = bvecs;
- r1_bio->behind_page_count = bio->bi_vcnt;
+ r1_bio->behind_page_count = vcnt;
set_bit(R1BIO_BehindIO, &r1_bio->state);
return;
do_sync_io:
- for (i = 0; i < bio->bi_vcnt; i++)
+ for (i = 0; i < vcnt; i++)
if (bvecs[i].bv_page)
put_page(bvecs[i].bv_page);
kfree(bvecs);
--
2.7.4
^ permalink raw reply related
* [PATCH v2 10/13] md: raid10: refactor code of read reshape's .bi_end_io
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
reshape read request is a bit special and requires one extra
bio which isn't allocated from r10buf_pool.
Refactor the .bi_end_io for read reshape, so that we can use
raid10's resync page mangement approach easily in the following
patches.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid10.c | 28 ++++++++++++++++++----------
1 file changed, 18 insertions(+), 10 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index c5f1a117494b..a9ddd4f14008 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1907,17 +1907,9 @@ static int raid10_remove_disk(struct mddev *mddev, struct md_rdev *rdev)
return err;
}
-static void end_sync_read(struct bio *bio)
+static void __end_sync_read(struct r10bio *r10_bio, struct bio *bio, int d)
{
- struct r10bio *r10_bio = bio->bi_private;
struct r10conf *conf = r10_bio->mddev->private;
- int d;
-
- if (bio == r10_bio->master_bio) {
- /* this is a reshape read */
- d = r10_bio->read_slot; /* really the read dev */
- } else
- d = find_bio_disk(conf, r10_bio, bio, NULL, NULL);
if (!bio->bi_error)
set_bit(R10BIO_Uptodate, &r10_bio->state);
@@ -1941,6 +1933,22 @@ static void end_sync_read(struct bio *bio)
}
}
+static void end_sync_read(struct bio *bio)
+{
+ struct r10bio *r10_bio = bio->bi_private;
+ struct r10conf *conf = r10_bio->mddev->private;
+ int d = find_bio_disk(conf, r10_bio, bio, NULL, NULL);
+
+ __end_sync_read(r10_bio, bio, d);
+}
+
+static void end_reshape_read(struct bio *bio)
+{
+ struct r10bio *r10_bio = bio->bi_private;
+
+ __end_sync_read(r10_bio, bio, r10_bio->read_slot);
+}
+
static void end_sync_request(struct r10bio *r10_bio)
{
struct mddev *mddev = r10_bio->mddev;
@@ -4464,7 +4472,7 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr,
read_bio->bi_iter.bi_sector = (r10_bio->devs[r10_bio->read_slot].addr
+ rdev->data_offset);
read_bio->bi_private = r10_bio;
- read_bio->bi_end_io = end_sync_read;
+ read_bio->bi_end_io = end_reshape_read;
bio_set_op_attrs(read_bio, REQ_OP_READ, 0);
read_bio->bi_flags &= (~0UL << BIO_RESET_BITS);
read_bio->bi_error = 0;
--
2.7.4
^ permalink raw reply related
* [PATCH v2 08/13] md: raid1: use bio helper in process_checks()
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Avoid to direct access to bvec table.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid1.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index d0cb5c026506..316bd6dd6cc1 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2108,6 +2108,7 @@ static void process_checks(struct r1bio *r1_bio)
int j;
int size;
int error;
+ struct bio_vec *bi;
struct bio *b = r1_bio->bios[i];
struct resync_pages *rp = get_resync_pages(b);
if (b->bi_end_io != end_sync_read)
@@ -2126,9 +2127,7 @@ static void process_checks(struct r1bio *r1_bio)
b->bi_private = rp;
size = b->bi_iter.bi_size;
- for (j = 0; j < vcnt ; j++) {
- struct bio_vec *bi;
- bi = &b->bi_io_vec[j];
+ bio_for_each_segment_all(bi, b, j) {
bi->bv_offset = 0;
if (size > PAGE_SIZE)
bi->bv_len = PAGE_SIZE;
@@ -2152,17 +2151,22 @@ static void process_checks(struct r1bio *r1_bio)
int error = sbio->bi_error;
struct page **ppages = get_resync_pages(pbio)->pages;
struct page **spages = get_resync_pages(sbio)->pages;
+ struct bio_vec *bi;
+ int page_len[RESYNC_PAGES];
if (sbio->bi_end_io != end_sync_read)
continue;
/* Now we can 'fixup' the error value */
sbio->bi_error = 0;
+ bio_for_each_segment_all(bi, sbio, j)
+ page_len[j] = bi->bv_len;
+
if (!error) {
for (j = vcnt; j-- ; ) {
if (memcmp(page_address(ppages[j]),
page_address(spages[j]),
- sbio->bi_io_vec[j].bv_len))
+ page_len[j]))
break;
}
} else
--
2.7.4
^ permalink raw reply related
* [PATCH v2 07/13] md: raid1: retrieve page from pre-allocated resync page array
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Now one page array is allocated for each resync bio, and we can
retrieve page from this table directly.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid1.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 900144f39630..d0cb5c026506 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1970,6 +1970,7 @@ static int fix_sync_read_error(struct r1bio *r1_bio)
struct mddev *mddev = r1_bio->mddev;
struct r1conf *conf = mddev->private;
struct bio *bio = r1_bio->bios[r1_bio->read_disk];
+ struct page **pages = get_resync_pages(bio)->pages;
sector_t sect = r1_bio->sector;
int sectors = r1_bio->sectors;
int idx = 0;
@@ -2003,7 +2004,7 @@ static int fix_sync_read_error(struct r1bio *r1_bio)
*/
rdev = conf->mirrors[d].rdev;
if (sync_page_io(rdev, sect, s<<9,
- bio->bi_io_vec[idx].bv_page,
+ pages[idx],
REQ_OP_READ, 0, false)) {
success = 1;
break;
@@ -2058,7 +2059,7 @@ static int fix_sync_read_error(struct r1bio *r1_bio)
continue;
rdev = conf->mirrors[d].rdev;
if (r1_sync_page_io(rdev, sect, s,
- bio->bi_io_vec[idx].bv_page,
+ pages[idx],
WRITE) == 0) {
r1_bio->bios[d]->bi_end_io = NULL;
rdev_dec_pending(rdev, mddev);
@@ -2073,7 +2074,7 @@ static int fix_sync_read_error(struct r1bio *r1_bio)
continue;
rdev = conf->mirrors[d].rdev;
if (r1_sync_page_io(rdev, sect, s,
- bio->bi_io_vec[idx].bv_page,
+ pages[idx],
READ) != 0)
atomic_add(s, &rdev->corrected_errors);
}
@@ -2149,6 +2150,8 @@ static void process_checks(struct r1bio *r1_bio)
struct bio *pbio = r1_bio->bios[primary];
struct bio *sbio = r1_bio->bios[i];
int error = sbio->bi_error;
+ struct page **ppages = get_resync_pages(pbio)->pages;
+ struct page **spages = get_resync_pages(sbio)->pages;
if (sbio->bi_end_io != end_sync_read)
continue;
@@ -2157,11 +2160,8 @@ static void process_checks(struct r1bio *r1_bio)
if (!error) {
for (j = vcnt; j-- ; ) {
- struct page *p, *s;
- p = pbio->bi_io_vec[j].bv_page;
- s = sbio->bi_io_vec[j].bv_page;
- if (memcmp(page_address(p),
- page_address(s),
+ if (memcmp(page_address(ppages[j]),
+ page_address(spages[j]),
sbio->bi_io_vec[j].bv_len))
break;
}
--
2.7.4
^ permalink raw reply related
* [PATCH v2 06/13] md: raid1: don't use bio's vec table to manage resync pages
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Now we allocate one page array for managing resync pages, instead
of using bio's vec table to do that, and the old way is very hacky
and won't work any more if multipage bvec is enabled.
The introduced cost is that we need to allocate (128 + 16) * raid_disks
bytes per r1_bio, and it is fine because the inflight r1_bio for
resync shouldn't be much, as pointed by Shaohua.
Also the bio_reset() in raid1_sync_request() is removed because
all bios are freshly new now and not necessary to reset any more.
This patch can be thought as a cleanup too
Suggested-by: Shaohua Li <shli@kernel.org>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid1.c | 83 ++++++++++++++++++++++++++++++++++--------------------
1 file changed, 53 insertions(+), 30 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index c442b4657e2f..900144f39630 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -77,6 +77,16 @@ static void lower_barrier(struct r1conf *conf, sector_t sector_nr);
#define raid1_log(md, fmt, args...) \
do { if ((md)->queue) blk_add_trace_msg((md)->queue, "raid1 " fmt, ##args); } while (0)
+static inline struct resync_pages *get_resync_pages(struct bio *bio)
+{
+ return bio->bi_private;
+}
+
+static inline struct r1bio *get_resync_r1bio(struct bio *bio)
+{
+ return get_resync_pages(bio)->raid_bio;
+}
+
static void * r1bio_pool_alloc(gfp_t gfp_flags, void *data)
{
struct pool_info *pi = data;
@@ -104,12 +114,18 @@ static void * r1buf_pool_alloc(gfp_t gfp_flags, void *data)
struct r1bio *r1_bio;
struct bio *bio;
int need_pages;
- int i, j;
+ int j;
+ struct resync_pages *rps;
r1_bio = r1bio_pool_alloc(gfp_flags, pi);
if (!r1_bio)
return NULL;
+ rps = kmalloc(sizeof(struct resync_pages) * pi->raid_disks,
+ gfp_flags);
+ if (!rps)
+ goto out_free_r1bio;
+
/*
* Allocate bios : 1 for reading, n-1 for writing
*/
@@ -129,22 +145,22 @@ static void * r1buf_pool_alloc(gfp_t gfp_flags, void *data)
need_pages = pi->raid_disks;
else
need_pages = 1;
- for (j = 0; j < need_pages; j++) {
+ for (j = 0; j < pi->raid_disks; j++) {
+ struct resync_pages *rp = &rps[j];
+
bio = r1_bio->bios[j];
- bio->bi_vcnt = RESYNC_PAGES;
-
- if (bio_alloc_pages(bio, gfp_flags))
- goto out_free_pages;
- }
- /* If not user-requests, copy the page pointers to all bios */
- if (!test_bit(MD_RECOVERY_REQUESTED, &pi->mddev->recovery)) {
- for (i=0; i<RESYNC_PAGES ; i++)
- for (j=1; j<pi->raid_disks; j++) {
- struct page *page =
- r1_bio->bios[0]->bi_io_vec[i].bv_page;
- get_page(page);
- r1_bio->bios[j]->bi_io_vec[i].bv_page = page;
- }
+
+ if (j < need_pages) {
+ if (resync_alloc_pages(rp, gfp_flags))
+ goto out_free_pages;
+ } else {
+ memcpy(rp, &rps[0], sizeof(*rp));
+ resync_get_all_pages(rp);
+ }
+
+ rp->idx = 0;
+ rp->raid_bio = r1_bio;
+ bio->bi_private = rp;
}
r1_bio->master_bio = NULL;
@@ -153,11 +169,14 @@ static void * r1buf_pool_alloc(gfp_t gfp_flags, void *data)
out_free_pages:
while (--j >= 0)
- bio_free_pages(r1_bio->bios[j]);
+ resync_free_pages(&rps[j]);
out_free_bio:
while (++j < pi->raid_disks)
bio_put(r1_bio->bios[j]);
+ kfree(rps);
+
+out_free_r1bio:
r1bio_pool_free(r1_bio, data);
return NULL;
}
@@ -165,14 +184,18 @@ static void * r1buf_pool_alloc(gfp_t gfp_flags, void *data)
static void r1buf_pool_free(void *__r1_bio, void *data)
{
struct pool_info *pi = data;
- int i,j;
+ int i;
struct r1bio *r1bio = __r1_bio;
+ struct resync_pages *rp = NULL;
- for (i = 0; i < RESYNC_PAGES; i++)
- for (j = pi->raid_disks; j-- ;)
- safe_put_page(r1bio->bios[j]->bi_io_vec[i].bv_page);
- for (i=0 ; i < pi->raid_disks; i++)
+ for (i = pi->raid_disks; i--; ) {
+ rp = get_resync_pages(r1bio->bios[i]);
+ resync_free_pages(rp);
bio_put(r1bio->bios[i]);
+ }
+
+ /* resync pages array stored in the 1st bio's .bi_private */
+ kfree(rp);
r1bio_pool_free(r1bio, data);
}
@@ -1849,7 +1872,7 @@ static int raid1_remove_disk(struct mddev *mddev, struct md_rdev *rdev)
static void end_sync_read(struct bio *bio)
{
- struct r1bio *r1_bio = bio->bi_private;
+ struct r1bio *r1_bio = get_resync_r1bio(bio);
update_head_pos(r1_bio->read_disk, r1_bio);
@@ -1868,7 +1891,7 @@ static void end_sync_read(struct bio *bio)
static void end_sync_write(struct bio *bio)
{
int uptodate = !bio->bi_error;
- struct r1bio *r1_bio = bio->bi_private;
+ struct r1bio *r1_bio = get_resync_r1bio(bio);
struct mddev *mddev = r1_bio->mddev;
struct r1conf *conf = mddev->private;
sector_t first_bad;
@@ -2085,6 +2108,7 @@ static void process_checks(struct r1bio *r1_bio)
int size;
int error;
struct bio *b = r1_bio->bios[i];
+ struct resync_pages *rp = get_resync_pages(b);
if (b->bi_end_io != end_sync_read)
continue;
/* fixup the bio for reuse, but preserve errno */
@@ -2097,7 +2121,8 @@ static void process_checks(struct r1bio *r1_bio)
conf->mirrors[i].rdev->data_offset;
b->bi_bdev = conf->mirrors[i].rdev->bdev;
b->bi_end_io = end_sync_read;
- b->bi_private = r1_bio;
+ rp->raid_bio = r1_bio;
+ b->bi_private = rp;
size = b->bi_iter.bi_size;
for (j = 0; j < vcnt ; j++) {
@@ -2755,7 +2780,6 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr,
for (i = 0; i < conf->raid_disks * 2; i++) {
struct md_rdev *rdev;
bio = r1_bio->bios[i];
- bio_reset(bio);
rdev = rcu_dereference(conf->mirrors[i].rdev);
if (rdev == NULL ||
@@ -2811,7 +2835,6 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr,
atomic_inc(&rdev->nr_pending);
bio->bi_iter.bi_sector = sector_nr + rdev->data_offset;
bio->bi_bdev = rdev->bdev;
- bio->bi_private = r1_bio;
if (test_bit(FailFast, &rdev->flags))
bio->bi_opf |= MD_FAILFAST;
}
@@ -2899,7 +2922,7 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr,
for (i = 0 ; i < conf->raid_disks * 2; i++) {
bio = r1_bio->bios[i];
if (bio->bi_end_io) {
- page = bio->bi_io_vec[bio->bi_vcnt].bv_page;
+ page = resync_fetch_page(get_resync_pages(bio));
/*
* won't fail because the vec table is big
@@ -2911,8 +2934,8 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr,
nr_sectors += len>>9;
sector_nr += len>>9;
sync_blocks -= (len>>9);
- } while (r1_bio->bios[disk]->bi_vcnt < RESYNC_PAGES);
- bio_full:
+ } while (resync_page_available(r1_bio->bios[disk]->bi_private));
+
r1_bio->sectors = nr_sectors;
if (mddev_is_clustered(mddev) &&
--
2.7.4
^ permalink raw reply related
* [PATCH v2 05/13] md: raid1: simplify r1buf_pool_free()
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
This patch gets each page's reference of each bio for resync,
then r1buf_pool_free() gets simplified a lot.
The same policy has been taken in raid10's buf pool allocation/free
too.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid1.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 25c9172db639..c442b4657e2f 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -139,9 +139,12 @@ static void * r1buf_pool_alloc(gfp_t gfp_flags, void *data)
/* If not user-requests, copy the page pointers to all bios */
if (!test_bit(MD_RECOVERY_REQUESTED, &pi->mddev->recovery)) {
for (i=0; i<RESYNC_PAGES ; i++)
- for (j=1; j<pi->raid_disks; j++)
- r1_bio->bios[j]->bi_io_vec[i].bv_page =
+ for (j=1; j<pi->raid_disks; j++) {
+ struct page *page =
r1_bio->bios[0]->bi_io_vec[i].bv_page;
+ get_page(page);
+ r1_bio->bios[j]->bi_io_vec[i].bv_page = page;
+ }
}
r1_bio->master_bio = NULL;
@@ -166,12 +169,8 @@ static void r1buf_pool_free(void *__r1_bio, void *data)
struct r1bio *r1bio = __r1_bio;
for (i = 0; i < RESYNC_PAGES; i++)
- for (j = pi->raid_disks; j-- ;) {
- if (j == 0 ||
- r1bio->bios[j]->bi_io_vec[i].bv_page !=
- r1bio->bios[0]->bi_io_vec[i].bv_page)
- safe_put_page(r1bio->bios[j]->bi_io_vec[i].bv_page);
- }
+ for (j = pi->raid_disks; j-- ;)
+ safe_put_page(r1bio->bios[j]->bi_io_vec[i].bv_page);
for (i=0 ; i < pi->raid_disks; i++)
bio_put(r1bio->bios[i]);
--
2.7.4
^ permalink raw reply related
* [PATCH v2 04/13] md: prepare for managing resync I/O pages in clean way
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Now resync I/O use bio's bec table to manage pages,
this way is very hacky, and may not work any more
once multipage bvec is introduced.
So introduce helpers and new data structure for
managing resync I/O pages more cleanly.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/md.h | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 1d63239a1be4..b5a638d85cb4 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -720,4 +720,58 @@ static inline void mddev_check_writesame(struct mddev *mddev, struct bio *bio)
#define RESYNC_BLOCK_SIZE (64*1024)
#define RESYNC_PAGES ((RESYNC_BLOCK_SIZE + PAGE_SIZE-1) / PAGE_SIZE)
+/* for managing resync I/O pages */
+struct resync_pages {
+ unsigned idx; /* for get/put page from the pool */
+ void *raid_bio;
+ struct page *pages[RESYNC_PAGES];
+};
+
+static inline int resync_alloc_pages(struct resync_pages *rp,
+ gfp_t gfp_flags)
+{
+ int i;
+
+ for (i = 0; i < RESYNC_PAGES; i++) {
+ rp->pages[i] = alloc_page(gfp_flags);
+ if (!rp->pages[i])
+ goto out_free;
+ }
+
+ return 0;
+
+ out_free:
+ while (--i >= 0)
+ __free_page(rp->pages[i]);
+ return -ENOMEM;
+}
+
+static inline void resync_free_pages(struct resync_pages *rp)
+{
+ int i;
+
+ for (i = 0; i < RESYNC_PAGES; i++)
+ __free_page(rp->pages[i]);
+}
+
+static inline void resync_get_all_pages(struct resync_pages *rp)
+{
+ int i;
+
+ for (i = 0; i < RESYNC_PAGES; i++)
+ get_page(rp->pages[i]);
+}
+
+static inline struct page *resync_fetch_page(struct resync_pages *rp)
+{
+ if (WARN_ON_ONCE(rp->idx >= RESYNC_PAGES))
+ return NULL;
+ return rp->pages[rp->idx++];
+}
+
+static inline bool resync_page_available(struct resync_pages *rp)
+{
+ return rp->idx < RESYNC_PAGES;
+}
+
#endif /* _MD_MD_H */
--
2.7.4
^ permalink raw reply related
* [PATCH v2 01/13] block: introduce bio_segments_all()
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
So that we can replace the direct access to .bi_vcnt.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
include/linux/bio.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 8e521194f6fc..3364b3ed90e7 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -293,6 +293,13 @@ static inline void bio_get_last_bvec(struct bio *bio, struct bio_vec *bv)
bv->bv_len = iter.bi_bvec_done;
}
+static inline unsigned bio_segments_all(struct bio *bio)
+{
+ WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED));
+
+ return bio->bi_vcnt;
+}
+
enum bip_flags {
BIP_BLOCK_INTEGRITY = 1 << 0, /* block layer owns integrity data */
BIP_MAPPED_INTEGRITY = 1 << 1, /* ref tag has been remapped */
--
2.7.4
^ permalink raw reply related
* [PATCH v2 02/13] md: raid1/raid10: don't handle failure of bio_add_page()
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
All bio_add_page() is for adding one page into resync bio,
which is big enough to hold RESYNC_PAGES pages, and
the current bio_add_page() doesn't check queue limit any more,
so it won't fail at all.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/raid1.c | 21 ++++++---------------
drivers/md/raid10.c | 41 ++++++++++-------------------------------
2 files changed, 16 insertions(+), 46 deletions(-)
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 0628c07dd16d..b3021355c7e2 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -2903,21 +2903,12 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr,
bio = r1_bio->bios[i];
if (bio->bi_end_io) {
page = bio->bi_io_vec[bio->bi_vcnt].bv_page;
- if (bio_add_page(bio, page, len, 0) == 0) {
- /* stop here */
- bio->bi_io_vec[bio->bi_vcnt].bv_page = page;
- while (i > 0) {
- i--;
- bio = r1_bio->bios[i];
- if (bio->bi_end_io==NULL)
- continue;
- /* remove last page from this bio */
- bio->bi_vcnt--;
- bio->bi_iter.bi_size -= len;
- bio_clear_flag(bio, BIO_SEG_VALID);
- }
- goto bio_full;
- }
+
+ /*
+ * won't fail because the vec table is big
+ * enough to hold all these pages
+ */
+ bio_add_page(bio, page, len, 0);
}
}
nr_sectors += len>>9;
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 33f6a535dc1f..ceb3acc793cf 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -3435,27 +3435,16 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
if (len == 0)
break;
for (bio= biolist ; bio ; bio=bio->bi_next) {
- struct bio *bio2;
page = bio->bi_io_vec[bio->bi_vcnt].bv_page;
- if (bio_add_page(bio, page, len, 0))
- continue;
-
- /* stop here */
- bio->bi_io_vec[bio->bi_vcnt].bv_page = page;
- for (bio2 = biolist;
- bio2 && bio2 != bio;
- bio2 = bio2->bi_next) {
- /* remove last page from this bio */
- bio2->bi_vcnt--;
- bio2->bi_iter.bi_size -= len;
- bio_clear_flag(bio2, BIO_SEG_VALID);
- }
- goto bio_full;
+ /*
+ * won't fail because the vec table is big enough
+ * to hold all these pages
+ */
+ bio_add_page(bio, page, len, 0);
}
nr_sectors += len>>9;
sector_nr += len>>9;
} while (biolist->bi_vcnt < RESYNC_PAGES);
- bio_full:
r10_bio->sectors = nr_sectors;
while (biolist) {
@@ -4528,25 +4517,15 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr,
if (len > PAGE_SIZE)
len = PAGE_SIZE;
for (bio = blist; bio ; bio = bio->bi_next) {
- struct bio *bio2;
- if (bio_add_page(bio, page, len, 0))
- continue;
-
- /* Didn't fit, must stop */
- for (bio2 = blist;
- bio2 && bio2 != bio;
- bio2 = bio2->bi_next) {
- /* Remove last page from this bio */
- bio2->bi_vcnt--;
- bio2->bi_iter.bi_size -= len;
- bio_clear_flag(bio2, BIO_SEG_VALID);
- }
- goto bio_full;
+ /*
+ * won't fail because the vec table is big enough
+ * to hold all these pages
+ */
+ bio_add_page(bio, page, len, 0);
}
sector_nr += len >> 9;
nr_sectors += len >> 9;
}
-bio_full:
rcu_read_unlock();
r10_bio->sectors = nr_sectors;
--
2.7.4
^ permalink raw reply related
* [PATCH v2 00/14] md: cleanup on direct access to bvec table
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In MD's resync I/O path, there are lots of direct access to bio's
bvec table. This patchset kills almost all, and the conversion
is quite straightforward. One root cause of direct access to bvec
table is that resync I/O uses the bio's bvec to manage pages.
In V1, as suggested by Shaohua, a new approach is used to manage
these pages for resync I/O, turns out code becomes more clean
and readable.
Once direct access to bvec table in MD is cleaned up, we may make
multipage bvec moving on.
V2:
- remove the patch for introducing/applying bio_remove_last_page()
V1:
- allocate page array to manage resync pages
Thanks,
Ming
Ming Lei (13):
block: introduce bio_segments_all()
md: raid1/raid10: don't handle failure of bio_add_page()
md: move two macros into md.h
md: prepare for managing resync I/O pages in clean way
md: raid1: simplify r1buf_pool_free()
md: raid1: don't use bio's vec table to manage resync pages
md: raid1: retrieve page from pre-allocated resync page array
md: raid1: use bio helper in process_checks()
md: raid1: use bio_segments_all()
md: raid10: refactor code of read reshape's .bi_end_io
md: raid10: don't use bio's vec table to manage resync pages
md: raid10: retrieve page from preallocated resync page array
md: raid10: avoid direct access to bvec table in
handle_reshape_read_error
drivers/md/md.h | 59 ++++++++++++++
drivers/md/raid1.c | 140 ++++++++++++++++++---------------
drivers/md/raid10.c | 220 ++++++++++++++++++++++++++++------------------------
include/linux/bio.h | 7 ++
4 files changed, 263 insertions(+), 163 deletions(-)
--
2.7.4
^ permalink raw reply
* [PATCH v2 03/13] md: move two macros into md.h
From: Ming Lei @ 2017-02-28 15:41 UTC (permalink / raw)
To: Shaohua Li, Jens Axboe, linux-raid, linux-block,
Christoph Hellwig; +Cc: Ming Lei
In-Reply-To: <1488296503-4987-1-git-send-email-tom.leiming@gmail.com>
Both raid1 and raid10 share common resync
block size and page count, so move them into md.h.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/md/md.h | 5 +++++
drivers/md/raid1.c | 2 --
drivers/md/raid10.c | 3 ---
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/md/md.h b/drivers/md/md.h
index b8859cbf84b6..1d63239a1be4 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -715,4 +715,9 @@ static inline void mddev_check_writesame(struct mddev *mddev, struct bio *bio)
!bdev_get_queue(bio->bi_bdev)->limits.max_write_same_sectors)
mddev->queue->limits.max_write_same_sectors = 0;
}
+
+/* Maximum size of each resync request */
+#define RESYNC_BLOCK_SIZE (64*1024)
+#define RESYNC_PAGES ((RESYNC_BLOCK_SIZE + PAGE_SIZE-1) / PAGE_SIZE)
+
#endif /* _MD_MD_H */
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index b3021355c7e2..25c9172db639 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -91,10 +91,8 @@ static void r1bio_pool_free(void *r1_bio, void *data)
kfree(r1_bio);
}
-#define RESYNC_BLOCK_SIZE (64*1024)
#define RESYNC_DEPTH 32
#define RESYNC_SECTORS (RESYNC_BLOCK_SIZE >> 9)
-#define RESYNC_PAGES ((RESYNC_BLOCK_SIZE + PAGE_SIZE-1) / PAGE_SIZE)
#define RESYNC_WINDOW (RESYNC_BLOCK_SIZE * RESYNC_DEPTH)
#define RESYNC_WINDOW_SECTORS (RESYNC_WINDOW >> 9)
#define CLUSTER_RESYNC_WINDOW (16 * RESYNC_WINDOW)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index ceb3acc793cf..c5f1a117494b 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -125,9 +125,6 @@ static void r10bio_pool_free(void *r10_bio, void *data)
kfree(r10_bio);
}
-/* Maximum size of each resync request */
-#define RESYNC_BLOCK_SIZE (64*1024)
-#define RESYNC_PAGES ((RESYNC_BLOCK_SIZE + PAGE_SIZE-1) / PAGE_SIZE)
/* amount of memory to reserve for resync requests */
#define RESYNC_WINDOW (1024*1024)
/* maximum number of concurrent requests, memory permitting */
--
2.7.4
^ permalink raw reply related
* [PATCH 09/12] block: advertize max atomic write limit
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
In-Reply-To: <20170228145737.19016-1-hch@lst.de>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/blk-settings.c | 22 ++++++++++++++++++++++
block/blk-sysfs.c | 12 ++++++++++++
include/linux/blkdev.h | 9 +++++++++
3 files changed, 43 insertions(+)
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 529e55f52a03..9279542472fb 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -93,6 +93,7 @@ void blk_set_default_limits(struct queue_limits *lim)
lim->virt_boundary_mask = 0;
lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
lim->max_sectors = lim->max_hw_sectors = BLK_SAFE_MAX_SECTORS;
+ lim->max_atomic_write_sectors = 0;
lim->max_dev_sectors = 0;
lim->chunk_sectors = 0;
lim->max_write_same_sectors = 0;
@@ -129,6 +130,7 @@ void blk_set_stacking_limits(struct queue_limits *lim)
lim->discard_zeroes_data = 1;
lim->max_segments = USHRT_MAX;
lim->max_hw_sectors = UINT_MAX;
+ lim->max_atomic_write_sectors = 0;
lim->max_segment_size = UINT_MAX;
lim->max_sectors = UINT_MAX;
lim->max_dev_sectors = UINT_MAX;
@@ -258,6 +260,24 @@ void blk_queue_max_hw_sectors(struct request_queue *q, unsigned int max_hw_secto
EXPORT_SYMBOL(blk_queue_max_hw_sectors);
/**
+ * blk_queue_max_atomic_write_sectors - maximum sectors written atomically
+ * @q: the request queue for the device
+ * @max_hw_sectors: max hardware sectors in the usual 512b unit
+ *
+ * Description:
+ * Enables a low level driver to advertise that it supports writing
+ * multi-sector I/O atomically. If the driver has any requirements
+ * in addition to the maximum size it should not set this field to
+ * indicate that it supports multi-sector atomic writes.
+ **/
+void blk_queue_max_atomic_write_sectors(struct request_queue *q,
+ unsigned int max_atomic_write_sectors)
+{
+ q->limits.max_atomic_write_sectors = max_atomic_write_sectors;
+}
+EXPORT_SYMBOL_GPL(blk_queue_max_atomic_write_sectors);
+
+/**
* blk_queue_chunk_sectors - set size of the chunk for this queue
* @q: the request queue for the device
* @chunk_sectors: chunk sectors in the usual 512b unit
@@ -541,6 +561,8 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
t->max_sectors = min_not_zero(t->max_sectors, b->max_sectors);
t->max_hw_sectors = min_not_zero(t->max_hw_sectors, b->max_hw_sectors);
t->max_dev_sectors = min_not_zero(t->max_dev_sectors, b->max_dev_sectors);
+ /* no support for stacking atomic writes */
+ t->max_atomic_write_sectors = 0;
t->max_write_same_sectors = min(t->max_write_same_sectors,
b->max_write_same_sectors);
t->max_write_zeroes_sectors = min(t->max_write_zeroes_sectors,
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 1dbce057592d..2f39009731f6 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -249,6 +249,12 @@ static ssize_t queue_max_hw_sectors_show(struct request_queue *q, char *page)
return queue_var_show(max_hw_sectors_kb, (page));
}
+static ssize_t queue_max_atomic_write_sectors_show(struct request_queue *q,
+ char *page)
+{
+ return queue_var_show(queue_max_atomic_write_sectors(q) << 1, page);
+}
+
#define QUEUE_SYSFS_BIT_FNS(name, flag, neg) \
static ssize_t \
queue_show_##name(struct request_queue *q, char *page) \
@@ -540,6 +546,11 @@ static struct queue_sysfs_entry queue_max_hw_sectors_entry = {
.show = queue_max_hw_sectors_show,
};
+static struct queue_sysfs_entry queue_max_atomic_write_sectors_entry = {
+ .attr = {.name = "max_atomic_write_sectors_kb", .mode = S_IRUGO },
+ .show = queue_max_atomic_write_sectors_show,
+};
+
static struct queue_sysfs_entry queue_max_segments_entry = {
.attr = {.name = "max_segments", .mode = S_IRUGO },
.show = queue_max_segments_show,
@@ -695,6 +706,7 @@ static struct attribute *default_attrs[] = {
&queue_requests_entry.attr,
&queue_ra_entry.attr,
&queue_max_hw_sectors_entry.attr,
+ &queue_max_atomic_write_sectors_entry.attr,
&queue_max_sectors_entry.attr,
&queue_max_segments_entry.attr,
&queue_max_integrity_segments_entry.attr,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 1ca8e8fd1078..c43d952557f9 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -323,6 +323,7 @@ struct queue_limits {
unsigned int alignment_offset;
unsigned int io_min;
unsigned int io_opt;
+ unsigned int max_atomic_write_sectors;
unsigned int max_discard_sectors;
unsigned int max_hw_discard_sectors;
unsigned int max_write_same_sectors;
@@ -1135,6 +1136,8 @@ extern void blk_cleanup_queue(struct request_queue *);
extern void blk_queue_make_request(struct request_queue *, make_request_fn *);
extern void blk_queue_bounce_limit(struct request_queue *, u64);
extern void blk_queue_max_hw_sectors(struct request_queue *, unsigned int);
+extern void blk_queue_max_atomic_write_sectors(struct request_queue *,
+ unsigned int);
extern void blk_queue_chunk_sectors(struct request_queue *, unsigned int);
extern void blk_queue_max_segments(struct request_queue *, unsigned short);
extern void blk_queue_max_segment_size(struct request_queue *, unsigned int);
@@ -1371,6 +1374,12 @@ static inline unsigned int queue_max_hw_sectors(struct request_queue *q)
return q->limits.max_hw_sectors;
}
+static inline unsigned int queue_max_atomic_write_sectors(
+ struct request_queue *q)
+{
+ return q->limits.max_atomic_write_sectors;
+}
+
static inline unsigned short queue_max_segments(struct request_queue *q)
{
return q->limits.max_segments;
--
2.11.0
^ permalink raw reply related
* [PATCH 08/12] xfs: implement the F_IOINFO fcntl
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
In-Reply-To: <20170228145737.19016-1-hch@lst.de>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
fs/xfs/xfs_file.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index a7d8324b59c5..4d955b3266df 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -898,6 +898,39 @@ xfs_file_dedupe_range(
return len;
}
+static uint16_t
+xfs_dio_alignment(struct file *file)
+{
+ struct xfs_inode *ip = XFS_I(file_inode(file));
+ struct xfs_mount *mp = ip->i_mount;
+
+ if (file->f_flags & O_ATOMIC)
+ return mp->m_sb.sb_blocksize;
+ else if (XFS_IS_REALTIME_INODE(ip))
+ return mp->m_rtdev_targp->bt_logical_sectorsize;
+ else
+ return mp->m_ddev_targp->bt_logical_sectorsize;
+}
+
+static int
+xfs_file_ioinfo(
+ struct file *file,
+ struct fcntl_ioinfo *fio)
+{
+ if (file->f_flags & O_DIRECT)
+ fio->fio_alignment = xfs_dio_alignment(file);
+
+ if (file->f_flags & O_ATOMIC) {
+ fio->fio_flags = FIO_FL_ATOMIC_OSYNC | FIO_FL_ATOMIC_FSYNC;
+ fio->fio_max_atomic = INT_MAX;
+ if (fio->fio_alignment)
+ fio->fio_max_atomic &= ~(fio->fio_alignment - 1);
+ }
+
+
+ return 0;
+};
+
STATIC int
xfs_file_open(
struct inode *inode,
@@ -1556,6 +1589,7 @@ const struct file_operations xfs_file_operations = {
.fallocate = xfs_file_fallocate,
.clone_file_range = xfs_file_clone_range,
.dedupe_file_range = xfs_file_dedupe_range,
+ .ioinfo = xfs_file_ioinfo,
};
const struct file_operations xfs_dir_file_operations = {
--
2.11.0
^ permalink raw reply related
* [PATCH 03/12] iomap: add a IOMAP_ATOMIC flag
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
In-Reply-To: <20170228145737.19016-1-hch@lst.de>
To pass through O_ATOMIC to the iomap_begin methods.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
fs/iomap.c | 13 +++++++++++--
include/linux/iomap.h | 1 +
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/fs/iomap.c b/fs/iomap.c
index 16a9d2b89cb6..096cbf573932 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -237,6 +237,10 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *iter,
{
struct inode *inode = iocb->ki_filp->f_mapping->host;
loff_t pos = iocb->ki_pos, ret = 0, written = 0;
+ unsigned flags = IOMAP_WRITE;
+
+ if (iocb->ki_filp->f_flags & O_ATOMIC)
+ flags |= IOMAP_ATOMIC;
while (iov_iter_count(iter)) {
ret = iomap_apply(inode, pos, iov_iter_count(iter),
@@ -452,8 +456,12 @@ int iomap_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf,
struct inode *inode = file_inode(vma->vm_file);
unsigned long length;
loff_t offset, size;
+ unsigned flags = IOMAP_WRITE | IOMAP_FAULT;
ssize_t ret;
+ if (vma->vm_file->f_flags & O_ATOMIC)
+ flags |= IOMAP_ATOMIC;
+
lock_page(page);
size = i_size_read(inode);
if ((page->mapping != inode->i_mapping) ||
@@ -471,8 +479,7 @@ int iomap_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf,
offset = page_offset(page);
while (length > 0) {
- ret = iomap_apply(inode, offset, length,
- IOMAP_WRITE | IOMAP_FAULT, ops, page,
+ ret = iomap_apply(inode, offset, length, flags, ops, page,
iomap_page_mkwrite_actor);
if (unlikely(ret <= 0))
goto out_unlock;
@@ -883,6 +890,8 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
} else {
dio->flags |= IOMAP_DIO_WRITE;
flags |= IOMAP_WRITE;
+ if (iocb->ki_filp->f_flags & O_ATOMIC)
+ flags |= IOMAP_ATOMIC;
}
if (mapping->nrpages) {
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index 891459caa278..a670ff18ccd6 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -51,6 +51,7 @@ struct iomap {
#define IOMAP_REPORT (1 << 2) /* report extent status, e.g. FIEMAP */
#define IOMAP_FAULT (1 << 3) /* mapping for page fault */
#define IOMAP_DIRECT (1 << 4) /* direct I/O */
+#define IOMAP_ATOMIC (1 << 5) /* atomic write vs power fail */
struct iomap_ops {
/*
--
2.11.0
^ permalink raw reply related
* [RFC] failure atomic writes for file systems and block devices
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
Hi all,
this series implements a new O_ATOMIC flag for failure atomic writes
to files. It is based on and tries to unify to earlier proposals,
the first one for block devices by Chris Mason:
https://lwn.net/Articles/573092/
and the second one for regular files, published by HP Research at
Usenix FAST 2015:
https://www.usenix.org/conference/fast15/technical-sessions/presentation/verma
It adds a new O_ATOMIC flag for open, which requests writes to be
failure-atomic, that is either the whole write makes it to persistent
storage, or none of it, even in case of power of other failures.
There are two implementation various of this: on block devices O_ATOMIC
must be combined with O_(D)SYNC so that storage devices that can handle
large writes atomically can simply do that without any additional work.
This case is supported by NVMe.
The second case is for file systems, where we simply write new blocks
out of places and then remap them into the file atomically on either
completion of an O_(D)SYNC write or when fsync is called explicitly.
The semantics of the latter case are explained in detail at the Usenix
paper above.
Last but not least a new fcntl is implemented to provide information
about I/O restrictions such as alignment requirements and the maximum
atomic write size.
The implementation is simple and clean, but I'm rather unhappy about
the interface as it has too many failure modes to bullet proof. For
one old kernels ignore unknown open flags silently, so applications
have to check the F_IOINFO fcntl before, which is a bit of a killer.
Because of that I've also not implemented any other validity checks
yet, as they might make thing even worse when an open on a not supported
file system or device fails, but not on an old kernel. Maybe we need
a new open version that checks arguments properly first?
Also I'm really worried about the NVMe failure modes - devices simply
advertise an atomic write size, with no way for the device to know
that the host requested a given write to be atomic, and thus no
error reporting. This is made worse by NVMe 1.2 adding per-namespace
atomic I/O parameters that devices can use to introduce additional
odd alignment quirks - while there is some language in the spec
requiring them not to weaken the per-controller guarantees it all
looks rather weak and I'm not too confident in all implementations
getting everything right.
Last but not least this depends on a few XFS patches, so to actually
apply / run the patches please use this git tree:
git://git.infradead.org/users/hch/vfs.git O_ATOMIC
Gitweb:
http://git.infradead.org/users/hch/vfs.git/shortlog/refs/heads/O_ATOMIC
^ permalink raw reply
* [PATCH 02/12] iomap: pass IOMAP_* flags to actors
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
In-Reply-To: <20170228145737.19016-1-hch@lst.de>
This will be needed to implement O_ATOMIC.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
fs/dax.c | 2 +-
fs/internal.h | 2 +-
fs/iomap.c | 39 +++++++++++++++++++++------------------
3 files changed, 23 insertions(+), 20 deletions(-)
diff --git a/fs/dax.c b/fs/dax.c
index 78b9651576c6..5d71fc5f0a08 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -997,7 +997,7 @@ static sector_t dax_iomap_sector(struct iomap *iomap, loff_t pos)
static loff_t
dax_iomap_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
+ struct iomap *iomap, unsigned flags)
{
struct iov_iter *iter = data;
loff_t end = pos + length, done = 0;
diff --git a/fs/internal.h b/fs/internal.h
index 11c6d89dce9c..1934fdb2bb27 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -179,7 +179,7 @@ extern long vfs_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
* iomap support:
*/
typedef loff_t (*iomap_actor_t)(struct inode *inode, loff_t pos, loff_t len,
- void *data, struct iomap *iomap);
+ void *data, struct iomap *iomap, unsigned flags);
loff_t iomap_apply(struct inode *inode, loff_t pos, loff_t length,
unsigned flags, const struct iomap_ops *ops, void *data,
diff --git a/fs/iomap.c b/fs/iomap.c
index 7f08ca03d95d..16a9d2b89cb6 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -76,7 +76,7 @@ iomap_apply(struct inode *inode, loff_t pos, loff_t length, unsigned flags,
* we can do the copy-in page by page without having to worry about
* failures exposing transient data.
*/
- written = actor(inode, pos, length, data, &iomap);
+ written = actor(inode, pos, length, data, &iomap, flags);
/*
* Now the data has been copied, commit the range we've copied. This
@@ -105,8 +105,9 @@ iomap_write_failed(struct inode *inode, loff_t pos, unsigned len)
}
static int
-iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags,
- struct page **pagep, struct iomap *iomap)
+iomap_write_begin(struct inode *inode, loff_t pos, unsigned len,
+ unsigned aop_flags, struct page **pagep, struct iomap *iomap,
+ unsigned flags)
{
pgoff_t index = pos >> PAGE_SHIFT;
struct page *page;
@@ -114,7 +115,7 @@ iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags,
BUG_ON(pos + len > iomap->offset + iomap->length);
- page = grab_cache_page_write_begin(inode->i_mapping, index, flags);
+ page = grab_cache_page_write_begin(inode->i_mapping, index, aop_flags);
if (!page)
return -ENOMEM;
@@ -146,18 +147,18 @@ iomap_write_end(struct inode *inode, loff_t pos, unsigned len,
static loff_t
iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
+ struct iomap *iomap, unsigned flags)
{
struct iov_iter *i = data;
long status = 0;
ssize_t written = 0;
- unsigned int flags = AOP_FLAG_NOFS;
+ unsigned int aop_flags = AOP_FLAG_NOFS;
/*
* Copies from kernel address space cannot fail (NFSD is a big user).
*/
if (!iter_is_iovec(i))
- flags |= AOP_FLAG_UNINTERRUPTIBLE;
+ aop_flags |= AOP_FLAG_UNINTERRUPTIBLE;
do {
struct page *page;
@@ -187,8 +188,8 @@ iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
break;
}
- status = iomap_write_begin(inode, pos, bytes, flags, &page,
- iomap);
+ status = iomap_write_begin(inode, pos, bytes, aop_flags, &page,
+ iomap, flags);
if (unlikely(status))
break;
@@ -268,7 +269,7 @@ __iomap_read_page(struct inode *inode, loff_t offset)
static loff_t
iomap_dirty_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
+ struct iomap *iomap, unsigned flags)
{
long status = 0;
ssize_t written = 0;
@@ -287,7 +288,7 @@ iomap_dirty_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
status = iomap_write_begin(inode, pos, bytes,
AOP_FLAG_NOFS | AOP_FLAG_UNINTERRUPTIBLE,
- &page, iomap);
+ &page, iomap, flags);
put_page(rpage);
if (unlikely(status))
return status;
@@ -333,13 +334,14 @@ iomap_file_dirty(struct inode *inode, loff_t pos, loff_t len,
EXPORT_SYMBOL_GPL(iomap_file_dirty);
static int iomap_zero(struct inode *inode, loff_t pos, unsigned offset,
- unsigned bytes, struct iomap *iomap)
+ unsigned bytes, struct iomap *iomap, unsigned flags)
{
struct page *page;
int status;
status = iomap_write_begin(inode, pos, bytes,
- AOP_FLAG_UNINTERRUPTIBLE | AOP_FLAG_NOFS, &page, iomap);
+ AOP_FLAG_UNINTERRUPTIBLE | AOP_FLAG_NOFS, &page, iomap,
+ flags);
if (status)
return status;
@@ -360,7 +362,7 @@ static int iomap_dax_zero(loff_t pos, unsigned offset, unsigned bytes,
static loff_t
iomap_zero_range_actor(struct inode *inode, loff_t pos, loff_t count,
- void *data, struct iomap *iomap)
+ void *data, struct iomap *iomap, unsigned flags)
{
bool *did_zero = data;
loff_t written = 0;
@@ -379,7 +381,8 @@ iomap_zero_range_actor(struct inode *inode, loff_t pos, loff_t count,
if (IS_DAX(inode))
status = iomap_dax_zero(pos, offset, bytes, iomap);
else
- status = iomap_zero(inode, pos, offset, bytes, iomap);
+ status = iomap_zero(inode, pos, offset, bytes, iomap,
+ flags);
if (status < 0)
return status;
@@ -429,7 +432,7 @@ EXPORT_SYMBOL_GPL(iomap_truncate_page);
static loff_t
iomap_page_mkwrite_actor(struct inode *inode, loff_t pos, loff_t length,
- void *data, struct iomap *iomap)
+ void *data, struct iomap *iomap, unsigned flags)
{
struct page *page = data;
int ret;
@@ -521,7 +524,7 @@ static int iomap_to_fiemap(struct fiemap_extent_info *fi,
static loff_t
iomap_fiemap_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
+ struct iomap *iomap, unsigned flags)
{
struct fiemap_ctx *ctx = data;
loff_t ret = length;
@@ -730,7 +733,7 @@ iomap_dio_zero(struct iomap_dio *dio, struct iomap *iomap, loff_t pos,
static loff_t
iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
- void *data, struct iomap *iomap)
+ void *data, struct iomap *iomap, unsigned flags)
{
struct iomap_dio *dio = data;
unsigned blkbits = blksize_bits(bdev_logical_block_size(iomap->bdev));
--
2.11.0
^ permalink raw reply related
* [PATCH 07/12] xfs: implement failure-atomic writes
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
In-Reply-To: <20170228145737.19016-1-hch@lst.de>
If O_ATOMIC is specified in the open flags this will cause XFS to
allocate new extents in the COW for even if overwriting existing data,
and not remap them into the data fork until ->fsync is called,
at which point the whole range will be atomically remapped into the
data fork. This allows applications to ѕafely overwrite data instead
of having to do double writes.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
fs/xfs/xfs_aops.c | 18 +++++++++-----
fs/xfs/xfs_aops.h | 4 ++-
fs/xfs/xfs_file.c | 17 +++++++++++++
fs/xfs/xfs_iomap.c | 18 ++++++++------
fs/xfs/xfs_reflink.c | 69 ++++++++++++++++++++++++++++++++++------------------
fs/xfs/xfs_reflink.h | 5 ++--
6 files changed, 91 insertions(+), 40 deletions(-)
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index c78b585b3d84..1c5efbb05b47 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -292,6 +292,7 @@ xfs_end_io(
if (unlikely(error)) {
switch (ioend->io_type) {
case XFS_IO_COW:
+ case XFS_IO_ATOMIC:
xfs_reflink_cancel_cow_range(ip, offset, size, 0);
break;
}
@@ -327,7 +328,9 @@ xfs_end_bio(
struct xfs_ioend *ioend = bio->bi_private;
struct xfs_mount *mp = XFS_I(ioend->io_inode)->i_mount;
- if (ioend->io_type == XFS_IO_UNWRITTEN || ioend->io_type == XFS_IO_COW)
+ if (ioend->io_type == XFS_IO_UNWRITTEN ||
+ ioend->io_type == XFS_IO_COW ||
+ ioend->io_type == XFS_IO_ATOMIC)
queue_work(mp->m_unwritten_workqueue, &ioend->io_work);
else if (ioend->io_append_trans)
queue_work(mp->m_data_workqueue, &ioend->io_work);
@@ -354,6 +357,7 @@ xfs_map_blocks(
return -EIO;
ASSERT(type != XFS_IO_COW);
+ ASSERT(type != XFS_IO_ATOMIC);
if (type == XFS_IO_UNWRITTEN)
bmapi_flags |= XFS_BMAPI_IGSTATE;
@@ -768,7 +772,8 @@ xfs_map_cow(
struct xfs_writepage_ctx *wpc,
struct inode *inode,
loff_t offset,
- unsigned int *new_type)
+ unsigned int *new_type,
+ bool atomic)
{
struct xfs_inode *ip = XFS_I(inode);
struct xfs_bmbt_irec imap;
@@ -778,10 +783,10 @@ xfs_map_cow(
/*
* If we already have a valid COW mapping keep using it.
*/
- if (wpc->io_type == XFS_IO_COW) {
+ if (wpc->io_type == XFS_IO_COW || wpc->io_type == XFS_IO_ATOMIC) {
wpc->imap_valid = xfs_imap_valid(inode, &wpc->imap, offset);
if (wpc->imap_valid) {
- *new_type = XFS_IO_COW;
+ *new_type = wpc->io_type;
return 0;
}
}
@@ -807,7 +812,7 @@ xfs_map_cow(
return error;
}
- wpc->io_type = *new_type = XFS_IO_COW;
+ wpc->io_type = *new_type = atomic ? XFS_IO_ATOMIC : XFS_IO_COW;
wpc->imap_valid = true;
wpc->imap = imap;
return 0;
@@ -886,7 +891,8 @@ xfs_writepage_map(
}
if (XFS_I(inode)->i_cowfp) {
- error = xfs_map_cow(wpc, inode, offset, &new_type);
+ error = xfs_map_cow(wpc, inode, offset, &new_type,
+ buffer_atomic(bh));
if (error)
goto out;
}
diff --git a/fs/xfs/xfs_aops.h b/fs/xfs/xfs_aops.h
index cc174ec6c2fd..798e653e68b6 100644
--- a/fs/xfs/xfs_aops.h
+++ b/fs/xfs/xfs_aops.h
@@ -29,6 +29,7 @@ enum {
XFS_IO_UNWRITTEN, /* covers allocated but uninitialized data */
XFS_IO_OVERWRITE, /* covers already allocated extent */
XFS_IO_COW, /* covers copy-on-write extent */
+ XFS_IO_ATOMIC, /* atomic write */
};
#define XFS_IO_TYPES \
@@ -36,7 +37,8 @@ enum {
{ XFS_IO_DELALLOC, "delalloc" }, \
{ XFS_IO_UNWRITTEN, "unwritten" }, \
{ XFS_IO_OVERWRITE, "overwrite" }, \
- { XFS_IO_COW, "CoW" }
+ { XFS_IO_COW, "CoW" }, \
+ { XFS_IO_ATOMIC, "atomic" }
/*
* Structure for buffered I/O completions.
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 086440e79b86..a7d8324b59c5 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -160,6 +160,12 @@ xfs_file_fsync(
else if (mp->m_logdev_targp != mp->m_ddev_targp)
xfs_blkdev_issue_flush(mp->m_ddev_targp);
+ if (file->f_flags & O_ATOMIC) {
+ error = xfs_reflink_end_cow(ip, start, end - start + 1);
+ if (error)
+ return error;
+ }
+
/*
* All metadata updates are logged, which means that we just have to
* flush the log up to the latest LSN that touched the inode. If we have
@@ -457,6 +463,9 @@ xfs_dio_write_end_io(
}
spin_unlock(&ip->i_flags_lock);
+ if (iocb->ki_filp->f_flags & O_ATOMIC)
+ return 0;
+
if (flags & IOMAP_DIO_COW) {
error = xfs_reflink_end_cow(ip, offset, size);
if (error)
@@ -529,6 +538,12 @@ xfs_file_dio_aio_write(
unaligned_io = 1;
/*
+ * We need filesystem block alignment to provide atomic commits.
+ */
+ if (file->f_flags & O_ATOMIC)
+ return -EINVAL;
+
+ /*
* We can't properly handle unaligned direct I/O to reflink
* files yet, as we can't unshare a partial block.
*/
@@ -892,6 +907,8 @@ xfs_file_open(
return -EFBIG;
if (XFS_FORCED_SHUTDOWN(XFS_M(inode->i_sb)))
return -EIO;
+ if (file->f_flags & O_ATOMIC)
+ printk_ratelimited("O_ATOMIC!\n");
return 0;
}
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 5d68b4279016..b686a6bd2db4 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -559,13 +559,14 @@ xfs_file_iomap_begin_delay(
eof = !xfs_iext_lookup_extent(ip, ifp, offset_fsb, &idx, &got);
if (!eof && got.br_startoff <= offset_fsb) {
- if (xfs_is_reflink_inode(ip)) {
+ if ((flags & IOMAP_ATOMIC) || xfs_is_reflink_inode(ip)) {
bool shared;
end_fsb = min(XFS_B_TO_FSB(mp, offset + count),
maxbytes_fsb);
xfs_trim_extent(&got, offset_fsb, end_fsb - offset_fsb);
- error = xfs_reflink_reserve_cow(ip, &got, &shared);
+ error = xfs_reflink_reserve_cow(ip, &got, &shared,
+ (flags & IOMAP_ATOMIC));
if (error)
goto out_unlock;
}
@@ -951,7 +952,7 @@ static inline bool need_excl_ilock(struct xfs_inode *ip, unsigned flags)
*/
if (xfs_is_reflink_inode(ip) && (flags & (IOMAP_WRITE | IOMAP_ZERO)))
return true;
- if ((flags & IOMAP_DIRECT) && (flags & IOMAP_WRITE))
+ if ((flags & (IOMAP_DIRECT | IOMAP_ATOMIC)) && (flags & IOMAP_WRITE))
return true;
return false;
}
@@ -976,7 +977,8 @@ xfs_file_iomap_begin(
return -EIO;
if (((flags & (IOMAP_WRITE | IOMAP_DIRECT)) == IOMAP_WRITE) &&
- !IS_DAX(inode) && !xfs_get_extsz_hint(ip)) {
+ ((flags & IOMAP_ATOMIC) ||
+ (!IS_DAX(inode) && !xfs_get_extsz_hint(ip)))) {
/* Reserve delalloc blocks for regular writeback. */
return xfs_file_iomap_begin_delay(inode, offset, length, flags,
iomap);
@@ -1008,15 +1010,17 @@ xfs_file_iomap_begin(
goto out_unlock;
}
- if ((flags & (IOMAP_WRITE | IOMAP_ZERO)) && xfs_is_reflink_inode(ip)) {
+ if ((flags & (IOMAP_WRITE | IOMAP_ZERO)) &&
+ ((flags & IOMAP_ATOMIC) || xfs_is_reflink_inode(ip))) {
if (flags & IOMAP_DIRECT) {
/* may drop and re-acquire the ilock */
error = xfs_reflink_allocate_cow(ip, &imap, &shared,
- &lockmode);
+ &lockmode, flags & IOMAP_ATOMIC);
if (error)
goto out_unlock;
} else {
- error = xfs_reflink_reserve_cow(ip, &imap, &shared);
+ error = xfs_reflink_reserve_cow(ip, &imap, &shared,
+ false);
if (error)
goto out_unlock;
}
diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c
index 4225b5e67b17..4702dd800ab8 100644
--- a/fs/xfs/xfs_reflink.c
+++ b/fs/xfs/xfs_reflink.c
@@ -264,9 +264,9 @@ int
xfs_reflink_reserve_cow(
struct xfs_inode *ip,
struct xfs_bmbt_irec *imap,
- bool *shared)
+ bool *shared,
+ bool always_cow)
{
- struct xfs_ifork *ifp = XFS_IFORK_PTR(ip, XFS_COW_FORK);
struct xfs_bmbt_irec got;
int error = 0;
bool eof = false, trimmed;
@@ -280,26 +280,30 @@ xfs_reflink_reserve_cow(
* extent list is generally faster than going out to the shared extent
* tree.
*/
-
- if (!xfs_iext_lookup_extent(ip, ifp, imap->br_startoff, &idx, &got))
+ if (!ip->i_cowfp) {
+ ASSERT(always_cow);
+ xfs_ifork_init_cow(ip);
eof = true;
- if (!eof && got.br_startoff <= imap->br_startoff) {
- trace_xfs_reflink_cow_found(ip, imap);
- xfs_trim_extent(imap, got.br_startoff, got.br_blockcount);
+ } else {
+ if (!xfs_iext_lookup_extent(ip, ip->i_cowfp, imap->br_startoff,
+ &idx, &got))
+ eof = true;
+ if (!eof && got.br_startoff <= imap->br_startoff) {
+ trace_xfs_reflink_cow_found(ip, imap);
+ xfs_trim_extent(imap, got.br_startoff,
+ got.br_blockcount);
+
+ *shared = true;
+ return 0;
+ }
- *shared = true;
- return 0;
+ /* Trim the mapping to the nearest shared extent boundary. */
+ error = xfs_reflink_trim_around_shared(ip, imap, shared,
+ &trimmed);
+ if (error || !*shared)
+ return error;
}
- /* Trim the mapping to the nearest shared extent boundary. */
- error = xfs_reflink_trim_around_shared(ip, imap, shared, &trimmed);
- if (error)
- return error;
-
- /* Not shared? Just report the (potentially capped) extent. */
- if (!*shared)
- return 0;
-
/*
* Fork all the shared blocks from our write offset until the end of
* the extent.
@@ -383,7 +387,8 @@ xfs_reflink_allocate_cow(
struct xfs_inode *ip,
struct xfs_bmbt_irec *imap,
bool *shared,
- uint *lockmode)
+ uint *lockmode,
+ bool always_cow)
{
struct xfs_mount *mp = ip->i_mount;
xfs_fileoff_t offset_fsb = imap->br_startoff;
@@ -399,15 +404,19 @@ xfs_reflink_allocate_cow(
xfs_extnum_t idx;
retry:
- ASSERT(xfs_is_reflink_inode(ip));
+ ASSERT(always_cow | xfs_is_reflink_inode(ip));
ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED));
+ if (!ip->i_cowfp) {
+ ASSERT(always_cow);
+ xfs_ifork_init_cow(ip);
+
/*
* Even if the extent is not shared we might have a preallocation for
* it in the COW fork. If so use it.
*/
- if (xfs_iext_lookup_extent(ip, ip->i_cowfp, offset_fsb, &idx, &got) &&
- got.br_startoff <= offset_fsb) {
+ } else if (xfs_iext_lookup_extent(ip, ip->i_cowfp, offset_fsb, &idx,
+ &got) && got.br_startoff <= offset_fsb) {
*shared = true;
/* If we have a real allocation in the COW fork we're done. */
@@ -418,7 +427,7 @@ xfs_reflink_allocate_cow(
}
xfs_trim_extent(imap, got.br_startoff, got.br_blockcount);
- } else {
+ } else if (!always_cow) {
error = xfs_reflink_trim_around_shared(ip, imap, shared, &trimmed);
if (error || !*shared)
goto out;
@@ -684,6 +693,7 @@ xfs_reflink_end_cow(
xfs_fileoff_t offset_fsb;
xfs_fileoff_t end_fsb;
xfs_fsblock_t firstfsb;
+ xfs_off_t new_size;
struct xfs_defer_ops dfops;
int error;
unsigned int resblks;
@@ -693,7 +703,7 @@ xfs_reflink_end_cow(
trace_xfs_reflink_end_cow(ip, offset, count);
/* No COW extents? That's easy! */
- if (ifp->if_bytes == 0)
+ if (!ifp || ifp->if_bytes == 0)
return 0;
offset_fsb = XFS_B_TO_FSBT(ip->i_mount, offset);
@@ -776,6 +786,17 @@ xfs_reflink_end_cow(
break;
}
+ /*
+ * Update the on-disk inode size if we completed an operation outside
+ * of the inode size. This can only happen for atomic writes, and not
+ * for actual reflinked files.
+ */
+ new_size = xfs_new_eof(ip, offset + count);
+ if (new_size) {
+ ip->i_d.di_size = new_size;
+ xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+ }
+
error = xfs_trans_commit(tp);
xfs_iunlock(ip, XFS_ILOCK_EXCL);
if (error)
diff --git a/fs/xfs/xfs_reflink.h b/fs/xfs/xfs_reflink.h
index 9416279b3c89..0360e2c0f3a5 100644
--- a/fs/xfs/xfs_reflink.h
+++ b/fs/xfs/xfs_reflink.h
@@ -27,9 +27,10 @@ extern int xfs_reflink_trim_around_shared(struct xfs_inode *ip,
struct xfs_bmbt_irec *irec, bool *shared, bool *trimmed);
extern int xfs_reflink_reserve_cow(struct xfs_inode *ip,
- struct xfs_bmbt_irec *imap, bool *shared);
+ struct xfs_bmbt_irec *imap, bool *shared, bool always_cow);
extern int xfs_reflink_allocate_cow(struct xfs_inode *ip,
- struct xfs_bmbt_irec *imap, bool *shared, uint *lockmode);
+ struct xfs_bmbt_irec *imap, bool *shared, uint *lockmode,
+ bool always_cow);
extern int xfs_reflink_convert_cow(struct xfs_inode *ip, xfs_off_t offset,
xfs_off_t count);
extern bool xfs_reflink_find_cow_mapping(struct xfs_inode *ip, xfs_off_t offset,
--
2.11.0
^ permalink raw reply related
* [PATCH 10/12] block_dev: set REQ_NOMERGE for O_ATOMIC writes
From: Christoph Hellwig @ 2017-02-28 14:57 UTC (permalink / raw)
To: linux-fsdevel, linux-xfs, linux-block
In-Reply-To: <20170228145737.19016-1-hch@lst.de>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
fs/block_dev.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 3c47614a4b32..4dd5c54cdefb 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -242,6 +242,10 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
task_io_account_write(ret);
}
+ /* don't merge atomic requests to avoid going over the limit */
+ if (iocb->ki_filp->f_flags & O_ATOMIC)
+ bio.bi_opf |= REQ_NOMERGE;
+
qc = submit_bio(&bio);
for (;;) {
set_current_state(TASK_UNINTERRUPTIBLE);
@@ -377,6 +381,10 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, int nr_pages)
task_io_account_write(bio->bi_iter.bi_size);
}
+ /* don't merge atomic requests to avoid going over the limit */
+ if (iocb->ki_filp->f_flags & O_ATOMIC)
+ bio->bi_opf |= REQ_NOMERGE;
+
dio->size += bio->bi_iter.bi_size;
pos += bio->bi_iter.bi_size;
--
2.11.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox