* Re: [PATCH 05/24] fs: Get proper reference for s_bdi
From: Jan Kara @ 2017-02-09 15:52 UTC (permalink / raw)
To: Boaz Harrosh; +Cc: Jan Kara, linux-fsdevel, Christoph Hellwig, linux-block
In-Reply-To: <d671b948-9449-b0df-27ac-79a768077a64@plexistor.com>
On Thu 09-02-17 16:36:13, Boaz Harrosh wrote:
> On 02/02/2017 07:34 PM, Jan Kara wrote:
> > So far we just relied on block device to hold a bdi reference for us
> > while the filesystem is mounted. While that works perfectly fine, it is
> > a bit awkward that we have a pointer to a refcounted structure in the
> > superblock without proper reference. So make s_bdi hold a proper
> > reference to block device's BDI. No filesystem using mount_bdev()
> > actually changes s_bdi so this is safe and will make bdev filesystems
> > work the same way as filesystems needing to set up their private bdi.
> >
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > ---
> > fs/super.c | 7 ++-----
> > 1 file changed, 2 insertions(+), 5 deletions(-)
> >
> > diff --git a/fs/super.c b/fs/super.c
> > index 31dc4c6450ef..dfb95ccd4351 100644
> > --- a/fs/super.c
> > +++ b/fs/super.c
> > @@ -1047,12 +1047,9 @@ static int set_bdev_super(struct super_block *s, void *data)
> > {
> > s->s_bdev = data;
> > s->s_dev = s->s_bdev->bd_dev;
> > + s->s_bdi = bdi_get(s->s_bdev->bd_bdi);
> > + s->s_iflags |= SB_I_DYNBDI;
> >
> > - /*
> > - * We set the bdi here to the queue backing, file systems can
> > - * overwrite this in ->fill_super()
> > - */
>
> Question: So I have an FS that uses mount_bdev but than goes and
> overrides sb->s_bdev in ->fill_super() anyway. This is because of two
> reasons. One because I have many more devices. (like btrfs I'm
> moulti-dev) but I like to use mount_bdev because of the somewhat delicate
> handling of automatic bind-mounts.
>
> For me it is a bigger hack to get the ref-counting and bind-mounts
> locking correctly then to bdi_put and say the new super_setup_bdi(sb) in
> fill_super. Would you expect problems?
No, that should work just fine.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply
* Re: [PATCH 0/10] block: Fix block device shutdown related races
From: Jan Kara @ 2017-02-09 15:48 UTC (permalink / raw)
To: Thiago Jung Bauermann
Cc: Jan Kara, Jens Axboe, linux-block, Christoph Hellwig, Tejun Heo,
Dan Williams, NeilBrown
In-Reply-To: <3243095.TlNvkxZ2GK@morokweng>
[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]
On Thu 09-02-17 12:52:47, Thiago Jung Bauermann wrote:
> Hello Jan,
>
> Am Donnerstag, 9. Februar 2017, 13:44:23 BRST schrieb Jan Kara:
> > People, please have a look at patches. The are mostly simple however the
> > interactions are rather complex so I may have missed something. Also I'm
> > happy for any additional testing these patches can get - I've stressed them
> > with Omar's script, tested memcg writeback, tested static (not udev managed)
> > device inodes.
>
> Thank you for these fixes. I will have them tested and report back how it
> goes.
>
> Can you tell which branch I should apply them on? I tried a number of branches
> in linux-block (and applied the bdi lifetime v3 patches if the branch didn't
> already had them) but this series either didn't apply or the build failed
> with:
>
> /home/bauermann/trabalho/src/linux-2.6.git/fs/block_dev.c: In function
> ‘bd_acquire’:
> /home/bauermann/trabalho/src/linux-2.6.git/fs/block_dev.c:1063:13: error:
> passing argument 1 of ‘bd_forget’ from incompatible pointer type [-
> Werror=incompatible-pointer-types]
> bd_forget(bdev);
> ^
> In file included from /home/bauermann/trabalho/src/linux-2.6.git/include/
> linux/device_cgroup.h:1:0,
> from /home/bauermann/trabalho/src/linux-2.6.git/fs/
> block_dev.c:14:
> /home/bauermann/trabalho/src/linux-2.6.git/include/linux/fs.h:2351:13: note:
> expected ‘struct inode *’ but argument is of type ‘struct block_device *’
> extern void bd_forget(struct inode *inode);
> ^
> cc1: some warnings being treated as errors
Indeed, I'm wondering how this could pass one of the tests I did... Hum.
Anyway thanks for spotting this and attached is a fixed up version of the
patch 3.
I've pushed out a branch with all BDI patches I have accumulated to
git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git bdi
It includes filesystem-bdi cleanup patches as well on top of these fixes.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
[-- Attachment #2: 0003-block-Revalidate-i_bdev-reference-in-bd_aquire.patch --]
[-- Type: text/x-patch, Size: 2001 bytes --]
>From aaf612333753b948a96aebe4a2f8066ed45ef164 Mon Sep 17 00:00:00 2001
From: Jan Kara <jack@suse.cz>
Date: Thu, 9 Feb 2017 12:16:30 +0100
Subject: [PATCH 03/10] block: Revalidate i_bdev reference in bd_aquire()
When a device gets removed, block device inode unhashed so that it is not
used anymore (bdget() will not find it anymore). Later when a new device
gets created with the same device number, we create new block device
inode. However there may be file system device inodes whose i_bdev still
points to the original block device inode and thus we get two active
block device inodes for the same device. They will share the same
gendisk so the only visible differences will be that page caches will
not be coherent and BDIs will be different (the old block device inode
still points to unregistered BDI).
Fix the problem by checking in bd_acquire() whether i_bdev still points
to active block device inode and re-lookup the block device if not. That
way any open of a block device happening after the old device has been
removed will get correct block device inode.
Signed-off-by: Jan Kara <jack@suse.cz>
---
fs/block_dev.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 601b71b76d7f..68e855fdce58 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -1043,13 +1043,22 @@ static struct block_device *bd_acquire(struct inode *inode)
spin_lock(&bdev_lock);
bdev = inode->i_bdev;
- if (bdev) {
+ if (bdev && !inode_unhashed(bdev->bd_inode)) {
bdgrab(bdev);
spin_unlock(&bdev_lock);
return bdev;
}
spin_unlock(&bdev_lock);
+ /*
+ * i_bdev references block device inode that was already shut down
+ * (corresponding device got removed). Remove the reference and look
+ * up block device inode again just in case new device got
+ * reestablished under the same device number.
+ */
+ if (bdev)
+ bd_forget(inode);
+
bdev = bdget(inode->i_rdev);
if (bdev) {
spin_lock(&bdev_lock);
--
2.10.2
^ permalink raw reply related
* Re: [PATCH 00/16] multiqueue for MMC/SD third try
From: Christoph Hellwig @ 2017-02-09 15:39 UTC (permalink / raw)
To: Linus Walleij
Cc: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente,
Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
> What immediately jumps out at you is that linear read/writes
> perform just as nicely or actually better with MQ than with the
> old block layer.
>
> What is amazing is that just a little randomness, such as the
> find . > /dev/null immediately seems to visibly regress with MQ.
> My best guess is that it is caused by the absence of the block
> scheduler.
The block tree now has a basic deadline scheduler. So for now I'd
suggest to kick off the discussion on all these prep patches,
and then retest with 4.11-rc1 or 2 that will have the scheduler.
^ permalink raw reply
* Re: [PATCH 1/1] dm-zoned: Zoned block device target
From: Mike Snitzer @ 2017-02-09 15:27 UTC (permalink / raw)
To: Damien Le Moal
Cc: dm-devel, Alasdair Kergon, Hannes Reinecke, Christoph Hellwig,
linux-block
In-Reply-To: <20170209041849.30964-1-damien.lemoal@wdc.com>
On Wed, Feb 08 2017 at 11:18pm -0500,
Damien Le Moal <damien.lemoal@wdc.com> wrote:
> The dm-zoned device mapper target provides transparent write access
> to zoned block devices (ZBC and ZAC compliant block devices).
> dm-zoned hides to the device user (a file system or an application
> doing raw block device accesses) any constraint imposed on write
> requests by the device. Write requests are processed using a
> combination of on-disk buffering using the device conventional zones,
> allowing any random write access to be safely executed, or direct in-place
> processing for requests aligned on a zone sequential write pointer position.
> A background reclaim process ensures that some conventional zones are always
> available for executing unaligned write requests. The reclaim process
> overhead is minimized by managing buffer zones in a least-recently-written
> order and first targeting the oldest buffer zones. Doing so, blocks under
> regular write access (such as metadata blocks of an FS) remain stored in
> conventional zones, resulting in no apparent write overhead.
>
> dm-zoned implementation focus on simplicity and on minimizing overhead
> (CPU, memory and storage overhead). For a 10TB host-managed disk with
> 256 MB zones, dm-zoned memory usage per disk instance is at most 4.5 MB
> and as little as 5 zones will be used internally for storing metadata
> and performing buffer zone reclaim operations. This is achieved using
> zone level indirection rather than a full block indirection system for
> managing block movement between zones.
>
> dm-zoned primary target is host-managed zoned block devices but it can also
> be used with host-aware device models to mitigate potential device-side
> performance degradation due to excessive random writing.
>
> dm-zoned backend devices can be formatted and checked using the dmzadm
> utility available at:
>
> https://github.com/hgst/dm-zoned-tools
>
> This patch applies on top of 4.10-rc7 tree.
>
> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Looks like this work has come a long way. While I am _still_ not
hearing from any customers (or partners) that SMR is a priority I do
acknowledge that that obviously isn't the case for WDC and others.
So, I'll sign up for reviewing this DM target, with a focus on DM
interface and implementation details, _after_ I get some technical
review (e.g. Reviewed-by) from people like Hannes and others more
invested in the SMR technologies. Basically I don't know enough about
the constraints of these devices to fully appreciate and catch some
issue in the particulars of the algorithms used to mitigate SMR's
quirks.
I'd also like to understand how invested WDC is in maintaining this DM
target for years and years if/when I were to merge this DM target into
mainline.
Thanks,
Mike
^ permalink raw reply
* [PATCH 16/16] RFC: mmc: switch MMC/SD to use blk-mq multiqueueing v3
From: Linus Walleij @ 2017-02-09 15:34 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
HACK ALERT: DO NOT MERGE THIS! IT IS A FYI PATCH FOR DISCUSSION
ONLY.
This is a totally new implementation of how to do multiqueue
in the MMC/SD stack. It is based on top of my refactorings in the
series which ends with this patch, and now makes proper use of
.init_request() and .exit_request() to initialize the per-request
hardware context, reusing good old struct mmc_queue_req which is
what is actually intended by these functions.
We kill off the kthread that was just calling blk_fetch_request()
and let blk-mq drive all traffic, nice, that is how it should work.
Due to having switched the submission mechanics around so that
the completion of requests is now triggered from the host
callbacks, we manage to keep the same performance for linear
reads/writes as we have for the old block layer.
Some open questions:
- We used to issue mmc_get_card() when the first request comes
in and mmc_put_card() when we get NULL from blk_fetch_request().
Well as we are not pushed any NULL requests anymore we need
another way for the queue to tell us it is idle, or we should
just set up a delayed work and release the card if no new
requests appear for some time.
- The flush was handled by issueing blk_end_request_all() in
the old scheduler. Is blk_mq_complete_request() really doing
the same job, or is there some extra magic needed here?
- We can sometime get a partial read from a MMC command, meaning
some of the request has been handled. We know how many bytes
were read/written. We used to report this to the block layer
using blk_end_request(old_req, 0, bytes_xfered) but the MQ
scheduler seems to be missing a command that reports
partial completion. How do we handle this?
Apart from that my only remaining worries are about the
block scheduler, but I hear Jens and Paolo are working to fix
this.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 66 ++++-----
drivers/mmc/core/core.c | 4 -
drivers/mmc/core/queue.c | 355 ++++++++++++++++-------------------------------
drivers/mmc/core/queue.h | 15 +-
4 files changed, 159 insertions(+), 281 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index f1008ce5376b..f977117f7435 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -28,6 +28,7 @@
#include <linux/hdreg.h>
#include <linux/kdev_t.h>
#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/mutex.h>
#include <linux/scatterlist.h>
#include <linux/string_helpers.h>
@@ -96,7 +97,6 @@ static DEFINE_SPINLOCK(mmc_blk_lock);
* There is one mmc_blk_data per slot.
*/
struct mmc_blk_data {
- spinlock_t lock;
struct device *parent;
struct gendisk *disk;
struct mmc_queue queue;
@@ -1188,7 +1188,7 @@ static void mmc_blk_issue_discard_rq(struct mmc_queue_req *mq_rq)
if (!err)
mmc_blk_reset_success(md, type);
fail:
- blk_end_request(mq_rq->req, err, blk_rq_bytes(mq_rq->req));
+ blk_mq_complete_request(mq_rq->req, err);
}
static void mmc_blk_issue_secdiscard_rq(struct mmc_queue_req *mq_rq)
@@ -1265,7 +1265,8 @@ static void mmc_blk_issue_flush(struct mmc_queue_req *mq_rq)
if (ret)
ret = -EIO;
- blk_end_request_all(mq_rq->req, ret);
+ /* FIXME: was using blk_end_request_all() to flush */
+ blk_mq_complete_request(mq_rq->req, ret);
}
/*
@@ -1589,12 +1590,15 @@ static bool mmc_blk_rw_cmd_err(struct mmc_blk_data *md, struct mmc_card *card,
int err;
err = mmc_sd_num_wr_blocks(card, &blocks);
- if (err)
+ if (err) {
req_pending = old_req_pending;
- else
- req_pending = blk_end_request(req, 0, blocks << 9);
+ } else {
+ blk_mq_complete_request(req, 0);
+ req_pending = false;
+ }
} else {
- req_pending = blk_end_request(req, 0, brq->data.bytes_xfered);
+ blk_mq_complete_request(req, 0);
+ req_pending = false;
}
return req_pending;
}
@@ -1630,33 +1634,18 @@ static void mmc_blk_rw_try_restart(struct mmc_queue_req *mq_rq)
void mmc_blk_rw_done_success(struct mmc_async_req *areq)
{
struct mmc_queue_req *mq_rq;
- struct mmc_blk_request *brq;
struct mmc_blk_data *md;
struct request *old_req;
- bool req_pending;
int type;
mq_rq = container_of(areq, struct mmc_queue_req, areq);
md = mq_rq->mq->blkdata;
- brq = &mq_rq->brq;
old_req = mq_rq->req;
type = rq_data_dir(old_req) == READ ? MMC_BLK_READ : MMC_BLK_WRITE;
mmc_queue_bounce_post(mq_rq);
mmc_blk_reset_success(md, type);
- req_pending = blk_end_request(old_req, 0,
- brq->data.bytes_xfered);
- /*
- * If the blk_end_request function returns non-zero even
- * though all data has been transferred and no errors
- * were returned by the host controller, it's a bug.
- */
- if (req_pending) {
- pr_err("%s BUG rq_tot %d d_xfer %d\n",
- __func__, blk_rq_bytes(old_req),
- brq->data.bytes_xfered);
- return;
- }
+ blk_mq_complete_request(old_req, 0);
}
/**
@@ -1702,9 +1691,16 @@ void mmc_blk_rw_done_error(struct mmc_async_req *areq,
* A block was successfully transferred.
*/
mmc_blk_reset_success(md, type);
-
- req_pending = blk_end_request(old_req, 0,
- brq->data.bytes_xfered);
+ /*
+ * FIXME:
+ * How do we handle a partial request?
+ * brq->data.bytes_xfered contains the number of
+ * successfully transfered bytes, how to report
+ * this to the MQ block layer and get a new, smaller
+ * request back? Currently just requeueing.
+ */
+ blk_mq_requeue_request(old_req, false);
+ req_pending = false;
break;
case MMC_BLK_CMD_ERR:
req_pending = mmc_blk_rw_cmd_err(md, card, brq, old_req, req_pending);
@@ -1754,12 +1750,9 @@ void mmc_blk_rw_done_error(struct mmc_async_req *areq,
* time, so we only reach here after trying to
* read a single sector.
*/
- req_pending = blk_end_request(old_req, -EIO,
- brq->data.blksz);
- if (!req_pending) {
- mmc_blk_rw_try_restart(mq_rq);
- return;
- }
+ blk_mq_complete_request(mq_rq->req, -EIO);
+ req_pending = false;
+ mmc_blk_rw_try_restart(mq_rq);
break;
case MMC_BLK_NOMEDIUM:
mmc_blk_rw_cmd_abort(card, old_req);
@@ -1819,7 +1812,8 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
ret = mmc_blk_part_switch(card, md);
if (ret) {
- blk_end_request_all(mq_rq->req, -EIO);
+ /* FIXME: was blk_end_request_all() to flush */
+ blk_mq_complete_request(mq_rq->req, -EIO);
return;
}
@@ -1830,7 +1824,6 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
card->host->areq = NULL;
}
mmc_blk_issue_discard_rq(mq_rq);
- mmc_queue_req_put(mq_rq);
} else if (req_op(mq_rq->req) == REQ_OP_SECURE_ERASE) {
/* complete ongoing async transfer before issuing secure erase*/
if (card->host->areq) {
@@ -1838,7 +1831,6 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
card->host->areq = NULL;
}
mmc_blk_issue_secdiscard_rq(mq_rq);
- mmc_queue_req_put(mq_rq);
} else if (req_op(mq_rq->req) == REQ_OP_FLUSH) {
/* complete ongoing async transfer before issuing flush */
if (card->host->areq) {
@@ -1846,7 +1838,6 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
card->host->areq = NULL;
}
mmc_blk_issue_flush(mq_rq);
- mmc_queue_req_put(mq_rq);
} else {
mmc_blk_issue_rw_rq(mq_rq);
}
@@ -1906,11 +1897,10 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
goto err_kfree;
}
- spin_lock_init(&md->lock);
INIT_LIST_HEAD(&md->part);
md->usage = 1;
- ret = mmc_init_queue(&md->queue, card, &md->lock, subname);
+ ret = mmc_init_queue(&md->queue, card, subname);
if (ret)
goto err_putdisk;
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 04666ad91df0..a81b6baa3bee 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -599,8 +599,6 @@ void mmc_finalize_areq(struct kthread_work *work)
{
struct mmc_async_req *areq =
container_of(work, struct mmc_async_req, finalization_work);
- struct mmc_queue_req *mq_rq = container_of(areq, struct mmc_queue_req,
- areq);
struct mmc_host *host = areq->host;
enum mmc_blk_status status = MMC_BLK_SUCCESS;
struct mmc_command *cmd;
@@ -648,7 +646,6 @@ void mmc_finalize_areq(struct kthread_work *work)
*/
mmc_blk_rw_done_error(areq, status);
complete(&areq->complete);
- mmc_queue_req_put(mq_rq);
return;
}
@@ -660,7 +657,6 @@ void mmc_finalize_areq(struct kthread_work *work)
complete(&areq->complete);
mmc_post_req(host, areq->mrq, 0);
mmc_blk_rw_done_success(areq);
- mmc_queue_req_put(mq_rq);
}
EXPORT_SYMBOL(mmc_finalize_areq);
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index e7ba5bef2df3..9850d7342763 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -14,6 +14,7 @@
#include <linux/kthread.h>
#include <linux/scatterlist.h>
#include <linux/dma-mapping.h>
+#include <linux/blk-mq.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
@@ -49,126 +50,6 @@ static int mmc_prep_request(struct request_queue *q, struct request *req)
return BLKPREP_OK;
}
-/**
- * Get an available queue item from the pool.
- */
-static struct mmc_queue_req *mmc_queue_req_get(struct mmc_queue *mq)
-{
- int i;
-
- /*
- * This simply cannot fail so we just spin here
- * until we get a queue request to work on.
- */
- while (1) {
- /* Just take the first unused queue request */
- spin_lock_irq(&mq->mqrq_lock);
- for (i = 0; i < mq->qdepth; i++) {
- if (!mq->mqrq[i].in_use) {
- mq->mqrq[i].in_use = true;
- spin_unlock_irq(&mq->mqrq_lock);
- return &mq->mqrq[i];
- }
- }
- spin_unlock_irq(&mq->mqrq_lock);
-
- pr_warn_once("%s: out of queue items, spinning\n", __func__);
- }
-}
-
-void mmc_queue_req_put(struct mmc_queue_req *mq_rq)
-{
- mq_rq->brq.mrq.data = NULL;
- mq_rq->req = NULL;
- spin_lock_irq(&mq_rq->mq->mqrq_lock);
- mq_rq->in_use = false;
- spin_unlock_irq(&mq_rq->mq->mqrq_lock);
-}
-
-static int mmc_queue_thread(void *d)
-{
- struct mmc_queue *mq = d;
- struct request_queue *q = mq->queue;
- bool claimed_host = false;
- struct mmc_queue_req *mq_rq;
-
- current->flags |= PF_MEMALLOC;
-
- down(&mq->thread_sem);
- do {
- struct request *req = NULL;
-
- set_current_state(TASK_INTERRUPTIBLE);
- spin_lock_irq(q->queue_lock);
- req = blk_fetch_request(q);
- spin_unlock_irq(q->queue_lock);
- mq->asleep = false;
-
- if (req) {
- bool req_is_special = mmc_req_is_special(req);
-
- mq_rq = mmc_queue_req_get(mq);
- mq_rq->req = req;
- if (!claimed_host)
- mmc_get_card(mq->card);
- set_current_state(TASK_RUNNING);
- mmc_blk_issue_rq(mq_rq);
- cond_resched();
- /*
- * Current request becomes previous request
- * and vice versa.
- * In case of special requests, current request
- * has been finished. Do not assign it to previous
- * request. Always unclaim the host after special
- * commands.
- */
- if (req_is_special) {
- mmc_put_card(mq->card);
- claimed_host = false;
- }
- } else {
- mq->asleep = true;
- if (kthread_should_stop()) {
- set_current_state(TASK_RUNNING);
- break;
- }
- up(&mq->thread_sem);
- schedule();
- down(&mq->thread_sem);
- }
- } while (1);
-
- if (claimed_host)
- mmc_put_card(mq->card);
-
- up(&mq->thread_sem);
-
- return 0;
-}
-
-/*
- * Generic MMC request handler. This is called for any queue on a
- * particular host. When the host is not busy, we look for a request
- * on any queue on this host, and attempt to issue it. This may
- * not be the queue we were asked to process.
- */
-static void mmc_request_fn(struct request_queue *q)
-{
- struct mmc_queue *mq = q->queuedata;
- struct request *req;
-
- if (!mq) {
- while ((req = blk_fetch_request(q)) != NULL) {
- req->rq_flags |= RQF_QUIET;
- __blk_end_request_all(req, -EIO);
- }
- return;
- }
-
- if (mq->asleep)
- wake_up_process(mq->thread);
-}
-
static struct scatterlist *mmc_alloc_sg(int sg_len, int *err)
{
struct scatterlist *sg;
@@ -205,55 +86,80 @@ static void mmc_queue_setup_discard(struct request_queue *q,
queue_flag_set_unlocked(QUEUE_FLAG_SECERASE, q);
}
-#ifdef CONFIG_MMC_BLOCK_BOUNCE
-static bool mmc_queue_alloc_bounce_bufs(struct mmc_queue *mq,
- unsigned int bouncesz)
+static int mmc_queue_rq(struct blk_mq_hw_ctx *hctx,
+ const struct blk_mq_queue_data *bd)
{
- int i;
+ struct mmc_queue_req *mq_rq = blk_mq_rq_to_pdu(bd->rq);
+ struct mmc_queue *mq = mq_rq->mq;
+ bool req_is_special = mmc_req_is_special(bd->rq);
- for (i = 0; i < mq->qdepth; i++) {
- mq->mqrq[i].bounce_buf = kmalloc(bouncesz, GFP_KERNEL);
- if (!mq->mqrq[i].bounce_buf)
- goto out_err;
+ /* start this request */
+ mq_rq->req = bd->rq;
+ mq_rq->brq.mrq.data = NULL;
+ blk_mq_start_request(mq_rq->req);
+
+ if (!mq->claimed_host) {
+ mmc_get_card(mq->card);
+ mq->claimed_host = true;
+ }
+ set_current_state(TASK_RUNNING);
+ mmc_blk_issue_rq(mq_rq);
+ cond_resched();
+ /*
+ * In case of special requests, current request
+ * has been finished. Always unclaim the host after special
+ * commands.
+ */
+ if (req_is_special) {
+ mmc_put_card(mq->card);
+ mq->claimed_host = false;
}
- return true;
+ /*
+ * FIXME: unclaim host after timeout?
+ * Can blk-mq notify us that there are no requests coming
+ * for a while so we can relax the host?
+ */
+ // if (claimed_host)
+ // mmc_put_card(mq->card);
-out_err:
- while (--i >= 0) {
- kfree(mq->mqrq[i].bounce_buf);
- mq->mqrq[i].bounce_buf = NULL;
- }
- pr_warn("%s: unable to allocate bounce buffers\n",
- mmc_card_name(mq->card));
- return false;
+ return BLK_MQ_RQ_QUEUE_OK;
}
-static int mmc_queue_alloc_bounce_sgs(struct mmc_queue *mq,
- unsigned int bouncesz)
+static int mmc_init_request(void *data, struct request *rq,
+ unsigned int hctx_idx, unsigned int request_idx,
+ unsigned int numa_node)
{
- int i, ret;
+ struct mmc_queue_req *mq_rq = blk_mq_rq_to_pdu(rq);
+ struct mmc_queue *mq = data;
+ struct mmc_card *card = mq->card;
+ struct mmc_host *host = card->host;
+ int ret;
- for (i = 0; i < mq->qdepth; i++) {
- mq->mqrq[i].sg = mmc_alloc_sg(1, &ret);
- if (ret)
- return ret;
+ mq_rq->dev = host->parent;
+ mq_rq->mq = mq;
+ /* Set up bounces etc */
+ dev_info(mq_rq->dev, "%s\n", __func__);
- mq->mqrq[i].bounce_sg = mmc_alloc_sg(bouncesz / 512, &ret);
- if (ret)
- return ret;
- }
+#ifdef CONFIG_MMC_BLOCK_BOUNCE
+ if (mq->bouncesz) {
+ mq_rq->bounce_buf = kmalloc(mq->bouncesz, GFP_KERNEL);
+ if (!mq_rq->bounce_buf)
+ return -ENOMEM;
+ if (mq->bouncesz > 512) {
+ mq_rq->sg = mmc_alloc_sg(1, &ret);
+ if (ret)
+ return ret;
- return 0;
-}
+ mq_rq->bounce_sg = mmc_alloc_sg(mq->bouncesz / 512,
+ &ret);
+ if (ret)
+ return ret;
+ }
+ } else
#endif
-
-static int mmc_queue_alloc_sgs(struct mmc_queue *mq, int max_segs)
-{
- int i, ret;
-
- for (i = 0; i < mq->qdepth; i++) {
- mq->mqrq[i].sg = mmc_alloc_sg(max_segs, &ret);
+ {
+ mq_rq->sg = mmc_alloc_sg(host->max_segs, &ret);
if (ret)
return ret;
}
@@ -261,26 +167,35 @@ static int mmc_queue_alloc_sgs(struct mmc_queue *mq, int max_segs)
return 0;
}
-static void mmc_queue_req_free_bufs(struct mmc_queue_req *mqrq)
+static void mmc_exit_request(void *data, struct request *rq,
+ unsigned int hctx_idx, unsigned int request_idx)
{
- kfree(mqrq->bounce_sg);
- mqrq->bounce_sg = NULL;
+ struct mmc_queue_req *mq_rq = blk_mq_rq_to_pdu(rq);
- kfree(mqrq->sg);
- mqrq->sg = NULL;
+ dev_info(mq_rq->dev, "%s: hctx_idx = %u, request_idx = %u\n",
+ __func__, hctx_idx, request_idx);
- kfree(mqrq->bounce_buf);
- mqrq->bounce_buf = NULL;
-}
+ kfree(mq_rq->bounce_sg);
+ mq_rq->bounce_sg = NULL;
-static void mmc_queue_reqs_free_bufs(struct mmc_queue *mq)
-{
- int i;
+ kfree(mq_rq->sg);
+ mq_rq->sg = NULL;
- for (i = 0; i < mq->qdepth; i++)
- mmc_queue_req_free_bufs(&mq->mqrq[i]);
+ kfree(mq_rq->bounce_buf);
+ mq_rq->bounce_buf = NULL;
}
+static struct blk_mq_ops mmc_mq_ops = {
+ .queue_rq = mmc_queue_rq,
+ .init_request = mmc_init_request,
+ .exit_request = mmc_exit_request,
+ /*
+ * .exit_request() will only be invoked if we explcitly call
+ * blk_mq_end_request() on all requests. Why would we do that,
+ * we will just call blk_mq_complete_request().
+ */
+};
+
/**
* mmc_init_queue - initialise a queue structure.
* @mq: mmc queue
@@ -291,31 +206,47 @@ static void mmc_queue_reqs_free_bufs(struct mmc_queue *mq)
* Initialise a MMC card request queue.
*/
int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
- spinlock_t *lock, const char *subname)
+ const char *subname)
{
struct mmc_host *host = card->host;
u64 limit = BLK_BOUNCE_HIGH;
bool bounce = false;
int ret = -ENOMEM;
- int i;
if (mmc_dev(host)->dma_mask && *mmc_dev(host)->dma_mask)
limit = (u64)dma_max_pfn(mmc_dev(host)) << PAGE_SHIFT;
mq->card = card;
- mq->queue = blk_init_queue(mmc_request_fn, lock);
- if (!mq->queue)
+ mq->tag_set.ops = &mmc_mq_ops;
+ /* The MMC/SD protocols have only one command pipe */
+ mq->tag_set.nr_hw_queues = 1;
+ /* Set this to 2 to simulate async requests */
+ mq->tag_set.queue_depth = 2;
+ /*
+ * The extra data allocated per block request.
+ */
+ mq->tag_set.cmd_size = sizeof(struct mmc_queue_req);
+ mq->tag_set.numa_node = NUMA_NO_NODE;
+ /* We use blocking requests */
+ mq->tag_set.flags = BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_BLOCKING;
+ // BLK_MQ_F_SG_MERGE?
+ mq->tag_set.driver_data = mq;
+
+ ret = blk_mq_alloc_tag_set(&mq->tag_set);
+ if (ret) {
+ dev_err(card->host->parent, "failed to allocate MQ tag set\n");
return -ENOMEM;
+ }
- mq->qdepth = 4;
- spin_lock_init(&mq->mqrq_lock);
- mq->mqrq = kcalloc(mq->qdepth, sizeof(struct mmc_queue_req),
- GFP_KERNEL);
- if (!mq->mqrq)
- goto blk_cleanup;
- for (i = 0; i < mq->qdepth; i++)
- mq->mqrq[i].mq = mq;
+ mq->queue = blk_mq_init_queue(&mq->tag_set);
+ if (!mq->queue) {
+ dev_err(card->host->parent, "failed to initialize block MQ\n");
+ goto cleanup_free_tag_set;
+ }
+
+ blk_queue_max_segments(mq->queue, host->max_segs);
+ mq->qdepth = 4;
mq->queue->queuedata = mq;
blk_queue_prep_rq(mq->queue, mmc_prep_request);
@@ -337,18 +268,15 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
if (bouncesz > (host->max_blk_count * 512))
bouncesz = host->max_blk_count * 512;
- if (bouncesz > 512 &&
- mmc_queue_alloc_bounce_bufs(mq, bouncesz)) {
+ if (bouncesz > 512) {
blk_queue_bounce_limit(mq->queue, BLK_BOUNCE_ANY);
blk_queue_max_hw_sectors(mq->queue, bouncesz / 512);
blk_queue_max_segments(mq->queue, bouncesz / 512);
blk_queue_max_segment_size(mq->queue, bouncesz);
-
- ret = mmc_queue_alloc_bounce_sgs(mq, bouncesz);
- if (ret)
- goto cleanup_queue;
bounce = true;
}
+
+ mq->bouncesz = bouncesz;
}
#endif
@@ -358,53 +286,29 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
min(host->max_blk_count, host->max_req_size / 512));
blk_queue_max_segments(mq->queue, host->max_segs);
blk_queue_max_segment_size(mq->queue, host->max_seg_size);
-
- ret = mmc_queue_alloc_sgs(mq, host->max_segs);
- if (ret)
- goto cleanup_queue;
- }
-
- sema_init(&mq->thread_sem, 1);
-
- mq->thread = kthread_run(mmc_queue_thread, mq, "mmcqd/%d%s",
- host->index, subname ? subname : "");
-
- if (IS_ERR(mq->thread)) {
- ret = PTR_ERR(mq->thread);
- goto cleanup_queue;
}
return 0;
- cleanup_queue:
- mmc_queue_reqs_free_bufs(mq);
- kfree(mq->mqrq);
- mq->mqrq = NULL;
-blk_cleanup:
- blk_cleanup_queue(mq->queue);
+cleanup_free_tag_set:
+ blk_mq_free_tag_set(&mq->tag_set);
+
return ret;
}
void mmc_cleanup_queue(struct mmc_queue *mq)
{
struct request_queue *q = mq->queue;
- unsigned long flags;
/* Make sure the queue isn't suspended, as that will deadlock */
mmc_queue_resume(mq);
- /* Then terminate our worker thread */
- kthread_stop(mq->thread);
-
/* Empty the queue */
- spin_lock_irqsave(q->queue_lock, flags);
q->queuedata = NULL;
blk_start_queue(q);
- spin_unlock_irqrestore(q->queue_lock, flags);
- mmc_queue_reqs_free_bufs(mq);
- kfree(mq->mqrq);
- mq->mqrq = NULL;
+ blk_cleanup_queue(mq->queue);
+ blk_mq_free_tag_set(&mq->tag_set);
mq->card = NULL;
}
@@ -414,23 +318,16 @@ EXPORT_SYMBOL(mmc_cleanup_queue);
* mmc_queue_suspend - suspend a MMC request queue
* @mq: MMC queue to suspend
*
- * Stop the block request queue, and wait for our thread to
- * complete any outstanding requests. This ensures that we
+ * Stop the block request queue. This ensures that we
* won't suspend while a request is being processed.
*/
void mmc_queue_suspend(struct mmc_queue *mq)
{
struct request_queue *q = mq->queue;
- unsigned long flags;
if (!mq->suspended) {
- mq->suspended |= true;
-
- spin_lock_irqsave(q->queue_lock, flags);
+ mq->suspended = true;
blk_stop_queue(q);
- spin_unlock_irqrestore(q->queue_lock, flags);
-
- down(&mq->thread_sem);
}
}
@@ -441,16 +338,10 @@ void mmc_queue_suspend(struct mmc_queue *mq)
void mmc_queue_resume(struct mmc_queue *mq)
{
struct request_queue *q = mq->queue;
- unsigned long flags;
if (mq->suspended) {
mq->suspended = false;
-
- up(&mq->thread_sem);
-
- spin_lock_irqsave(q->queue_lock, flags);
blk_start_queue(q);
- spin_unlock_irqrestore(q->queue_lock, flags);
}
}
diff --git a/drivers/mmc/core/queue.h b/drivers/mmc/core/queue.h
index 886a05482b74..0a30fcc115ee 100644
--- a/drivers/mmc/core/queue.h
+++ b/drivers/mmc/core/queue.h
@@ -4,6 +4,7 @@
#include <linux/types.h>
#include <linux/spinlock.h>
#include <linux/blkdev.h>
+#include <linux/blk-mq.h>
#include <linux/mmc/core.h>
#include <linux/mmc/host.h>
@@ -28,7 +29,7 @@ struct mmc_blk_request {
};
struct mmc_queue_req {
- bool in_use;
+ struct device *dev;
struct request *req;
struct mmc_blk_request brq;
struct scatterlist *sg;
@@ -41,20 +42,20 @@ struct mmc_queue_req {
struct mmc_queue {
struct mmc_card *card;
- struct task_struct *thread;
- struct semaphore thread_sem;
bool suspended;
- bool asleep;
+ bool claimed_host;
struct mmc_blk_data *blkdata;
struct request_queue *queue;
+ struct blk_mq_tag_set tag_set;
spinlock_t mqrq_lock;
struct mmc_queue_req *mqrq;
unsigned int qdepth;
+#ifdef CONFIG_MMC_BLOCK_BOUNCE
+ unsigned int bouncesz;
+#endif
};
-extern void mmc_queue_req_put(struct mmc_queue_req *mq_rq);
-extern int mmc_init_queue(struct mmc_queue *, struct mmc_card *, spinlock_t *,
- const char *);
+extern int mmc_init_queue(struct mmc_queue *, struct mmc_card *, const char *);
extern void mmc_cleanup_queue(struct mmc_queue *);
extern void mmc_queue_suspend(struct mmc_queue *);
extern void mmc_queue_resume(struct mmc_queue *);
--
2.9.3
^ permalink raw reply related
* [PATCH 15/16] mmc: queue: issue requests in massive parallel
From: Linus Walleij @ 2017-02-09 15:34 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
This makes a crucial change to the issueing mechanism for the
MMC requests:
Before commit "mmc: core: move the asynchronous post-processing"
some parallelism on the read/write requests was achieved by
speculatively postprocessing a request and re-preprocess and
re-issue the request if something went wrong, which we discover
later when checking for an error.
This is kind of ugly. Instead we need a mechanism like here:
We issue requests, and when they come back from the hardware,
we know if they finished successfully or not. If the request
was successful, we complete the asynchronous request and let a
new request immediately start on the hardware. If, and only if,
it returned an error from the hardware we go down the error
path.
This is achieved by splitting the work path from the hardware
in two: a successful path ending up calling down to
mmc_blk_rw_done_success() and an errorpath calling down to
mmc_blk_rw_done_error().
This has a profound effect: we reintroduce the parallelism on
the successful path as mmc_post_req() can now be called in
while the next request is in transit (just like prior to
commit "mmc: core: move the asynchronous post-processing")
but ALSO we can call mmc_queue_bounce_post() and
blk_end_request() in parallel.
The latter has the profound effect of issuing a new request
again so that we actually need to have at least three requests
in transit at the same time: we haven't yet dropped the
reference to our struct mmc_queue_req so we need at least
three. I put the pool to 4 requests for now.
I expect the imrovement to be noticeable on systems that use
bounce buffers since they can now process requests in parallel
with post-processing their bounce buffers, but I don't have a
test target for that.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 61 +++++++++++++++++++++++++++++++++++++-----------
drivers/mmc/core/block.h | 4 +++-
drivers/mmc/core/core.c | 27 ++++++++++++++++++---
drivers/mmc/core/queue.c | 2 +-
4 files changed, 75 insertions(+), 19 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index acca15cc1807..f1008ce5376b 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -1622,8 +1622,51 @@ static void mmc_blk_rw_try_restart(struct mmc_queue_req *mq_rq)
mmc_restart_areq(mq->card->host, &mq_rq->areq);
}
-void mmc_blk_rw_done(struct mmc_async_req *areq,
- enum mmc_blk_status status)
+/**
+ * Final handling of an asynchronous request if there was no error.
+ * This is the common path that we take when everything is nice
+ * and smooth. The status from the command is always MMC_BLK_SUCCESS.
+ */
+void mmc_blk_rw_done_success(struct mmc_async_req *areq)
+{
+ struct mmc_queue_req *mq_rq;
+ struct mmc_blk_request *brq;
+ struct mmc_blk_data *md;
+ struct request *old_req;
+ bool req_pending;
+ int type;
+
+ mq_rq = container_of(areq, struct mmc_queue_req, areq);
+ md = mq_rq->mq->blkdata;
+ brq = &mq_rq->brq;
+ old_req = mq_rq->req;
+ type = rq_data_dir(old_req) == READ ? MMC_BLK_READ : MMC_BLK_WRITE;
+
+ mmc_queue_bounce_post(mq_rq);
+ mmc_blk_reset_success(md, type);
+ req_pending = blk_end_request(old_req, 0,
+ brq->data.bytes_xfered);
+ /*
+ * If the blk_end_request function returns non-zero even
+ * though all data has been transferred and no errors
+ * were returned by the host controller, it's a bug.
+ */
+ if (req_pending) {
+ pr_err("%s BUG rq_tot %d d_xfer %d\n",
+ __func__, blk_rq_bytes(old_req),
+ brq->data.bytes_xfered);
+ return;
+ }
+}
+
+/**
+ * Error, recapture, retry etc for asynchronous requests.
+ * This is the error path that we take when there is bad status
+ * coming back from the hardware and we need to do a bit of
+ * cleverness.
+ */
+void mmc_blk_rw_done_error(struct mmc_async_req *areq,
+ enum mmc_blk_status status)
{
struct mmc_queue *mq;
struct mmc_queue_req *mq_rq;
@@ -1652,6 +1695,8 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
switch (status) {
case MMC_BLK_SUCCESS:
+ pr_err("%s: MMC_BLK_SUCCESS on error path\n", __func__);
+ /* This should not happen: anyway fall through */
case MMC_BLK_PARTIAL:
/*
* A block was successfully transferred.
@@ -1660,18 +1705,6 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
req_pending = blk_end_request(old_req, 0,
brq->data.bytes_xfered);
- /*
- * If the blk_end_request function returns non-zero even
- * though all data has been transferred and no errors
- * were returned by the host controller, it's a bug.
- */
- if (status == MMC_BLK_SUCCESS && req_pending) {
- pr_err("%s BUG rq_tot %d d_xfer %d\n",
- __func__, blk_rq_bytes(old_req),
- brq->data.bytes_xfered);
- mmc_blk_rw_cmd_abort(card, old_req);
- return;
- }
break;
case MMC_BLK_CMD_ERR:
req_pending = mmc_blk_rw_cmd_err(md, card, brq, old_req, req_pending);
diff --git a/drivers/mmc/core/block.h b/drivers/mmc/core/block.h
index 0326fa5d8217..eae47ae7c903 100644
--- a/drivers/mmc/core/block.h
+++ b/drivers/mmc/core/block.h
@@ -5,7 +5,9 @@ struct mmc_async_req;
enum mmc_blk_status;
struct mmc_queue_req;
-void mmc_blk_rw_done(struct mmc_async_req *areq, enum mmc_blk_status status);
+void mmc_blk_rw_done_success(struct mmc_async_req *areq);
+void mmc_blk_rw_done_error(struct mmc_async_req *areq,
+ enum mmc_blk_status status);
void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq);
#endif
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 50a8942b98c2..04666ad91df0 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -634,11 +634,32 @@ void mmc_finalize_areq(struct kthread_work *work)
mmc_start_bkops(host->card, true);
}
- /* Successfully postprocess the old request at this point */
- mmc_post_req(host, areq->mrq, 0);
- mmc_blk_rw_done(areq, status);
+ /*
+ * Postprocess the old request at this point:
+ * on success: take a fast path!
+ * on error: take the slow path, retrying etc
+ */
+ if (status != MMC_BLK_SUCCESS) {
+ mmc_post_req(host, areq->mrq, 0);
+ /*
+ * This call can lead to retransmissions using
+ * mmc_restart_areq() so do not complete until
+ * after this call!
+ */
+ mmc_blk_rw_done_error(areq, status);
+ complete(&areq->complete);
+ mmc_queue_req_put(mq_rq);
+ return;
+ }
+ /*
+ * There will not be any retransmissions etc
+ * at this point, so let the next request get
+ * access to the hardware.
+ */
complete(&areq->complete);
+ mmc_post_req(host, areq->mrq, 0);
+ mmc_blk_rw_done_success(areq);
mmc_queue_req_put(mq_rq);
}
EXPORT_SYMBOL(mmc_finalize_areq);
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index cab0f51dbb4d..e7ba5bef2df3 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -307,7 +307,7 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
if (!mq->queue)
return -ENOMEM;
- mq->qdepth = 2;
+ mq->qdepth = 4;
spin_lock_init(&mq->mqrq_lock);
mq->mqrq = kcalloc(mq->qdepth, sizeof(struct mmc_queue_req),
GFP_KERNEL);
--
2.9.3
^ permalink raw reply related
* [PATCH 14/16] mmc: queue: get/put struct mmc_queue_req
From: Linus Walleij @ 2017-02-09 15:34 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
The per-hardware-transaction struct mmc_queue_req is assigned
from a pool of 2 requests using a current/previous scheme and
then swapped around.
This is confusing, especially if we need more than two to make
our work efficient and parallel.
Rewrite the mechanism to have a pool of struct mmc_queue_req
and take one when we need one and put it back when we don't
need it anymore.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 3 +++
drivers/mmc/core/core.c | 4 ++++
drivers/mmc/core/queue.c | 57 ++++++++++++++++++++++++++++++++++++++----------
drivers/mmc/core/queue.h | 8 ++++---
4 files changed, 57 insertions(+), 15 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index 628a22b9bf41..acca15cc1807 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -1797,6 +1797,7 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
card->host->areq = NULL;
}
mmc_blk_issue_discard_rq(mq_rq);
+ mmc_queue_req_put(mq_rq);
} else if (req_op(mq_rq->req) == REQ_OP_SECURE_ERASE) {
/* complete ongoing async transfer before issuing secure erase*/
if (card->host->areq) {
@@ -1804,6 +1805,7 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
card->host->areq = NULL;
}
mmc_blk_issue_secdiscard_rq(mq_rq);
+ mmc_queue_req_put(mq_rq);
} else if (req_op(mq_rq->req) == REQ_OP_FLUSH) {
/* complete ongoing async transfer before issuing flush */
if (card->host->areq) {
@@ -1811,6 +1813,7 @@ void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
card->host->areq = NULL;
}
mmc_blk_issue_flush(mq_rq);
+ mmc_queue_req_put(mq_rq);
} else {
mmc_blk_issue_rw_rq(mq_rq);
}
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 03c290e5e2c9..50a8942b98c2 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -39,6 +39,7 @@
#define CREATE_TRACE_POINTS
#include <trace/events/mmc.h>
+#include "queue.h"
#include "block.h"
#include "core.h"
#include "card.h"
@@ -598,6 +599,8 @@ void mmc_finalize_areq(struct kthread_work *work)
{
struct mmc_async_req *areq =
container_of(work, struct mmc_async_req, finalization_work);
+ struct mmc_queue_req *mq_rq = container_of(areq, struct mmc_queue_req,
+ areq);
struct mmc_host *host = areq->host;
enum mmc_blk_status status = MMC_BLK_SUCCESS;
struct mmc_command *cmd;
@@ -636,6 +639,7 @@ void mmc_finalize_areq(struct kthread_work *work)
mmc_blk_rw_done(areq, status);
complete(&areq->complete);
+ mmc_queue_req_put(mq_rq);
}
EXPORT_SYMBOL(mmc_finalize_areq);
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index c4e1ced55796..cab0f51dbb4d 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -49,6 +49,42 @@ static int mmc_prep_request(struct request_queue *q, struct request *req)
return BLKPREP_OK;
}
+/**
+ * Get an available queue item from the pool.
+ */
+static struct mmc_queue_req *mmc_queue_req_get(struct mmc_queue *mq)
+{
+ int i;
+
+ /*
+ * This simply cannot fail so we just spin here
+ * until we get a queue request to work on.
+ */
+ while (1) {
+ /* Just take the first unused queue request */
+ spin_lock_irq(&mq->mqrq_lock);
+ for (i = 0; i < mq->qdepth; i++) {
+ if (!mq->mqrq[i].in_use) {
+ mq->mqrq[i].in_use = true;
+ spin_unlock_irq(&mq->mqrq_lock);
+ return &mq->mqrq[i];
+ }
+ }
+ spin_unlock_irq(&mq->mqrq_lock);
+
+ pr_warn_once("%s: out of queue items, spinning\n", __func__);
+ }
+}
+
+void mmc_queue_req_put(struct mmc_queue_req *mq_rq)
+{
+ mq_rq->brq.mrq.data = NULL;
+ mq_rq->req = NULL;
+ spin_lock_irq(&mq_rq->mq->mqrq_lock);
+ mq_rq->in_use = false;
+ spin_unlock_irq(&mq_rq->mq->mqrq_lock);
+}
+
static int mmc_queue_thread(void *d)
{
struct mmc_queue *mq = d;
@@ -62,17 +98,17 @@ static int mmc_queue_thread(void *d)
do {
struct request *req = NULL;
- spin_lock_irq(q->queue_lock);
set_current_state(TASK_INTERRUPTIBLE);
+ spin_lock_irq(q->queue_lock);
req = blk_fetch_request(q);
- mq->asleep = false;
- mq_rq = mq->mqrq_cur;
- mq_rq->req = req;
spin_unlock_irq(q->queue_lock);
+ mq->asleep = false;
if (req) {
bool req_is_special = mmc_req_is_special(req);
+ mq_rq = mmc_queue_req_get(mq);
+ mq_rq->req = req;
if (!claimed_host)
mmc_get_card(mq->card);
set_current_state(TASK_RUNNING);
@@ -87,13 +123,9 @@ static int mmc_queue_thread(void *d)
* commands.
*/
if (req_is_special) {
- mq->mqrq_cur->req = NULL;
mmc_put_card(mq->card);
claimed_host = false;
}
- mq->mqrq_prev->brq.mrq.data = NULL;
- mq->mqrq_prev->req = NULL;
- swap(mq->mqrq_prev, mq->mqrq_cur);
} else {
mq->asleep = true;
if (kthread_should_stop()) {
@@ -265,6 +297,7 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
u64 limit = BLK_BOUNCE_HIGH;
bool bounce = false;
int ret = -ENOMEM;
+ int i;
if (mmc_dev(host)->dma_mask && *mmc_dev(host)->dma_mask)
limit = (u64)dma_max_pfn(mmc_dev(host)) << PAGE_SHIFT;
@@ -275,14 +308,14 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
return -ENOMEM;
mq->qdepth = 2;
+ spin_lock_init(&mq->mqrq_lock);
mq->mqrq = kcalloc(mq->qdepth, sizeof(struct mmc_queue_req),
GFP_KERNEL);
if (!mq->mqrq)
goto blk_cleanup;
- mq->mqrq_cur = &mq->mqrq[0];
- mq->mqrq_cur->mq = mq;
- mq->mqrq_prev = &mq->mqrq[1];
- mq->mqrq_prev->mq = mq;
+ for (i = 0; i < mq->qdepth; i++)
+ mq->mqrq[i].mq = mq;
+
mq->queue->queuedata = mq;
blk_queue_prep_rq(mq->queue, mmc_prep_request);
diff --git a/drivers/mmc/core/queue.h b/drivers/mmc/core/queue.h
index c18d3f908433..886a05482b74 100644
--- a/drivers/mmc/core/queue.h
+++ b/drivers/mmc/core/queue.h
@@ -2,6 +2,7 @@
#define MMC_QUEUE_H
#include <linux/types.h>
+#include <linux/spinlock.h>
#include <linux/blkdev.h>
#include <linux/mmc/core.h>
#include <linux/mmc/host.h>
@@ -27,6 +28,7 @@ struct mmc_blk_request {
};
struct mmc_queue_req {
+ bool in_use;
struct request *req;
struct mmc_blk_request brq;
struct scatterlist *sg;
@@ -45,12 +47,12 @@ struct mmc_queue {
bool asleep;
struct mmc_blk_data *blkdata;
struct request_queue *queue;
+ spinlock_t mqrq_lock;
struct mmc_queue_req *mqrq;
- struct mmc_queue_req *mqrq_cur;
- struct mmc_queue_req *mqrq_prev;
- int qdepth;
+ unsigned int qdepth;
};
+extern void mmc_queue_req_put(struct mmc_queue_req *mq_rq);
extern int mmc_init_queue(struct mmc_queue *, struct mmc_card *, spinlock_t *,
const char *);
extern void mmc_cleanup_queue(struct mmc_queue *);
--
2.9.3
^ permalink raw reply related
* [PATCH 13/16] mmc: queue: issue struct mmc_queue_req items
From: Linus Walleij @ 2017-02-09 15:34 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
Instead of passing two pointers around and messing and reassigning
to the left and right, issue mmc_queue_req and dereference
the queue from the request where needed. The struct mmc_queue_req
is the thing that has a lifecycle after all: this is what we are
keepin in out queue. Augment all users to be passed the struct
mmc_queue_req as well.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 88 ++++++++++++++++++++++++------------------------
drivers/mmc/core/block.h | 5 ++-
drivers/mmc/core/queue.c | 6 ++--
3 files changed, 50 insertions(+), 49 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index 4952a105780e..628a22b9bf41 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -1151,9 +1151,9 @@ int mmc_access_rpmb(struct mmc_queue *mq)
return false;
}
-static void mmc_blk_issue_discard_rq(struct mmc_queue *mq, struct request *req)
+static void mmc_blk_issue_discard_rq(struct mmc_queue_req *mq_rq)
{
- struct mmc_blk_data *md = mq->blkdata;
+ struct mmc_blk_data *md = mq_rq->mq->blkdata;
struct mmc_card *card = md->queue.card;
unsigned int from, nr, arg;
int err = 0, type = MMC_BLK_DISCARD;
@@ -1163,8 +1163,8 @@ static void mmc_blk_issue_discard_rq(struct mmc_queue *mq, struct request *req)
goto fail;
}
- from = blk_rq_pos(req);
- nr = blk_rq_sectors(req);
+ from = blk_rq_pos(mq_rq->req);
+ nr = blk_rq_sectors(mq_rq->req);
if (mmc_can_discard(card))
arg = MMC_DISCARD_ARG;
@@ -1188,13 +1188,12 @@ static void mmc_blk_issue_discard_rq(struct mmc_queue *mq, struct request *req)
if (!err)
mmc_blk_reset_success(md, type);
fail:
- blk_end_request(req, err, blk_rq_bytes(req));
+ blk_end_request(mq_rq->req, err, blk_rq_bytes(mq_rq->req));
}
-static void mmc_blk_issue_secdiscard_rq(struct mmc_queue *mq,
- struct request *req)
+static void mmc_blk_issue_secdiscard_rq(struct mmc_queue_req *mq_rq)
{
- struct mmc_blk_data *md = mq->blkdata;
+ struct mmc_blk_data *md = mq_rq->mq->blkdata;
struct mmc_card *card = md->queue.card;
unsigned int from, nr, arg;
int err = 0, type = MMC_BLK_SECDISCARD;
@@ -1204,8 +1203,8 @@ static void mmc_blk_issue_secdiscard_rq(struct mmc_queue *mq,
goto out;
}
- from = blk_rq_pos(req);
- nr = blk_rq_sectors(req);
+ from = blk_rq_pos(mq_rq->req);
+ nr = blk_rq_sectors(mq_rq->req);
if (mmc_can_trim(card) && !mmc_erase_group_aligned(card, from, nr))
arg = MMC_SECURE_TRIM1_ARG;
@@ -1253,12 +1252,12 @@ static void mmc_blk_issue_secdiscard_rq(struct mmc_queue *mq,
if (!err)
mmc_blk_reset_success(md, type);
out:
- blk_end_request(req, err, blk_rq_bytes(req));
+ blk_end_request(mq_rq->req, err, blk_rq_bytes(mq_rq->req));
}
-static void mmc_blk_issue_flush(struct mmc_queue *mq, struct request *req)
+static void mmc_blk_issue_flush(struct mmc_queue_req *mq_rq)
{
- struct mmc_blk_data *md = mq->blkdata;
+ struct mmc_blk_data *md = mq_rq->mq->blkdata;
struct mmc_card *card = md->queue.card;
int ret = 0;
@@ -1266,7 +1265,7 @@ static void mmc_blk_issue_flush(struct mmc_queue *mq, struct request *req)
if (ret)
ret = -EIO;
- blk_end_request_all(req, ret);
+ blk_end_request_all(mq_rq->req, ret);
}
/*
@@ -1614,11 +1613,13 @@ static void mmc_blk_rw_cmd_abort(struct mmc_card *card, struct request *req)
* @mq: the queue with the card and host to restart
* @req: a new request that want to be started after the current one
*/
-static void mmc_blk_rw_try_restart(struct mmc_queue *mq)
+static void mmc_blk_rw_try_restart(struct mmc_queue_req *mq_rq)
{
+ struct mmc_queue *mq = mq_rq->mq;
+
/* Proceed and try to restart the current async request */
- mmc_blk_rw_rq_prep(mq->mqrq_cur, mq->card, 0, mq);
- mmc_restart_areq(mq->card->host, &mq->mqrq_cur->areq);
+ mmc_blk_rw_rq_prep(mq_rq, mq->card, 0, mq);
+ mmc_restart_areq(mq->card->host, &mq_rq->areq);
}
void mmc_blk_rw_done(struct mmc_async_req *areq,
@@ -1676,11 +1677,11 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
req_pending = mmc_blk_rw_cmd_err(md, card, brq, old_req, req_pending);
if (mmc_blk_reset(md, host, type)) {
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
}
if (!req_pending) {
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
}
break;
@@ -1693,7 +1694,7 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
if (!mmc_blk_reset(md, host, type))
break;
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
case MMC_BLK_DATA_ERR: {
int err;
@@ -1702,7 +1703,7 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
break;
if (err == -ENODEV) {
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
}
/* Fall through */
@@ -1723,19 +1724,19 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
req_pending = blk_end_request(old_req, -EIO,
brq->data.blksz);
if (!req_pending) {
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
}
break;
case MMC_BLK_NOMEDIUM:
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
default:
pr_err("%s: Unhandled return value (%d)",
old_req->rq_disk->disk_name, status);
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq);
+ mmc_blk_rw_try_restart(mq_rq);
return;
}
@@ -1747,15 +1748,16 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
mmc_blk_rw_rq_prep(mq_rq, card,
disable_multi, mq);
mq_rq->brq.retune_retry_done = retune_retry_done;
- mmc_restart_areq(host, &mq->mqrq_cur->areq);
+ mmc_restart_areq(host, &mq_rq->areq);
}
}
-static void mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *new_req)
+static void mmc_blk_issue_rw_rq(struct mmc_queue_req *mq_rq)
{
+ struct mmc_queue *mq = mq_rq->mq;
struct mmc_card *card = mq->card;
- if (!new_req) {
+ if (!mq_rq->req) {
pr_err("%s: NULL request!\n", __func__);
return;
}
@@ -1765,54 +1767,52 @@ static void mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *new_req)
* multiple read or write is allowed
*/
if (mmc_large_sector(card) &&
- !IS_ALIGNED(blk_rq_sectors(new_req), 8)) {
+ !IS_ALIGNED(blk_rq_sectors(mq_rq->req), 8)) {
pr_err("%s: Transfer size is not 4KB sector size aligned\n",
- new_req->rq_disk->disk_name);
- mmc_blk_rw_cmd_abort(card, new_req);
+ mq_rq->req->rq_disk->disk_name);
+ mmc_blk_rw_cmd_abort(card, mq_rq->req);
return;
}
- mmc_blk_rw_rq_prep(mq->mqrq_cur, card, 0, mq);
- mmc_start_areq(card->host, &mq->mqrq_cur->areq);
+ mmc_blk_rw_rq_prep(mq_rq, card, 0, mq);
+ mmc_start_areq(card->host, &mq_rq->areq);
}
-void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
+void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq)
{
int ret;
- struct mmc_blk_data *md = mq->blkdata;
+ struct mmc_blk_data *md = mq_rq->mq->blkdata;
struct mmc_card *card = md->queue.card;
ret = mmc_blk_part_switch(card, md);
if (ret) {
- if (req) {
- blk_end_request_all(req, -EIO);
- }
+ blk_end_request_all(mq_rq->req, -EIO);
return;
}
- if (req && req_op(req) == REQ_OP_DISCARD) {
+ if (req_op(mq_rq->req) == REQ_OP_DISCARD) {
/* complete ongoing async transfer before issuing discard */
if (card->host->areq) {
wait_for_completion(&card->host->areq->complete);
card->host->areq = NULL;
}
- mmc_blk_issue_discard_rq(mq, req);
- } else if (req && req_op(req) == REQ_OP_SECURE_ERASE) {
+ mmc_blk_issue_discard_rq(mq_rq);
+ } else if (req_op(mq_rq->req) == REQ_OP_SECURE_ERASE) {
/* complete ongoing async transfer before issuing secure erase*/
if (card->host->areq) {
wait_for_completion(&card->host->areq->complete);
card->host->areq = NULL;
}
- mmc_blk_issue_secdiscard_rq(mq, req);
- } else if (req && req_op(req) == REQ_OP_FLUSH) {
+ mmc_blk_issue_secdiscard_rq(mq_rq);
+ } else if (req_op(mq_rq->req) == REQ_OP_FLUSH) {
/* complete ongoing async transfer before issuing flush */
if (card->host->areq) {
wait_for_completion(&card->host->areq->complete);
card->host->areq = NULL;
}
- mmc_blk_issue_flush(mq, req);
+ mmc_blk_issue_flush(mq_rq);
} else {
- mmc_blk_issue_rw_rq(mq, req);
+ mmc_blk_issue_rw_rq(mq_rq);
}
}
diff --git a/drivers/mmc/core/block.h b/drivers/mmc/core/block.h
index b4b489911599..0326fa5d8217 100644
--- a/drivers/mmc/core/block.h
+++ b/drivers/mmc/core/block.h
@@ -3,10 +3,9 @@
struct mmc_async_req;
enum mmc_blk_status;
-struct mmc_queue;
-struct request;
+struct mmc_queue_req;
void mmc_blk_rw_done(struct mmc_async_req *areq, enum mmc_blk_status status);
-void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req);
+void mmc_blk_issue_rq(struct mmc_queue_req *mq_rq);
#endif
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index c9f28de7b0f4..c4e1ced55796 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -54,6 +54,7 @@ static int mmc_queue_thread(void *d)
struct mmc_queue *mq = d;
struct request_queue *q = mq->queue;
bool claimed_host = false;
+ struct mmc_queue_req *mq_rq;
current->flags |= PF_MEMALLOC;
@@ -65,7 +66,8 @@ static int mmc_queue_thread(void *d)
set_current_state(TASK_INTERRUPTIBLE);
req = blk_fetch_request(q);
mq->asleep = false;
- mq->mqrq_cur->req = req;
+ mq_rq = mq->mqrq_cur;
+ mq_rq->req = req;
spin_unlock_irq(q->queue_lock);
if (req) {
@@ -74,7 +76,7 @@ static int mmc_queue_thread(void *d)
if (!claimed_host)
mmc_get_card(mq->card);
set_current_state(TASK_RUNNING);
- mmc_blk_issue_rq(mq, req);
+ mmc_blk_issue_rq(mq_rq);
cond_resched();
/*
* Current request becomes previous request
--
2.9.3
^ permalink raw reply related
* [PATCH 12/16] mmc: queue: stop flushing the pipeline with NULL
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
Remove all the pipeline flush: i.e. repeatedly sending NULL
down to the core layer to flush out asynchronous requests,
and also sending NULL after "special" commands to achieve the
same flush.
Instead: let the "special" commands wait for any ongoing
asynchronous transfers using the completion, and apart from
that expect the core.c and block.c layers to deal with the
ongoing requests autonomously without any "push" from the
queue.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 80 +++++++++++++++++-------------------------------
drivers/mmc/core/core.c | 37 ++++++++++------------
drivers/mmc/core/queue.c | 18 ++++++++---
include/linux/mmc/core.h | 5 ++-
4 files changed, 60 insertions(+), 80 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index 0bd9070f5f2e..4952a105780e 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -1753,42 +1753,27 @@ void mmc_blk_rw_done(struct mmc_async_req *areq,
static void mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *new_req)
{
- enum mmc_blk_status status;
- struct mmc_async_req *new_areq;
- struct mmc_async_req *old_areq;
struct mmc_card *card = mq->card;
- if (!new_req && !mq->mqrq_prev->req)
+ if (!new_req) {
+ pr_err("%s: NULL request!\n", __func__);
return;
+ }
- if (new_req) {
- /*
- * When 4KB native sector is enabled, only 8 blocks
- * multiple read or write is allowed
- */
- if (mmc_large_sector(card) &&
- !IS_ALIGNED(blk_rq_sectors(new_req), 8)) {
- pr_err("%s: Transfer size is not 4KB sector size aligned\n",
- new_req->rq_disk->disk_name);
- mmc_blk_rw_cmd_abort(card, new_req);
- return;
- }
-
- mmc_blk_rw_rq_prep(mq->mqrq_cur, card, 0, mq);
- new_areq = &mq->mqrq_cur->areq;
- } else
- new_areq = NULL;
-
- old_areq = mmc_start_areq(card->host, new_areq, &status);
- if (!old_areq) {
- /*
- * We have just put the first request into the pipeline
- * and there is nothing more to do until it is
- * complete.
- */
+ /*
+ * When 4KB native sector is enabled, only 8 blocks
+ * multiple read or write is allowed
+ */
+ if (mmc_large_sector(card) &&
+ !IS_ALIGNED(blk_rq_sectors(new_req), 8)) {
+ pr_err("%s: Transfer size is not 4KB sector size aligned\n",
+ new_req->rq_disk->disk_name);
+ mmc_blk_rw_cmd_abort(card, new_req);
return;
}
- /* FIXME: yes, we just disregard the old_areq */
+
+ mmc_blk_rw_rq_prep(mq->mqrq_cur, card, 0, mq);
+ mmc_start_areq(card->host, &mq->mqrq_cur->areq);
}
void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
@@ -1796,48 +1781,39 @@ void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
int ret;
struct mmc_blk_data *md = mq->blkdata;
struct mmc_card *card = md->queue.card;
- bool req_is_special = mmc_req_is_special(req);
-
- if (req && !mq->mqrq_prev->req)
- /* claim host only for the first request */
- mmc_get_card(card);
ret = mmc_blk_part_switch(card, md);
if (ret) {
if (req) {
blk_end_request_all(req, -EIO);
}
- goto out;
+ return;
}
if (req && req_op(req) == REQ_OP_DISCARD) {
/* complete ongoing async transfer before issuing discard */
- if (card->host->areq)
- mmc_blk_issue_rw_rq(mq, NULL);
+ if (card->host->areq) {
+ wait_for_completion(&card->host->areq->complete);
+ card->host->areq = NULL;
+ }
mmc_blk_issue_discard_rq(mq, req);
} else if (req && req_op(req) == REQ_OP_SECURE_ERASE) {
/* complete ongoing async transfer before issuing secure erase*/
- if (card->host->areq)
- mmc_blk_issue_rw_rq(mq, NULL);
+ if (card->host->areq) {
+ wait_for_completion(&card->host->areq->complete);
+ card->host->areq = NULL;
+ }
mmc_blk_issue_secdiscard_rq(mq, req);
} else if (req && req_op(req) == REQ_OP_FLUSH) {
/* complete ongoing async transfer before issuing flush */
- if (card->host->areq)
- mmc_blk_issue_rw_rq(mq, NULL);
+ if (card->host->areq) {
+ wait_for_completion(&card->host->areq->complete);
+ card->host->areq = NULL;
+ }
mmc_blk_issue_flush(mq, req);
} else {
mmc_blk_issue_rw_rq(mq, req);
}
-
-out:
- if (!req || req_is_special)
- /*
- * Release host when there are no more requests
- * and after special request(discard, flush) is done.
- * In case sepecial request, there is no reentry to
- * the 'mmc_blk_issue_rq' with 'mqrq_prev->req'.
- */
- mmc_put_card(card);
}
static inline int mmc_blk_readonly(struct mmc_card *card)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 34337ef6705e..03c290e5e2c9 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -667,42 +667,37 @@ EXPORT_SYMBOL(mmc_restart_areq);
* return the completed request. If there is no ongoing request, NULL
* is returned without waiting. NULL is not an error condition.
*/
-struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
- struct mmc_async_req *areq,
- enum mmc_blk_status *ret_stat)
+int mmc_start_areq(struct mmc_host *host,
+ struct mmc_async_req *areq)
{
- enum mmc_blk_status status;
- int start_err = 0;
+ int ret;
struct mmc_async_req *previous = host->areq;
/* Prepare a new request */
- if (areq)
- mmc_pre_req(host, areq->mrq);
+ if (!areq) {
+ pr_err("%s: NULL asynchronous request!\n", __func__);
+ return -EIO;
+ }
+
+ mmc_pre_req(host, areq->mrq);
/* Finalize previous request, if there is one */
if (previous)
wait_for_completion(&previous->complete);
- status = MMC_BLK_SUCCESS;
- if (ret_stat)
- *ret_stat = status;
-
/* Fine so far, start the new request! */
- if (status == MMC_BLK_SUCCESS && areq) {
- init_completion(&areq->complete);
- start_err = __mmc_start_data_req(host, areq->mrq);
- }
+ init_completion(&areq->complete);
+ ret = __mmc_start_data_req(host, areq->mrq);
/* Cancel a prepared request if it was not started. */
- if ((status != MMC_BLK_SUCCESS || start_err) && areq)
+ if (ret) {
mmc_post_req(host, areq->mrq, -EINVAL);
-
- if (status != MMC_BLK_SUCCESS)
host->areq = NULL;
- else
- host->areq = areq;
+ pr_err("%s: failed to start request\n", __func__);
+ }
+ host->areq = areq;
- return previous;
+ return ret;
}
EXPORT_SYMBOL(mmc_start_areq);
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index ae6837317fe0..c9f28de7b0f4 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -53,6 +53,7 @@ static int mmc_queue_thread(void *d)
{
struct mmc_queue *mq = d;
struct request_queue *q = mq->queue;
+ bool claimed_host = false;
current->flags |= PF_MEMALLOC;
@@ -67,9 +68,11 @@ static int mmc_queue_thread(void *d)
mq->mqrq_cur->req = req;
spin_unlock_irq(q->queue_lock);
- if (req || mq->mqrq_prev->req) {
+ if (req) {
bool req_is_special = mmc_req_is_special(req);
+ if (!claimed_host)
+ mmc_get_card(mq->card);
set_current_state(TASK_RUNNING);
mmc_blk_issue_rq(mq, req);
cond_resched();
@@ -78,11 +81,14 @@ static int mmc_queue_thread(void *d)
* and vice versa.
* In case of special requests, current request
* has been finished. Do not assign it to previous
- * request.
+ * request. Always unclaim the host after special
+ * commands.
*/
- if (req_is_special)
+ if (req_is_special) {
mq->mqrq_cur->req = NULL;
-
+ mmc_put_card(mq->card);
+ claimed_host = false;
+ }
mq->mqrq_prev->brq.mrq.data = NULL;
mq->mqrq_prev->req = NULL;
swap(mq->mqrq_prev, mq->mqrq_cur);
@@ -97,6 +103,10 @@ static int mmc_queue_thread(void *d)
down(&mq->thread_sem);
}
} while (1);
+
+ if (claimed_host)
+ mmc_put_card(mq->card);
+
up(&mq->thread_sem);
return 0;
diff --git a/include/linux/mmc/core.h b/include/linux/mmc/core.h
index 55b45dcddee6..af651e723ba2 100644
--- a/include/linux/mmc/core.h
+++ b/include/linux/mmc/core.h
@@ -160,9 +160,8 @@ struct mmc_async_req;
void mmc_finalize_areq(struct kthread_work *work);
int mmc_restart_areq(struct mmc_host *host, struct mmc_async_req *areq);
-struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
- struct mmc_async_req *areq,
- enum mmc_blk_status *ret_stat);
+int mmc_start_areq(struct mmc_host *host,
+ struct mmc_async_req *areq);
void mmc_wait_for_req(struct mmc_host *host, struct mmc_request *mrq);
int mmc_wait_for_cmd(struct mmc_host *host, struct mmc_command *cmd,
int retries);
--
2.9.3
^ permalink raw reply related
* [PATCH 11/16] mmc: block: shuffle retry and error handling
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
Instead of doing retries at the same time as trying to submit new
requests, do the retries when the request is reported as completed
by the driver, in the finalization worker.
This is achieved by letting the core worker call back into the block
layer using mmc_blk_rw_done(), that will read the status and repeatedly
try to hammer the request using single request etc by calling back to
the core layer using mmc_restart_areq()
The beauty of it is that the completion will not complete until the
block layer has had the opportunity to hammer a bit at the card using
a bunch of different approaches in the while() loop in
mmc_blk_rw_done()
The algorithm for recapture, retry and handle errors is essentially
identical to the one we used to have in mmc_blk_issue_rw_rq(),
only augmented to get called in another path.
We have to add and initialize a pointer back to the struct mmc_queue
from the struct mmc_queue_req to find the queue from the asynchronous
request.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 307 +++++++++++++++++++++++------------------------
drivers/mmc/core/block.h | 3 +
drivers/mmc/core/core.c | 23 +++-
drivers/mmc/core/queue.c | 2 +
drivers/mmc/core/queue.h | 1 +
include/linux/mmc/core.h | 1 +
include/linux/mmc/host.h | 1 -
7 files changed, 177 insertions(+), 161 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index c459d80c66bf..0bd9070f5f2e 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -1614,182 +1614,181 @@ static void mmc_blk_rw_cmd_abort(struct mmc_card *card, struct request *req)
* @mq: the queue with the card and host to restart
* @req: a new request that want to be started after the current one
*/
-static void mmc_blk_rw_try_restart(struct mmc_queue *mq, struct request *req)
+static void mmc_blk_rw_try_restart(struct mmc_queue *mq)
{
- if (!req)
- return;
-
- /*
- * If the card was removed, just cancel everything and return.
- */
- if (mmc_card_removed(mq->card)) {
- req->rq_flags |= RQF_QUIET;
- blk_end_request_all(req, -EIO);
- return;
- }
- /* Else proceed and try to restart the current async request */
+ /* Proceed and try to restart the current async request */
mmc_blk_rw_rq_prep(mq->mqrq_cur, mq->card, 0, mq);
- mmc_start_areq(mq->card->host, &mq->mqrq_cur->areq, NULL);
+ mmc_restart_areq(mq->card->host, &mq->mqrq_cur->areq);
}
-static void mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *new_req)
+void mmc_blk_rw_done(struct mmc_async_req *areq,
+ enum mmc_blk_status status)
{
- struct mmc_blk_data *md = mq->blkdata;
- struct mmc_card *card = md->queue.card;
- struct mmc_blk_request *brq;
- int disable_multi = 0, retry = 0, type, retune_retry_done = 0;
- enum mmc_blk_status status;
+ struct mmc_queue *mq;
struct mmc_queue_req *mq_rq;
+ struct mmc_blk_request *brq;
+ struct mmc_blk_data *md;
struct request *old_req;
- struct mmc_async_req *new_areq;
- struct mmc_async_req *old_areq;
+ struct mmc_card *card;
+ struct mmc_host *host;
+ int disable_multi = 0, retry = 0, type, retune_retry_done = 0;
bool req_pending = true;
- if (!new_req && !mq->mqrq_prev->req)
- return;
-
- do {
- if (new_req) {
- /*
- * When 4KB native sector is enabled, only 8 blocks
- * multiple read or write is allowed
- */
- if (mmc_large_sector(card) &&
- !IS_ALIGNED(blk_rq_sectors(new_req), 8)) {
- pr_err("%s: Transfer size is not 4KB sector size aligned\n",
- new_req->rq_disk->disk_name);
- mmc_blk_rw_cmd_abort(card, new_req);
- return;
- }
-
- mmc_blk_rw_rq_prep(mq->mqrq_cur, card, 0, mq);
- new_areq = &mq->mqrq_cur->areq;
- } else
- new_areq = NULL;
-
- old_areq = mmc_start_areq(card->host, new_areq, &status);
- if (!old_areq) {
- /*
- * We have just put the first request into the pipeline
- * and there is nothing more to do until it is
- * complete.
- */
- return;
- }
-
+ /*
+ * An asynchronous request has been completed and we proceed
+ * to handle the result of it.
+ */
+ mq_rq = container_of(areq, struct mmc_queue_req, areq);
+ mq = mq_rq->mq;
+ md = mq->blkdata;
+ card = mq->card;
+ host = card->host;
+ brq = &mq_rq->brq;
+ old_req = mq_rq->req;
+ type = rq_data_dir(old_req) == READ ? MMC_BLK_READ : MMC_BLK_WRITE;
+
+ mmc_queue_bounce_post(mq_rq);
+
+ switch (status) {
+ case MMC_BLK_SUCCESS:
+ case MMC_BLK_PARTIAL:
/*
- * An asynchronous request has been completed and we proceed
- * to handle the result of it.
+ * A block was successfully transferred.
*/
- mq_rq = container_of(old_areq, struct mmc_queue_req, areq);
- brq = &mq_rq->brq;
- old_req = mq_rq->req;
- type = rq_data_dir(old_req) == READ ? MMC_BLK_READ : MMC_BLK_WRITE;
- mmc_queue_bounce_post(mq_rq);
-
- switch (status) {
- case MMC_BLK_SUCCESS:
- case MMC_BLK_PARTIAL:
- /*
- * A block was successfully transferred.
- */
- mmc_blk_reset_success(md, type);
+ mmc_blk_reset_success(md, type);
- req_pending = blk_end_request(old_req, 0,
- brq->data.bytes_xfered);
- /*
- * If the blk_end_request function returns non-zero even
- * though all data has been transferred and no errors
- * were returned by the host controller, it's a bug.
- */
- if (status == MMC_BLK_SUCCESS && req_pending) {
- pr_err("%s BUG rq_tot %d d_xfer %d\n",
- __func__, blk_rq_bytes(old_req),
- brq->data.bytes_xfered);
- mmc_blk_rw_cmd_abort(card, old_req);
- return;
- }
- break;
- case MMC_BLK_CMD_ERR:
- req_pending = mmc_blk_rw_cmd_err(md, card, brq, old_req, req_pending);
- if (mmc_blk_reset(md, card->host, type)) {
- mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq, new_req);
- return;
- }
- if (!req_pending) {
- mmc_blk_rw_try_restart(mq, new_req);
- return;
- }
- break;
- case MMC_BLK_RETRY:
- retune_retry_done = brq->retune_retry_done;
- if (retry++ < 5)
- break;
- /* Fall through */
- case MMC_BLK_ABORT:
- if (!mmc_blk_reset(md, card->host, type))
- break;
+ req_pending = blk_end_request(old_req, 0,
+ brq->data.bytes_xfered);
+ /*
+ * If the blk_end_request function returns non-zero even
+ * though all data has been transferred and no errors
+ * were returned by the host controller, it's a bug.
+ */
+ if (status == MMC_BLK_SUCCESS && req_pending) {
+ pr_err("%s BUG rq_tot %d d_xfer %d\n",
+ __func__, blk_rq_bytes(old_req),
+ brq->data.bytes_xfered);
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq, new_req);
return;
- case MMC_BLK_DATA_ERR: {
- int err;
-
- err = mmc_blk_reset(md, card->host, type);
- if (!err)
- break;
- if (err == -ENODEV) {
- mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq, new_req);
- return;
- }
- /* Fall through */
}
- case MMC_BLK_ECC_ERR:
- if (brq->data.blocks > 1) {
- /* Redo read one sector at a time */
- pr_warn("%s: retrying using single block read\n",
- old_req->rq_disk->disk_name);
- disable_multi = 1;
- break;
- }
- /*
- * After an error, we redo I/O one sector at a
- * time, so we only reach here after trying to
- * read a single sector.
- */
- req_pending = blk_end_request(old_req, -EIO,
- brq->data.blksz);
- if (!req_pending) {
- mmc_blk_rw_try_restart(mq, new_req);
- return;
- }
- break;
- case MMC_BLK_NOMEDIUM:
+ break;
+ case MMC_BLK_CMD_ERR:
+ req_pending = mmc_blk_rw_cmd_err(md, card, brq, old_req, req_pending);
+ if (mmc_blk_reset(md, host, type)) {
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq, new_req);
+ mmc_blk_rw_try_restart(mq);
return;
- default:
- pr_err("%s: Unhandled return value (%d)",
- old_req->rq_disk->disk_name, status);
+ }
+ if (!req_pending) {
+ mmc_blk_rw_try_restart(mq);
+ return;
+ }
+ break;
+ case MMC_BLK_RETRY:
+ retune_retry_done = brq->retune_retry_done;
+ if (retry++ < 5)
+ break;
+ /* Fall through */
+ case MMC_BLK_ABORT:
+ if (!mmc_blk_reset(md, host, type))
+ break;
+ mmc_blk_rw_cmd_abort(card, old_req);
+ mmc_blk_rw_try_restart(mq);
+ return;
+ case MMC_BLK_DATA_ERR: {
+ int err;
+ err = mmc_blk_reset(md, host, type);
+ if (!err)
+ break;
+ if (err == -ENODEV) {
mmc_blk_rw_cmd_abort(card, old_req);
- mmc_blk_rw_try_restart(mq, new_req);
+ mmc_blk_rw_try_restart(mq);
return;
}
+ /* Fall through */
+ }
+ case MMC_BLK_ECC_ERR:
+ if (brq->data.blocks > 1) {
+ /* Redo read one sector at a time */
+ pr_warn("%s: retrying using single block read\n",
+ old_req->rq_disk->disk_name);
+ disable_multi = 1;
+ break;
+ }
+ /*
+ * After an error, we redo I/O one sector at a
+ * time, so we only reach here after trying to
+ * read a single sector.
+ */
+ req_pending = blk_end_request(old_req, -EIO,
+ brq->data.blksz);
+ if (!req_pending) {
+ mmc_blk_rw_try_restart(mq);
+ return;
+ }
+ break;
+ case MMC_BLK_NOMEDIUM:
+ mmc_blk_rw_cmd_abort(card, old_req);
+ mmc_blk_rw_try_restart(mq);
+ return;
+ default:
+ pr_err("%s: Unhandled return value (%d)",
+ old_req->rq_disk->disk_name, status);
+ mmc_blk_rw_cmd_abort(card, old_req);
+ mmc_blk_rw_try_restart(mq);
+ return;
+ }
- if (req_pending) {
- /*
- * In case of a incomplete request
- * prepare it again and resend.
- */
- mmc_blk_rw_rq_prep(mq_rq, card,
- disable_multi, mq);
- mmc_start_areq(card->host,
- &mq_rq->areq, NULL);
- mq_rq->brq.retune_retry_done = retune_retry_done;
+ if (req_pending) {
+ /*
+ * In case of a incomplete request
+ * prepare it again and resend.
+ */
+ mmc_blk_rw_rq_prep(mq_rq, card,
+ disable_multi, mq);
+ mq_rq->brq.retune_retry_done = retune_retry_done;
+ mmc_restart_areq(host, &mq->mqrq_cur->areq);
+ }
+}
+
+static void mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *new_req)
+{
+ enum mmc_blk_status status;
+ struct mmc_async_req *new_areq;
+ struct mmc_async_req *old_areq;
+ struct mmc_card *card = mq->card;
+
+ if (!new_req && !mq->mqrq_prev->req)
+ return;
+
+ if (new_req) {
+ /*
+ * When 4KB native sector is enabled, only 8 blocks
+ * multiple read or write is allowed
+ */
+ if (mmc_large_sector(card) &&
+ !IS_ALIGNED(blk_rq_sectors(new_req), 8)) {
+ pr_err("%s: Transfer size is not 4KB sector size aligned\n",
+ new_req->rq_disk->disk_name);
+ mmc_blk_rw_cmd_abort(card, new_req);
+ return;
}
- } while (req_pending);
+
+ mmc_blk_rw_rq_prep(mq->mqrq_cur, card, 0, mq);
+ new_areq = &mq->mqrq_cur->areq;
+ } else
+ new_areq = NULL;
+
+ old_areq = mmc_start_areq(card->host, new_areq, &status);
+ if (!old_areq) {
+ /*
+ * We have just put the first request into the pipeline
+ * and there is nothing more to do until it is
+ * complete.
+ */
+ return;
+ }
+ /* FIXME: yes, we just disregard the old_areq */
}
void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
diff --git a/drivers/mmc/core/block.h b/drivers/mmc/core/block.h
index 860ca7c8df86..b4b489911599 100644
--- a/drivers/mmc/core/block.h
+++ b/drivers/mmc/core/block.h
@@ -1,9 +1,12 @@
#ifndef _MMC_CORE_BLOCK_H
#define _MMC_CORE_BLOCK_H
+struct mmc_async_req;
+enum mmc_blk_status;
struct mmc_queue;
struct request;
+void mmc_blk_rw_done(struct mmc_async_req *areq, enum mmc_blk_status status);
void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req);
#endif
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 4b84f18518ac..34337ef6705e 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -39,6 +39,7 @@
#define CREATE_TRACE_POINTS
#include <trace/events/mmc.h>
+#include "block.h"
#include "core.h"
#include "card.h"
#include "bus.h"
@@ -632,13 +633,25 @@ void mmc_finalize_areq(struct kthread_work *work)
/* Successfully postprocess the old request at this point */
mmc_post_req(host, areq->mrq, 0);
+ mmc_blk_rw_done(areq, status);
- areq->finalization_status = status;
complete(&areq->complete);
}
EXPORT_SYMBOL(mmc_finalize_areq);
/**
+ * mmc_restart_areq() - restart an asynchronous request
+ * @host: MMC host to restart the command on
+ * @areq: the asynchronous request to restart
+ */
+int mmc_restart_areq(struct mmc_host *host,
+ struct mmc_async_req *areq)
+{
+ return __mmc_start_data_req(host, areq->mrq);
+}
+EXPORT_SYMBOL(mmc_restart_areq);
+
+/**
* mmc_start_areq - start an asynchronous request
* @host: MMC host to start command
* @areq: asynchronous request to start
@@ -667,12 +680,10 @@ struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
mmc_pre_req(host, areq->mrq);
/* Finalize previous request, if there is one */
- if (previous) {
+ if (previous)
wait_for_completion(&previous->complete);
- status = previous->finalization_status;
- } else {
- status = MMC_BLK_SUCCESS;
- }
+
+ status = MMC_BLK_SUCCESS;
if (ret_stat)
*ret_stat = status;
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index bc116709c806..ae6837317fe0 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -268,7 +268,9 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card,
if (!mq->mqrq)
goto blk_cleanup;
mq->mqrq_cur = &mq->mqrq[0];
+ mq->mqrq_cur->mq = mq;
mq->mqrq_prev = &mq->mqrq[1];
+ mq->mqrq_prev->mq = mq;
mq->queue->queuedata = mq;
blk_queue_prep_rq(mq->queue, mmc_prep_request);
diff --git a/drivers/mmc/core/queue.h b/drivers/mmc/core/queue.h
index 39d8e710287e..c18d3f908433 100644
--- a/drivers/mmc/core/queue.h
+++ b/drivers/mmc/core/queue.h
@@ -34,6 +34,7 @@ struct mmc_queue_req {
struct scatterlist *bounce_sg;
unsigned int bounce_sg_len;
struct mmc_async_req areq;
+ struct mmc_queue *mq;
};
struct mmc_queue {
diff --git a/include/linux/mmc/core.h b/include/linux/mmc/core.h
index 5db0fb722c37..55b45dcddee6 100644
--- a/include/linux/mmc/core.h
+++ b/include/linux/mmc/core.h
@@ -159,6 +159,7 @@ struct mmc_card;
struct mmc_async_req;
void mmc_finalize_areq(struct kthread_work *work);
+int mmc_restart_areq(struct mmc_host *host, struct mmc_async_req *areq);
struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
struct mmc_async_req *areq,
enum mmc_blk_status *ret_stat);
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index a7c0ed887391..47d80b8470cd 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -171,7 +171,6 @@ struct mmc_async_req {
*/
enum mmc_blk_status (*err_check)(struct mmc_card *, struct mmc_async_req *);
struct kthread_work finalization_work;
- enum mmc_blk_status finalization_status;
struct completion complete;
struct mmc_host *host;
};
--
2.9.3
^ permalink raw reply related
* [PATCH 10/16] mmc: queue: simplify queue logic
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
The if() statment checking if there is no current or previous
request is now just looking ahead at something that will be
concluded a few lines below. Simplify the logic by moving the
assignment of .asleep.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/queue.c | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index a845fe8d4fd1..bc116709c806 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -64,14 +64,6 @@ static int mmc_queue_thread(void *d)
set_current_state(TASK_INTERRUPTIBLE);
req = blk_fetch_request(q);
mq->asleep = false;
- if (!req) {
- /*
- * Dispatch queue is empty so set flags for
- * mmc_request_fn() to wake us up.
- */
- if (!mq->mqrq_prev->req)
- mq->asleep = true;
- }
mq->mqrq_cur->req = req;
spin_unlock_irq(q->queue_lock);
@@ -95,6 +87,7 @@ static int mmc_queue_thread(void *d)
mq->mqrq_prev->req = NULL;
swap(mq->mqrq_prev, mq->mqrq_cur);
} else {
+ mq->asleep = true;
if (kthread_should_stop()) {
set_current_state(TASK_RUNNING);
break;
--
2.9.3
^ permalink raw reply related
* [PATCH 09/16] mmc: core: kill off the context info
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
The last member of the context info: is_waiting_last_req is
just assigned values, never checked. Delete that and the whole
context info as a result.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/bus.c | 1 -
drivers/mmc/core/core.c | 13 -------------
drivers/mmc/core/core.h | 2 --
drivers/mmc/core/queue.c | 9 +--------
include/linux/mmc/host.h | 9 ---------
5 files changed, 1 insertion(+), 33 deletions(-)
diff --git a/drivers/mmc/core/bus.c b/drivers/mmc/core/bus.c
index 301246513a37..22ed11ac961b 100644
--- a/drivers/mmc/core/bus.c
+++ b/drivers/mmc/core/bus.c
@@ -348,7 +348,6 @@ int mmc_add_card(struct mmc_card *card)
#ifdef CONFIG_DEBUG_FS
mmc_add_card_debugfs(card);
#endif
- mmc_init_context_info(card->host);
card->dev.of_node = mmc_of_find_child_device(card->host, 0);
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 933a4d1f20d5..4b84f18518ac 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -2975,19 +2975,6 @@ void mmc_unregister_pm_notifier(struct mmc_host *host)
}
#endif
-/**
- * mmc_init_context_info() - init synchronization context
- * @host: mmc host
- *
- * Init struct context_info needed to implement asynchronous
- * request mechanism, used by mmc core, host driver and mmc requests
- * supplier.
- */
-void mmc_init_context_info(struct mmc_host *host)
-{
- host->context_info.is_waiting_last_req = false;
-}
-
static int __init mmc_init(void)
{
int ret;
diff --git a/drivers/mmc/core/core.h b/drivers/mmc/core/core.h
index 8a95c82554be..620bea373c3a 100644
--- a/drivers/mmc/core/core.h
+++ b/drivers/mmc/core/core.h
@@ -90,8 +90,6 @@ void mmc_remove_host_debugfs(struct mmc_host *host);
void mmc_add_card_debugfs(struct mmc_card *card);
void mmc_remove_card_debugfs(struct mmc_card *card);
-void mmc_init_context_info(struct mmc_host *host);
-
int mmc_execute_tuning(struct mmc_card *card);
int mmc_hs200_to_hs400(struct mmc_card *card);
int mmc_hs400_to_hs200(struct mmc_card *card);
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index 63927ffd6825..a845fe8d4fd1 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -53,7 +53,6 @@ static int mmc_queue_thread(void *d)
{
struct mmc_queue *mq = d;
struct request_queue *q = mq->queue;
- struct mmc_context_info *cntx = &mq->card->host->context_info;
current->flags |= PF_MEMALLOC;
@@ -65,15 +64,12 @@ static int mmc_queue_thread(void *d)
set_current_state(TASK_INTERRUPTIBLE);
req = blk_fetch_request(q);
mq->asleep = false;
- cntx->is_waiting_last_req = false;
if (!req) {
/*
* Dispatch queue is empty so set flags for
* mmc_request_fn() to wake us up.
*/
- if (mq->mqrq_prev->req)
- cntx->is_waiting_last_req = true;
- else
+ if (!mq->mqrq_prev->req)
mq->asleep = true;
}
mq->mqrq_cur->req = req;
@@ -123,7 +119,6 @@ static void mmc_request_fn(struct request_queue *q)
{
struct mmc_queue *mq = q->queuedata;
struct request *req;
- struct mmc_context_info *cntx;
if (!mq) {
while ((req = blk_fetch_request(q)) != NULL) {
@@ -133,8 +128,6 @@ static void mmc_request_fn(struct request_queue *q)
return;
}
- cntx = &mq->card->host->context_info;
-
if (mq->asleep)
wake_up_process(mq->thread);
}
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index 970d7f9b1eba..a7c0ed887391 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -192,14 +192,6 @@ struct mmc_slot {
void *handler_priv;
};
-/**
- * mmc_context_info - synchronization details for mmc context
- * @is_waiting_last_req mmc context waiting for single running request
- */
-struct mmc_context_info {
- bool is_waiting_last_req;
-};
-
struct regulator;
struct mmc_pwrseq;
@@ -373,7 +365,6 @@ struct mmc_host {
struct dentry *debugfs_root;
struct mmc_async_req *areq; /* active async req */
- struct mmc_context_info context_info; /* async synchronization info */
/* finalization work thread, handles finalizing requests */
struct kthread_worker req_done_worker;
--
2.9.3
^ permalink raw reply related
* [PATCH 08/16] mmc: core: do away with is_new_req
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
The host context member "is_new_req" is only assigned values,
never checked. Delete it.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 1 -
drivers/mmc/core/queue.c | 5 -----
include/linux/mmc/host.h | 2 --
3 files changed, 8 deletions(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index fcb40ade9b82..933a4d1f20d5 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -2985,7 +2985,6 @@ void mmc_unregister_pm_notifier(struct mmc_host *host)
*/
void mmc_init_context_info(struct mmc_host *host)
{
- host->context_info.is_new_req = false;
host->context_info.is_waiting_last_req = false;
}
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index 73250ed8f093..63927ffd6825 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -66,7 +66,6 @@ static int mmc_queue_thread(void *d)
req = blk_fetch_request(q);
mq->asleep = false;
cntx->is_waiting_last_req = false;
- cntx->is_new_req = false;
if (!req) {
/*
* Dispatch queue is empty so set flags for
@@ -136,10 +135,6 @@ static void mmc_request_fn(struct request_queue *q)
cntx = &mq->card->host->context_info;
- if (cntx->is_waiting_last_req) {
- cntx->is_new_req = true;
- }
-
if (mq->asleep)
wake_up_process(mq->thread);
}
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index cbb40682024a..970d7f9b1eba 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -194,11 +194,9 @@ struct mmc_slot {
/**
* mmc_context_info - synchronization details for mmc context
- * @is_new_req wake up reason was new request
* @is_waiting_last_req mmc context waiting for single running request
*/
struct mmc_context_info {
- bool is_new_req;
bool is_waiting_last_req;
};
--
2.9.3
^ permalink raw reply related
* [PATCH 07/16] mmc: core: do away with is_done_rcv
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
The "is_done_rcv" in the context info for the host is no longer
needed: it is clear from context (ha!) that as long as we are
waiting for the asynchronous request to come to completion,
we are not done receiving data, and when the finalization work
has run and completed the completion, we are indeed done.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 40 ++++++++++++++++------------------------
include/linux/mmc/host.h | 2 --
2 files changed, 16 insertions(+), 26 deletions(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 8ecf61e51662..fcb40ade9b82 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -416,10 +416,8 @@ EXPORT_SYMBOL(mmc_start_bkops);
static void mmc_wait_data_done(struct mmc_request *mrq)
{
struct mmc_host *host = mrq->host;
- struct mmc_context_info *context_info = &host->context_info;
struct mmc_async_req *areq = host->areq;
- context_info->is_done_rcv = true;
/* Schedule a work to deal with finalizing this request */
kthread_queue_work(&host->req_done_worker, &areq->finalization_work);
}
@@ -551,7 +549,7 @@ EXPORT_SYMBOL(mmc_wait_for_req_done);
bool mmc_is_req_done(struct mmc_host *host, struct mmc_request *mrq)
{
if (host->areq)
- return host->context_info.is_done_rcv;
+ return completion_done(&host->areq->complete);
else
return completion_done(&mrq->completion);
}
@@ -600,29 +598,24 @@ void mmc_finalize_areq(struct kthread_work *work)
struct mmc_async_req *areq =
container_of(work, struct mmc_async_req, finalization_work);
struct mmc_host *host = areq->host;
- struct mmc_context_info *context_info = &host->context_info;
enum mmc_blk_status status = MMC_BLK_SUCCESS;
+ struct mmc_command *cmd;
- if (context_info->is_done_rcv) {
- struct mmc_command *cmd;
-
- context_info->is_done_rcv = false;
- cmd = areq->mrq->cmd;
+ cmd = areq->mrq->cmd;
- if (!cmd->error || !cmd->retries ||
- mmc_card_removed(host->card)) {
- status = areq->err_check(host->card,
- areq);
- } else {
- mmc_retune_recheck(host);
- pr_info("%s: req failed (CMD%u): %d, retrying...\n",
- mmc_hostname(host),
- cmd->opcode, cmd->error);
- cmd->retries--;
- cmd->error = 0;
- __mmc_start_request(host, areq->mrq);
- return; /* wait for done/new event again */
- }
+ if (!cmd->error || !cmd->retries ||
+ mmc_card_removed(host->card)) {
+ status = areq->err_check(host->card,
+ areq);
+ } else {
+ mmc_retune_recheck(host);
+ pr_info("%s: req failed (CMD%u): %d, retrying...\n",
+ mmc_hostname(host),
+ cmd->opcode, cmd->error);
+ cmd->retries--;
+ cmd->error = 0;
+ __mmc_start_request(host, areq->mrq);
+ return; /* wait for done/new event again */
}
mmc_retune_release(host);
@@ -2993,7 +2986,6 @@ void mmc_unregister_pm_notifier(struct mmc_host *host)
void mmc_init_context_info(struct mmc_host *host)
{
host->context_info.is_new_req = false;
- host->context_info.is_done_rcv = false;
host->context_info.is_waiting_last_req = false;
}
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index c5f61f2f2310..cbb40682024a 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -194,12 +194,10 @@ struct mmc_slot {
/**
* mmc_context_info - synchronization details for mmc context
- * @is_done_rcv wake up reason was done request
* @is_new_req wake up reason was new request
* @is_waiting_last_req mmc context waiting for single running request
*/
struct mmc_context_info {
- bool is_done_rcv;
bool is_new_req;
bool is_waiting_last_req;
};
--
2.9.3
^ permalink raw reply related
* [PATCH 06/16] mmc: core: replace waitqueue with worker
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
The waitqueue in the host context is there to signal back from
mmc_request_done() through mmc_wait_data_done() that the hardware
is done with a command, and when the wait is over, the core
will typically submit the next asynchronous request that is pending
just waiting for the hardware to be available.
This is in the way for letting the mmc_request_done() trigger the
report up to the block layer that a block request is finished.
Re-jig this as a first step, remvoving the waitqueue and introducing
a work that will run after a completed asynchronous request,
finalizing that request, including retransmissions, and eventually
reporting back with a completion and a status code to the
asynchronous issue method.
This had the upside that we can remove the MMC_BLK_NEW_REQUEST
status code and the "new_request" state in the request queue
that is only there to make the state machine spin out
the first time we send a request.
Introduce a workqueue in the host for handling just this, and
then a work and completion in the asynchronous request to deal
with this mechanism.
This is a central change that let us do many other changes since
we have broken the submit and complete code paths in two, and we
can potentially remove the NULL flushing of the asynchronous
pipeline and report block requests as finished directly from
the worker.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/block.c | 7 ++--
drivers/mmc/core/core.c | 84 +++++++++++++++++++++++-------------------------
drivers/mmc/core/queue.c | 6 ----
drivers/mmc/core/queue.h | 1 -
include/linux/mmc/core.h | 3 +-
include/linux/mmc/host.h | 7 ++--
6 files changed, 51 insertions(+), 57 deletions(-)
diff --git a/drivers/mmc/core/block.c b/drivers/mmc/core/block.c
index c49c90dba839..c459d80c66bf 100644
--- a/drivers/mmc/core/block.c
+++ b/drivers/mmc/core/block.c
@@ -1562,6 +1562,8 @@ static void mmc_blk_rw_rq_prep(struct mmc_queue_req *mqrq,
mqrq->areq.mrq = &brq->mrq;
mqrq->areq.err_check = mmc_blk_err_check;
+ mqrq->areq.host = card->host;
+ kthread_init_work(&mqrq->areq.finalization_work, mmc_finalize_areq);
mmc_queue_bounce_pre(mqrq);
}
@@ -1672,8 +1674,6 @@ static void mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *new_req)
* and there is nothing more to do until it is
* complete.
*/
- if (status == MMC_BLK_NEW_REQUEST)
- mq->new_request = true;
return;
}
@@ -1811,7 +1811,6 @@ void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
goto out;
}
- mq->new_request = false;
if (req && req_op(req) == REQ_OP_DISCARD) {
/* complete ongoing async transfer before issuing discard */
if (card->host->areq)
@@ -1832,7 +1831,7 @@ void mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
}
out:
- if ((!req && !mq->new_request) || req_is_special)
+ if (!req || req_is_special)
/*
* Release host when there are no more requests
* and after special request(discard, flush) is done.
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 663799240635..8ecf61e51662 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -415,10 +415,13 @@ EXPORT_SYMBOL(mmc_start_bkops);
*/
static void mmc_wait_data_done(struct mmc_request *mrq)
{
- struct mmc_context_info *context_info = &mrq->host->context_info;
+ struct mmc_host *host = mrq->host;
+ struct mmc_context_info *context_info = &host->context_info;
+ struct mmc_async_req *areq = host->areq;
context_info->is_done_rcv = true;
- wake_up_interruptible(&context_info->wait);
+ /* Schedule a work to deal with finalizing this request */
+ kthread_queue_work(&host->req_done_worker, &areq->finalization_work);
}
static void mmc_wait_done(struct mmc_request *mrq)
@@ -592,43 +595,34 @@ static void mmc_post_req(struct mmc_host *host, struct mmc_request *mrq,
* will return MMC_BLK_SUCCESS if no request was
* going on.
*/
-static enum mmc_blk_status mmc_finalize_areq(struct mmc_host *host)
+void mmc_finalize_areq(struct kthread_work *work)
{
+ struct mmc_async_req *areq =
+ container_of(work, struct mmc_async_req, finalization_work);
+ struct mmc_host *host = areq->host;
struct mmc_context_info *context_info = &host->context_info;
- enum mmc_blk_status status;
-
- if (!host->areq)
- return MMC_BLK_SUCCESS;
-
- while (1) {
- wait_event_interruptible(context_info->wait,
- (context_info->is_done_rcv ||
- context_info->is_new_req));
+ enum mmc_blk_status status = MMC_BLK_SUCCESS;
- if (context_info->is_done_rcv) {
- struct mmc_command *cmd;
+ if (context_info->is_done_rcv) {
+ struct mmc_command *cmd;
- context_info->is_done_rcv = false;
- cmd = host->areq->mrq->cmd;
+ context_info->is_done_rcv = false;
+ cmd = areq->mrq->cmd;
- if (!cmd->error || !cmd->retries ||
- mmc_card_removed(host->card)) {
- status = host->areq->err_check(host->card,
- host->areq);
- break; /* return status */
- } else {
- mmc_retune_recheck(host);
- pr_info("%s: req failed (CMD%u): %d, retrying...\n",
- mmc_hostname(host),
- cmd->opcode, cmd->error);
- cmd->retries--;
- cmd->error = 0;
- __mmc_start_request(host, host->areq->mrq);
- continue; /* wait for done/new event again */
- }
+ if (!cmd->error || !cmd->retries ||
+ mmc_card_removed(host->card)) {
+ status = areq->err_check(host->card,
+ areq);
+ } else {
+ mmc_retune_recheck(host);
+ pr_info("%s: req failed (CMD%u): %d, retrying...\n",
+ mmc_hostname(host),
+ cmd->opcode, cmd->error);
+ cmd->retries--;
+ cmd->error = 0;
+ __mmc_start_request(host, areq->mrq);
+ return; /* wait for done/new event again */
}
-
- return MMC_BLK_NEW_REQUEST;
}
mmc_retune_release(host);
@@ -644,10 +638,12 @@ static enum mmc_blk_status mmc_finalize_areq(struct mmc_host *host)
}
/* Successfully postprocess the old request at this point */
- mmc_post_req(host, host->areq->mrq, 0);
+ mmc_post_req(host, areq->mrq, 0);
- return status;
+ areq->finalization_status = status;
+ complete(&areq->complete);
}
+EXPORT_SYMBOL(mmc_finalize_areq);
/**
* mmc_start_areq - start an asynchronous request
@@ -677,18 +673,21 @@ struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
if (areq)
mmc_pre_req(host, areq->mrq);
- /* Finalize previous request */
- status = mmc_finalize_areq(host);
+ /* Finalize previous request, if there is one */
+ if (previous) {
+ wait_for_completion(&previous->complete);
+ status = previous->finalization_status;
+ } else {
+ status = MMC_BLK_SUCCESS;
+ }
if (ret_stat)
*ret_stat = status;
- /* The previous request is still going on... */
- if (status == MMC_BLK_NEW_REQUEST)
- return NULL;
-
/* Fine so far, start the new request! */
- if (status == MMC_BLK_SUCCESS && areq)
+ if (status == MMC_BLK_SUCCESS && areq) {
+ init_completion(&areq->complete);
start_err = __mmc_start_data_req(host, areq->mrq);
+ }
/* Cancel a prepared request if it was not started. */
if ((status != MMC_BLK_SUCCESS || start_err) && areq)
@@ -2996,7 +2995,6 @@ void mmc_init_context_info(struct mmc_host *host)
host->context_info.is_new_req = false;
host->context_info.is_done_rcv = false;
host->context_info.is_waiting_last_req = false;
- init_waitqueue_head(&host->context_info.wait);
}
static int __init mmc_init(void)
diff --git a/drivers/mmc/core/queue.c b/drivers/mmc/core/queue.c
index 5cb369c2664b..73250ed8f093 100644
--- a/drivers/mmc/core/queue.c
+++ b/drivers/mmc/core/queue.c
@@ -86,11 +86,6 @@ static int mmc_queue_thread(void *d)
set_current_state(TASK_RUNNING);
mmc_blk_issue_rq(mq, req);
cond_resched();
- if (mq->new_request) {
- mq->new_request = false;
- continue; /* fetch again */
- }
-
/*
* Current request becomes previous request
* and vice versa.
@@ -143,7 +138,6 @@ static void mmc_request_fn(struct request_queue *q)
if (cntx->is_waiting_last_req) {
cntx->is_new_req = true;
- wake_up_interruptible(&cntx->wait);
}
if (mq->asleep)
diff --git a/drivers/mmc/core/queue.h b/drivers/mmc/core/queue.h
index e298f100101b..39d8e710287e 100644
--- a/drivers/mmc/core/queue.h
+++ b/drivers/mmc/core/queue.h
@@ -40,7 +40,6 @@ struct mmc_queue {
struct mmc_card *card;
struct task_struct *thread;
struct semaphore thread_sem;
- bool new_request;
bool suspended;
bool asleep;
struct mmc_blk_data *blkdata;
diff --git a/include/linux/mmc/core.h b/include/linux/mmc/core.h
index a0c63ea28796..5db0fb722c37 100644
--- a/include/linux/mmc/core.h
+++ b/include/linux/mmc/core.h
@@ -9,6 +9,7 @@
#define LINUX_MMC_CORE_H
#include <linux/completion.h>
+#include <linux/kthread.h>
#include <linux/types.h>
struct mmc_data;
@@ -23,7 +24,6 @@ enum mmc_blk_status {
MMC_BLK_DATA_ERR,
MMC_BLK_ECC_ERR,
MMC_BLK_NOMEDIUM,
- MMC_BLK_NEW_REQUEST,
};
struct mmc_command {
@@ -158,6 +158,7 @@ struct mmc_request {
struct mmc_card;
struct mmc_async_req;
+void mmc_finalize_areq(struct kthread_work *work);
struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
struct mmc_async_req *areq,
enum mmc_blk_status *ret_stat);
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index b04f8cd51c82..c5f61f2f2310 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -14,6 +14,7 @@
#include <linux/device.h>
#include <linux/fault-inject.h>
#include <linux/kthread.h>
+#include <linux/completion.h>
#include <linux/mmc/core.h>
#include <linux/mmc/card.h>
@@ -169,6 +170,10 @@ struct mmc_async_req {
* Returns 0 if success otherwise non zero.
*/
enum mmc_blk_status (*err_check)(struct mmc_card *, struct mmc_async_req *);
+ struct kthread_work finalization_work;
+ enum mmc_blk_status finalization_status;
+ struct completion complete;
+ struct mmc_host *host;
};
/**
@@ -192,13 +197,11 @@ struct mmc_slot {
* @is_done_rcv wake up reason was done request
* @is_new_req wake up reason was new request
* @is_waiting_last_req mmc context waiting for single running request
- * @wait wait queue
*/
struct mmc_context_info {
bool is_done_rcv;
bool is_new_req;
bool is_waiting_last_req;
- wait_queue_head_t wait;
};
struct regulator;
--
2.9.3
^ permalink raw reply related
* [PATCH 05/16] mmc: core: add a kthread for completing requests
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
As we want to complete requests autonomously from feeding the
host with new requests, we create a worker thread to deal with
this specifically in response to the callback from a host driver.
This patch just adds the worker, later patches will make use of
it.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 6 ++++++
drivers/mmc/core/host.c | 2 +-
include/linux/mmc/host.h | 5 +++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 0972c649ea7a..663799240635 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -2794,6 +2794,10 @@ void mmc_start_host(struct mmc_host *host)
host->f_init = max(freqs[0], host->f_min);
host->rescan_disable = 0;
host->ios.power_mode = MMC_POWER_UNDEFINED;
+ /* Worker for completing requests */
+ host->req_done_worker_task = kthread_run(kthread_worker_fn,
+ &host->req_done_worker,
+ "mmc%d-reqdone", host->index);
if (!(host->caps2 & MMC_CAP2_NO_PRESCAN_POWERUP)) {
mmc_claim_host(host);
@@ -2818,6 +2822,8 @@ void mmc_stop_host(struct mmc_host *host)
host->rescan_disable = 1;
cancel_delayed_work_sync(&host->detect);
+ kthread_flush_worker(&host->req_done_worker);
+ kthread_stop(host->req_done_worker_task);
/* clear pm flags now and let card drivers set them as needed */
host->pm_flags = 0;
diff --git a/drivers/mmc/core/host.c b/drivers/mmc/core/host.c
index 98f25ffb4258..d33e2b260bf3 100644
--- a/drivers/mmc/core/host.c
+++ b/drivers/mmc/core/host.c
@@ -388,7 +388,7 @@ struct mmc_host *mmc_alloc_host(int extra, struct device *dev)
init_waitqueue_head(&host->wq);
INIT_DELAYED_WORK(&host->detect, mmc_rescan);
setup_timer(&host->retune_timer, mmc_retune_timer, (unsigned long)host);
-
+ kthread_init_worker(&host->req_done_worker);
/*
* By default, hosts do not support SGIO or large requests.
* They have to set these according to their abilities.
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index 97699d55b2ae..b04f8cd51c82 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -13,6 +13,7 @@
#include <linux/sched.h>
#include <linux/device.h>
#include <linux/fault-inject.h>
+#include <linux/kthread.h>
#include <linux/mmc/core.h>
#include <linux/mmc/card.h>
@@ -375,6 +376,10 @@ struct mmc_host {
struct mmc_async_req *areq; /* active async req */
struct mmc_context_info context_info; /* async synchronization info */
+ /* finalization work thread, handles finalizing requests */
+ struct kthread_worker req_done_worker;
+ struct task_struct *req_done_worker_task;
+
/* Ongoing data transfer that allows commands during transfer */
struct mmc_request *ongoing_mrq;
--
2.9.3
^ permalink raw reply related
* [PATCH 04/16] mmc: core: move the asynchronous post-processing
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
This moves the asynchronous post-processing of a request over
to the finalization function.
The patch has a slight semantic change:
Both places will be in the code path for if (host->areq) and
in the same sequence, but before this patch, the next request
was started before performing post-processing.
The effect is that whereas before, the post- and preprocessing
happened after starting the next request, now the preprocessing
will happen after the request is done and before the next has
started which would cut half of the pre/post optimizations out.
The idea is to later move the finalization to a worker started
by mmc_request_done() and introduce a completion where the code
now has a TODO comment so that we can push in a new request
as soon as the host has completed the previous one.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 8dbed198750f..0972c649ea7a 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -643,6 +643,9 @@ static enum mmc_blk_status mmc_finalize_areq(struct mmc_host *host)
mmc_start_bkops(host->card, true);
}
+ /* Successfully postprocess the old request at this point */
+ mmc_post_req(host, host->areq->mrq, 0);
+
return status;
}
@@ -687,10 +690,6 @@ struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
if (status == MMC_BLK_SUCCESS && areq)
start_err = __mmc_start_data_req(host, areq->mrq);
- /* Postprocess the old request at this point */
- if (host->areq)
- mmc_post_req(host, host->areq->mrq, 0);
-
/* Cancel a prepared request if it was not started. */
if ((status != MMC_BLK_SUCCESS || start_err) && areq)
mmc_post_req(host, areq->mrq, -EINVAL);
--
2.9.3
^ permalink raw reply related
* [PATCH 03/16] mmc: core: refactor mmc_request_done()
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
We have this construction:
if (a && b && !c)
finalize;
else
block;
finalize;
Which is equivalent by boolean logic to:
if (!a || !b || c)
block;
finalize;
Which is simpler code.
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 27 ++++++++++++++++-----------
1 file changed, 16 insertions(+), 11 deletions(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index b2e7a6dfcbf0..8dbed198750f 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -172,14 +172,16 @@ void mmc_request_done(struct mmc_host *host, struct mmc_request *mrq)
trace_mmc_request_done(host, mrq);
- if (err && cmd->retries && !mmc_card_removed(host->card)) {
- /*
- * Request starter must handle retries - see
- * mmc_wait_for_req_done().
- */
- if (mrq->done)
- mrq->done(mrq);
- } else {
+ /*
+ * We list various conditions for the command to be considered
+ * properly done:
+ *
+ * - There was no error, OK fine then
+ * - We are not doing some kind of retry
+ * - The card was removed (...so just complete everything no matter
+ * if there are errors or retries)
+ */
+ if (!err || !cmd->retries || mmc_card_removed(host->card)) {
mmc_should_fail_request(host, mrq);
if (!host->ongoing_mrq)
@@ -211,10 +213,13 @@ void mmc_request_done(struct mmc_host *host, struct mmc_request *mrq)
mrq->stop->resp[0], mrq->stop->resp[1],
mrq->stop->resp[2], mrq->stop->resp[3]);
}
-
- if (mrq->done)
- mrq->done(mrq);
}
+ /*
+ * Request starter must handle retries - see
+ * mmc_wait_for_req_done().
+ */
+ if (mrq->done)
+ mrq->done(mrq);
}
EXPORT_SYMBOL(mmc_request_done);
--
2.9.3
^ permalink raw reply related
* [PATCH 02/16] mmc: core: refactor asynchronous request finalization
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
mmc_wait_for_data_req_done() is called in exactly one place,
and having it spread out is making things hard to oversee.
Factor this function into mmc_finalize_areq().
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 86 +++++++++++++++++++------------------------------
1 file changed, 33 insertions(+), 53 deletions(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 53065d1cebf7..b2e7a6dfcbf0 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -485,56 +485,6 @@ static int __mmc_start_req(struct mmc_host *host, struct mmc_request *mrq)
return err;
}
-/*
- * mmc_wait_for_data_req_done() - wait for request completed
- * @host: MMC host to prepare the command.
- * @mrq: MMC request to wait for
- *
- * Blocks MMC context till host controller will ack end of data request
- * execution or new request notification arrives from the block layer.
- * Handles command retries.
- *
- * Returns enum mmc_blk_status after checking errors.
- */
-static enum mmc_blk_status mmc_wait_for_data_req_done(struct mmc_host *host,
- struct mmc_request *mrq)
-{
- struct mmc_command *cmd;
- struct mmc_context_info *context_info = &host->context_info;
- enum mmc_blk_status status;
-
- while (1) {
- wait_event_interruptible(context_info->wait,
- (context_info->is_done_rcv ||
- context_info->is_new_req));
-
- if (context_info->is_done_rcv) {
- context_info->is_done_rcv = false;
- cmd = mrq->cmd;
-
- if (!cmd->error || !cmd->retries ||
- mmc_card_removed(host->card)) {
- status = host->areq->err_check(host->card,
- host->areq);
- break; /* return status */
- } else {
- mmc_retune_recheck(host);
- pr_info("%s: req failed (CMD%u): %d, retrying...\n",
- mmc_hostname(host),
- cmd->opcode, cmd->error);
- cmd->retries--;
- cmd->error = 0;
- __mmc_start_request(host, mrq);
- continue; /* wait for done/new event again */
- }
- }
-
- return MMC_BLK_NEW_REQUEST;
- }
- mmc_retune_release(host);
- return status;
-}
-
void mmc_wait_for_req_done(struct mmc_host *host, struct mmc_request *mrq)
{
struct mmc_command *cmd;
@@ -639,14 +589,44 @@ static void mmc_post_req(struct mmc_host *host, struct mmc_request *mrq,
*/
static enum mmc_blk_status mmc_finalize_areq(struct mmc_host *host)
{
+ struct mmc_context_info *context_info = &host->context_info;
enum mmc_blk_status status;
if (!host->areq)
return MMC_BLK_SUCCESS;
- status = mmc_wait_for_data_req_done(host, host->areq->mrq);
- if (status == MMC_BLK_NEW_REQUEST)
- return status;
+ while (1) {
+ wait_event_interruptible(context_info->wait,
+ (context_info->is_done_rcv ||
+ context_info->is_new_req));
+
+ if (context_info->is_done_rcv) {
+ struct mmc_command *cmd;
+
+ context_info->is_done_rcv = false;
+ cmd = host->areq->mrq->cmd;
+
+ if (!cmd->error || !cmd->retries ||
+ mmc_card_removed(host->card)) {
+ status = host->areq->err_check(host->card,
+ host->areq);
+ break; /* return status */
+ } else {
+ mmc_retune_recheck(host);
+ pr_info("%s: req failed (CMD%u): %d, retrying...\n",
+ mmc_hostname(host),
+ cmd->opcode, cmd->error);
+ cmd->retries--;
+ cmd->error = 0;
+ __mmc_start_request(host, host->areq->mrq);
+ continue; /* wait for done/new event again */
+ }
+ }
+
+ return MMC_BLK_NEW_REQUEST;
+ }
+
+ mmc_retune_release(host);
/*
* Check BKOPS urgency for each R1 response
--
2.9.3
^ permalink raw reply related
* [PATCH 01/16] mmc: core: move some code in mmc_start_areq()
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
In-Reply-To: <20170209153403.9730-1-linus.walleij@linaro.org>
"previous" is a better name for the variable storing the previous
asynchronous request, better than the opaque name "data" atleast.
We see that we assign the return status to the returned variable
on all code paths, so we might as well just do that immediately
after calling mmc_finalize_areq().
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
drivers/mmc/core/core.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 41b4cd01fccc..53065d1cebf7 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -683,7 +683,7 @@ struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
{
enum mmc_blk_status status;
int start_err = 0;
- struct mmc_async_req *data = host->areq;
+ struct mmc_async_req *previous = host->areq;
/* Prepare a new request */
if (areq)
@@ -691,13 +691,12 @@ struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
/* Finalize previous request */
status = mmc_finalize_areq(host);
+ if (ret_stat)
+ *ret_stat = status;
/* The previous request is still going on... */
- if (status == MMC_BLK_NEW_REQUEST) {
- if (ret_stat)
- *ret_stat = status;
+ if (status == MMC_BLK_NEW_REQUEST)
return NULL;
- }
/* Fine so far, start the new request! */
if (status == MMC_BLK_SUCCESS && areq)
@@ -716,9 +715,7 @@ struct mmc_async_req *mmc_start_areq(struct mmc_host *host,
else
host->areq = areq;
- if (ret_stat)
- *ret_stat = status;
- return data;
+ return previous;
}
EXPORT_SYMBOL(mmc_start_areq);
--
2.9.3
^ permalink raw reply related
* [PATCH 00/16] multiqueue for MMC/SD third try
From: Linus Walleij @ 2017-02-09 15:33 UTC (permalink / raw)
To: linux-mmc, Ulf Hansson, Adrian Hunter, Paolo Valente
Cc: Chunyan Zhang, Baolin Wang, linux-block, Jens Axboe,
Christoph Hellwig, Arnd Bergmann, Linus Walleij
The following is the latest attempt at a rewriting the MMC/SD
stack to cope with multiqueueing.
If you just want to grab a branch and test the patches with
your hardware, I put a git branch with this series here:
https://git.kernel.org/cgit/linux/kernel/git/linusw/linux-stericsson.git/log/?h=mmc-mq-next-2017-02-09
It's based on Ulf's v4.10-rc3-based tree, so quick reminder:
git checkout -b test v4.10-rc3
git pull git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-stericsson.git mmc-mq-next-2017-02-09
Should get you a testable "test" branch.
These patches are clearly v4.12 material. They get increasinly
controversial and needing review the further into the series
you go. The last patch for multiqueue is marked RFC for a
reason.
Every time I do this it seems to be an extensive rewrite of the
whole world. Anyways this is based on the other ~16 patches that
were already merged for the upcoming v4.11.
The rationale for this approach was Arnd's suggestion to try to
switch the MMC/SD stack around so as to complete requests as
quickly as possible from the device driver so that new requests
can be issued. We are doing this now: the polling loop that was
pulling NULL out of the request queue and driving the pipeline
with a loop is gone.
We are not issueing new requests from interrupt context: I still
have to post a work for it. I don't know if that is possible.
There is the retune and background operations that need to be
checked after every command and yeah, it needs to happen in
blocking context as far as I know.
We have parallelism in pre/post hooks also with multiqueue.
All asynchronous optimization that was there for the old block layer
is now also there for multiqueue. There is even a new interesting
optimization that make bounce buffers be bounced asynchronously
with this change.
We still use the trick to set the queue depth to 2 to get two
parallel requests pushed down to the host.
Adrian: I know I made quite extensive violence on your queueue
handling reusing it in a way that is probably totally counter to
your command queueing patch series. I'm sorry. I guess you can
see where it is going if you follow the series. I also killed the
host context, right off, after reducing the synchronization needs
to zero. I hope you will be interested in the result though!
Does this perform? The numbers follow. I will discuss my
conclusions after the figures. All the tests are done on a cold
booted Ux500 system.
Before this patch series, based on my earlier cleanups
and refactorings on Ulf's next branch ending with
"mmc: core: start to break apart mmc_start_areq()":
time dd if=/dev/mmcblk0 of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.0GB) copied, 45.126404 seconds, 22.7MB/s
real 0m 45.13s
user 0m 0.02s
sys 0m 7.60s
mount /dev/mmcblk0p1 /mnt/
cd /mnt/
time find . > /dev/null
real 0m 3.61s
user 0m 0.30s
sys 0m 1.56s
Command line used: iozone -az -i0 -i1 -i2 -s 20m -I -f /mnt/foo.test
Output is in kBytes/sec
random random
kB reclen write rewrite read reread read write
20480 4 2046 2114 5981 6008 5971 40
20480 8 4825 4622 9104 9118 9070 81
20480 16 5767 5929 12250 12253 12209 166
20480 32 6242 6303 14920 14917 14879 337
20480 64 6598 5907 16758 16760 16739 695
20480 128 6807 6837 17863 17869 17788 1387
20480 256 6922 6925 18497 18490 18482 3076
20480 512 7273 7313 18636 18407 18829 7344
20480 1024 7339 7332 17695 18785 18472 7441
20480 2048 7419 7471 19166 18812 18797 7474
20480 4096 7598 7714 21006 20975 21180 7708
20480 8192 7632 7830 22328 22315 22201 7828
20480 16384 7412 7903 23070 23046 22849 7913
With "mmc: core: move the asynchronous post-processing"
time dd if=/dev/mmcblk0 of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.0GB) copied, 52.166992 seconds, 19.6MB/s
real 0m 52.17s
user 0m 0.01s
sys 0m 6.96s
mount /dev/mmcblk0p1 /mnt/
cd /mnt/
time find . > /dev/null
real 0m 3.88s
user 0m 0.35s
sys 0m 1.60s
Command line used: iozone -az -i0 -i1 -i2 -s 20m -I -f /mnt/foo.test
Output is in kBytes/sec
random random
kB reclen write rewrite read reread read write
20480 4 2072 2200 6030 6066 6005 40
20480 8 4847 5106 9174 9178 9123 81
20480 16 5791 5934 12301 12299 12260 166
20480 32 6252 6311 14906 14943 14919 337
20480 64 6607 6699 16776 16787 16756 690
20480 128 6836 6880 17868 17880 17873 1419
20480 256 6967 6955 18442 17112 18490 3072
20480 512 7320 7359 18818 18738 18477 7310
20480 1024 7350 7426 18297 18551 18357 7429
20480 2048 7439 7476 18035 19111 17670 7486
20480 4096 7655 7728 19688 19557 19758 7738
20480 8192 7640 7848 20675 20718 20787 7823
20480 16384 7489 7934 21225 21186 21555 7943
With "mmc: queue: issue requests in massive parallel"
time dd if=/dev/mmcblk0 of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.0GB) copied, 49.308167 seconds, 20.8MB/s
real 0m 49.31s
user 0m 0.00s
sys 0m 7.11s
mount /dev/mmcblk0p1 /mnt/
cd /mnt/
time find . > /dev/null
real 0m 3.70s
user 0m 0.19s
sys 0m 1.73s
Command line used: iozone -az -i0 -i1 -i2 -s 20m -I -f /mnt/foo.test
Output is in kBytes/sec
random random
kB reclen write rewrite read reread read write
20480 4 1709 1761 5963 5321 5909 40
20480 8 4736 5059 9089 9092 9055 81
20480 16 5772 5928 12217 12229 12184 165
20480 32 6237 6279 14898 14899 14875 336
20480 64 6599 6663 16759 16760 16741 683
20480 128 6804 6790 17869 17869 17864 1393
20480 256 6863 6883 18485 18488 18501 3105
20480 512 7223 7249 18807 18810 18812 7259
20480 1024 7311 7321 18684 18467 18201 7328
20480 2048 7405 7457 18560 18044 18343 7451
20480 4096 7596 7684 20742 21154 21153 7711
20480 8192 7593 7802 21743 21721 22090 7804
20480 16384 7399 7873 21539 22670 22828 7876
With "RFC: mmc: switch MMC/SD to use blk-mq multiqueueing v3"
time dd if=/dev/mmcblk0 of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.0GB) copied, 46.240479 seconds, 22.1MB/s
real 0m 46.25s
user 0m 0.03s
sys 0m 6.42s
mount /dev/mmcblk0p1 /mnt/
cd /mnt/
time find . > /dev/null
real 0m 4.13s
user 0m 0.40s
sys 0m 1.64s
Command line used: iozone -az -i0 -i1 -i2 -s 20m -I -f /mnt/foo.test
Output is in kBytes/sec
random random
kB reclen write rewrite read reread read write
20480 4 1786 1806 6055 6061 5360 40
20480 8 4849 5088 9167 9175 9120 81
20480 16 5807 5975 12273 12256 12240 166
20480 32 6275 6317 14929 14931 14905 338
20480 64 6629 6708 16755 16783 16758 688
20480 128 6856 6884 17890 17804 17873 1420
20480 256 6927 6946 18104 17826 18389 3038
20480 512 7296 7280 18720 18752 18819 7284
20480 1024 7286 7415 18583 18598 18516 7403
20480 2048 7435 7470 18378 18268 18682 7471
20480 4096 7670 7786 21364 21275 20761 7766
20480 8192 7637 7868 22193 21994 22100 7850
20480 16384 7416 7921 23050 23051 22726 7955
The iozone results seem a bit consistent and all values seem to
be noisy and not say much. I don't know why really, maybe the test
is simply not relevant, the tests don't seem to be significantly
affected by any of the patches, so let's focus on the dd and find
tests.
You can see there are three steps:
- I do some necessary refactoring and need to move postprocessing
to after the requests have been completed. This clearly, as you
can see, introduce a performance regression in the dd test with
the patch:
"mmc: core: move the asynchronous post-processing"
It seems the random seek with find isn't much affected.
- I continue the refactoring and get to the point of issueing
requests immediately after every successful transfer, and the
dd performance is restored with patch
"mmc: queue: issue requests in massive parallel"
- Then I add multiqueue on top of the cake. So before the change
we have the nice performance we want so we can study the effect
of just introducing multiqueueing in the last patch
"RFC: mmc: switch MMC/SD to use blk-mq multiqueueing v3"
What immediately jumps out at you is that linear read/writes
perform just as nicely or actually better with MQ than with the
old block layer.
What is amazing is that just a little randomness, such as the
find . > /dev/null immediately seems to visibly regress with MQ.
My best guess is that it is caused by the absence of the block
scheduler.
I do not know if my conclusions are right or anything, please
scrutinize.
Linus Walleij (16):
mmc: core: move some code in mmc_start_areq()
mmc: core: refactor asynchronous request finalization
mmc: core: refactor mmc_request_done()
mmc: core: move the asynchronous post-processing
mmc: core: add a kthread for completing requests
mmc: core: replace waitqueue with worker
mmc: core: do away with is_done_rcv
mmc: core: do away with is_new_req
mmc: core: kill off the context info
mmc: queue: simplify queue logic
mmc: block: shuffle retry and error handling
mmc: queue: stop flushing the pipeline with NULL
mmc: queue: issue struct mmc_queue_req items
mmc: queue: get/put struct mmc_queue_req
mmc: queue: issue requests in massive parallel
RFC: mmc: switch MMC/SD to use blk-mq multiqueueing v3
drivers/mmc/core/block.c | 426 +++++++++++++++++++++++------------------------
drivers/mmc/core/block.h | 10 +-
drivers/mmc/core/bus.c | 1 -
drivers/mmc/core/core.c | 228 ++++++++++++-------------
drivers/mmc/core/core.h | 2 -
drivers/mmc/core/host.c | 2 +-
drivers/mmc/core/queue.c | 337 ++++++++++++++-----------------------
drivers/mmc/core/queue.h | 21 ++-
include/linux/mmc/core.h | 9 +-
include/linux/mmc/host.h | 24 +--
10 files changed, 481 insertions(+), 579 deletions(-)
--
2.9.3
^ permalink raw reply
* Re: [PATCH] block: sed-opal: reduce stack size of ioctl handler
From: Arnd Bergmann @ 2017-02-09 14:57 UTC (permalink / raw)
To: Scott Bauer
Cc: Jens Axboe, Rafael Antognolli, Michael Ellerman,
Linux Kernel Mailing List, linux-nvme, linux-block,
Paul Mackerras, Benjamin Herrenschmidt, linuxppc-dev, hch,
Jonathan Derrick
In-Reply-To: <20170208221221.GA10983@sbauer-Z170X-UD5>
On Wed, Feb 8, 2017 at 11:12 PM, Scott Bauer <scott.bauer@intel.com> wrote:
> On Wed, Feb 08, 2017 at 02:58:28PM -0700, Scott Bauer wrote:
>> Thank you for the report. We want to keep the function calls agnostic to userland.
>> In the future we will have in-kernel callers and I don't want to have to do any
>> get_fs(KERNEL_DS) wizardry.
>>
>> Instead I think we can use a union to lessen the stack burden. I tested this patch just now
>> with config_ksasan and was able to build.
>
> Nack on this patch, it only really masks the issue. Keith pointed out we have a call chain
> up to this ioctl then deeper down into nvme then the block layer. If we use 25% of the stack
> just for this function it's still too dangerous and we'll run into corruption later on and not
> remember this fix. I'll come up with another solution.
I think there are two issues that cause the stack frame to explode
with KASAN, and
your patch addresses one but not the other:
1. checks for variables being accessed after they go out of scope.
This is solved by the
union at the start of the function, as they never go out of scope now.
2. checks for array overflows when passing a local variable by
reference to another
function that is not inlined.
To solve the second problem while keeping the in-kernel interface, the
approach that
David suggesteed would work, or you could have a wrapper around each function to
do the copy_from_user in a more obvious but verbose way as I had in my version.
With David's approach, you could actually replace the switch() with a
lookup table
as well.
Arnd
^ permalink raw reply
* Re: [PATCH 22/24] ubifs: Convert to separately allocated bdi
From: Richard Weinberger @ 2017-02-09 14:56 UTC (permalink / raw)
To: Jan Kara
Cc: linux-fsdevel, Christoph Hellwig, linux-block, Artem Bityutskiy,
Adrian Hunter, linux-mtd
In-Reply-To: <20170209121728.GE16825@quack2.suse.cz>
Am 09.02.2017 um 13:17 schrieb Jan Kara:
>> So ->capabilities is now zero by default since you use __GFP_ZERO in
>> bdi_alloc().
>> At least for UBIFS I'll add a comment on this, otherwise it is not so
>> clear that UBIFS wants a BDI with no capabilities and how it achieves that.
>
> OK, I've modified the comment to:
>
> * Read-ahead will be disabled because @sb->s_bdi->ra_pages is 0. Also
> * @sb->s_bdi->capabilities are initialized to 0 so there won't be any
> * writeback happening.
> */
Nice!
Thanks,
//richard
^ permalink raw reply
* Re: [PATCH 0/10] block: Fix block device shutdown related races
From: Thiago Jung Bauermann @ 2017-02-09 14:52 UTC (permalink / raw)
To: Jan Kara
Cc: Jens Axboe, linux-block, Christoph Hellwig, Tejun Heo,
Dan Williams, NeilBrown
In-Reply-To: <20170209124433.2626-1-jack@suse.cz>
Hello Jan,
Am Donnerstag, 9. Februar 2017, 13:44:23 BRST schrieb Jan Kara:
> People, please have a look at patches. The are mostly simple however the
> interactions are rather complex so I may have missed something. Also I'm
> happy for any additional testing these patches can get - I've stressed th=
em
> with Omar's script, tested memcg writeback, tested static (not udev manag=
ed)
> device inodes.
Thank you for these fixes. I will have them tested and report back how it=20
goes.
Can you tell which branch I should apply them on? I tried a number of branc=
hes=20
in linux-block (and applied the bdi lifetime v3 patches if the branch didn'=
t=20
already had them) but this series either didn't apply or the build failed=20
with:
/home/bauermann/trabalho/src/linux-2.6.git/fs/block_dev.c: In function=20
=E2=80=98bd_acquire=E2=80=99:
/home/bauermann/trabalho/src/linux-2.6.git/fs/block_dev.c:1063:13: error:=20
passing argument 1 of =E2=80=98bd_forget=E2=80=99 from incompatible pointer=
type [-
Werror=3Dincompatible-pointer-types]
bd_forget(bdev);
^
In file included from /home/bauermann/trabalho/src/linux-2.6.git/include/
linux/device_cgroup.h:1:0,
from /home/bauermann/trabalho/src/linux-2.6.git/fs/
block_dev.c:14:
/home/bauermann/trabalho/src/linux-2.6.git/include/linux/fs.h:2351:13: note=
:=20
expected =E2=80=98struct inode *=E2=80=99 but argument is of type =E2=80=98=
struct block_device *=E2=80=99
extern void bd_forget(struct inode *inode);
^
cc1: some warnings being treated as errors
=2D-=20
Thiago Jung Bauermann
IBM Linux Technology Center
^ permalink raw reply
* Re: [PATCH 05/24] fs: Get proper reference for s_bdi
From: Boaz Harrosh @ 2017-02-09 14:36 UTC (permalink / raw)
To: Jan Kara, linux-fsdevel; +Cc: Christoph Hellwig, linux-block
In-Reply-To: <20170202173422.3240-6-jack@suse.cz>
On 02/02/2017 07:34 PM, Jan Kara wrote:
> So far we just relied on block device to hold a bdi reference for us
> while the filesystem is mounted. While that works perfectly fine, it is
> a bit awkward that we have a pointer to a refcounted structure in the
> superblock without proper reference. So make s_bdi hold a proper
> reference to block device's BDI. No filesystem using mount_bdev()
> actually changes s_bdi so this is safe and will make bdev filesystems
> work the same way as filesystems needing to set up their private bdi.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
> fs/super.c | 7 ++-----
> 1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/fs/super.c b/fs/super.c
> index 31dc4c6450ef..dfb95ccd4351 100644
> --- a/fs/super.c
> +++ b/fs/super.c
> @@ -1047,12 +1047,9 @@ static int set_bdev_super(struct super_block *s, void *data)
> {
> s->s_bdev = data;
> s->s_dev = s->s_bdev->bd_dev;
> + s->s_bdi = bdi_get(s->s_bdev->bd_bdi);
> + s->s_iflags |= SB_I_DYNBDI;
>
> - /*
> - * We set the bdi here to the queue backing, file systems can
> - * overwrite this in ->fill_super()
> - */
Question: So I have an FS that uses mount_bdev but than goes and overrides sb->s_bdev
in ->fill_super() anyway. This is because of two reasons. One because I have many
more devices. (like btrfs I'm moulti-dev) but I like to use mount_bdev because of the
somewhat delicate handling of automatic bind-mounts.
For me it is a bigger hack to get the ref-counting and bind-mounts locking correctly
then to bdi_put and say the new super_setup_bdi(sb) in fill_super. Would you expect
problems?
Thanks for any help
Boaz
> - s->s_bdi = bdev_get_queue(s->s_bdev)->backing_dev_info;
> return 0;
> }
>
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox