[RFC PATCH 00/11] [FS, MM, block, MMC]: eMMC High Priority Interrupt Feature

[RFC PATCH 00/11] [FS, MM, block, MMC]: eMMC High Priority Interrupt Feature

[Venkatraman S]

(This is still work in progress. Posting for early review and suggestions)

Standard eMMC (Embedded MultiMedia Card) specification expects to execute
one request at a time. If some requests are more important than others, they
can't be aborted while the flash procedure is in progress.

New versions of the eMMC standard (4.41 and above) specfies a feature 
called High Priority Interrupt (HPI). This enables an ongoing transaction
to be aborted using a special command (HPI command) so that the card is ready
to receive new commands immediately. Then the new request can be submitted
to the card, and optionally the interrupted command can be resumed again.

Some restrictions exist on when and how the command can be used. For example,
only write and write-like commands (ERASE) can be preempted, and the urgent
request must be a read.

In order to support this in software,
a) At the top level, some policy decisions have to be made on what is
worth preempting for.
	This implementation uses the demand paging requests and swap
read requests as potential reads worth preempting an ongoing long write.
	This is expected to provide improved responsiveness for smarphones
with multitasking capabilities - example would be launch a email application
while a video capture session (which causes long writes) is ongoing.
b) At the block handler, the higher priority request should be queued
  ahead of the pending requests in the elevator
c) At the MMC block and core level, transactions have to executed to 
enforce the rules of the MMC spec and make a reasonable tradeoff if the
ongoing command is really worth preempting. (For example, is it too close
to completing already ?).
	The current implementation uses a fixed time logic. If 10ms has
already elapsed since the first request was submitted, then a new high
priority request would not cause a HPI, as it is expected that the first
request would finish soon. Further work is needed to dynamically tune
this value (maybe through sysfs) or automatically determine based on
average write times of previous requests.
d) At the lowest level (MMC host controllers), support interface to 
provide a transition path for ongoing transactions to be aborted and the
controller to be ready to receive next command.

	More information about this feature can be found at
Jedec Specification:-
	http://www.jedec.org/standards-documents/docs/jesd84-a441
Presentation on eMMC4.41 features:-
	http://www.jedec.org/sites/.default/files/Victor_Tsai.pdf

Still TODO:-
	* Provide sysfs entry for tuning the threshold time for preemption
	* Implement resuming of the preempted request by checking
	  CORRECTLY_PROGRAMMED_SECTORS_NUM field.
	* Provide implementation of abort_req for omap_hsmmc_driver
		(This is almost done, needs a few tests)

Acknowledgements:-	
	In no particular order, thanks to Arnd Bergmann and  Saugata Das
from Linaro, Ilan Smith and Alex Lemberg from Sandisk, Luca Porzio from Micron
devices, Yejin Moon, Jae Hoon Chung from Samsung and others.

----
	This patch series depends on a few other related cleanups
in MMC driver. All the patches and the dependent series can be
pulled from 
	git://github.com/svenkatr/linux.git my/mmc/3.4/foreground-hpiv1

Ilan Smith (3):
  fs: Add demand paging markers to filesystem
  mm: Add page swapping markers to memory management
  block: Expedite DMPG and SWAPIN requests ahead of the queue

Venkatraman S (8):
  block: Add queue attributes to manage dpmg and swapin requests
  mmc: Add BKOPS field offsets
  mmc: core: Helper function for finding preemptible command
  mmc: core: add preemptibility tracking fields to mmc command
  mmc: core: Add MMC abort interface
  mmc: block: Detect HPI support in card and host controller
  mmc: core: Utility function for mmc preemption sequence
  mmc: block: Implement HPI invocation and handling for foreground
    requests

 block/blk-core.c          |   18 ++++++
 block/blk-sysfs.c         |   16 ++++++
 drivers/mmc/card/block.c  |  138 ++++++++++++++++++++++++++++++++++++++++-----
 drivers/mmc/card/queue.h  |    1 +
 drivers/mmc/core/core.c   |   28 +++++++++
 fs/mpage.c                |    2 +
 include/linux/bio.h       |    8 +++
 include/linux/blk_types.h |    4 ++
 include/linux/blkdev.h    |    8 +++
 include/linux/mmc/core.h  |   19 +++++++
 include/linux/mmc/host.h  |    1 +
 include/linux/mmc/mmc.h   |    2 +
 mm/page_io.c              |    3 +-
 13 files changed, 233 insertions(+), 15 deletions(-)

-- 
1.7.10.rc2

[RFC PATCH 01/11] fs: Add demand paging markers to filesystem

[Venkatraman S]

From: Ilan Smith <ilan.smith@sandisk.com>

Add attribute to identify demand paging requests.
Mark readpages with demand paging attribute.

Signed-off-by: Ilan Smith <ilan.smith@sandisk.com>
Signed-off-by: Alex Lemberg <alex.lemberg@sandisk.com>
Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 fs/mpage.c                |    2 ++
 include/linux/bio.h       |    7 +++++++
 include/linux/blk_types.h |    2 ++
 3 files changed, 11 insertions(+)

diff --git a/fs/mpage.c b/fs/mpage.c
index 0face1c..8b144f5 100644
--- a/fs/mpage.c
+++ b/fs/mpage.c
@@ -386,6 +386,8 @@ mpage_readpages(struct address_space *mapping, struct list_head *pages,
 					&last_block_in_bio, &map_bh,
 					&first_logical_block,
 					get_block);
+			if (bio)
+				bio->bi_rw |= REQ_RW_DMPG;
 		}
 		page_cache_release(page);
 	}
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 4d94eb8..264e0ef 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -57,6 +57,13 @@
 	(bio)->bi_rw |= ((unsigned long) (prio) << BIO_PRIO_SHIFT);	\
 } while (0)
 
+static inline bool bio_rw_flagged(struct bio *bio, unsigned long flag)
+{
+	return ((bio->bi_rw & flag)  != 0);
+}
+
+#define bio_dmpg(bio)	bio_rw_flagged(bio, REQ_RW_DMPG)
+
 /*
  * various member access, note that bio_data should of course not be used
  * on highmem page vectors
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 4053cbd..87feb80 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -150,6 +150,7 @@ enum rq_flag_bits {
 	__REQ_FLUSH_SEQ,	/* request for flush sequence */
 	__REQ_IO_STAT,		/* account I/O stat */
 	__REQ_MIXED_MERGE,	/* merge of different types, fail separately */
+	__REQ_RW_DMPG,
 	__REQ_NR_BITS,		/* stops here */
 };
 
@@ -191,5 +192,6 @@ enum rq_flag_bits {
 #define REQ_IO_STAT		(1 << __REQ_IO_STAT)
 #define REQ_MIXED_MERGE		(1 << __REQ_MIXED_MERGE)
 #define REQ_SECURE		(1 << __REQ_SECURE)
+#define REQ_RW_DMPG		(1 << __REQ_RW_DMPG)
 
 #endif /* __LINUX_BLK_TYPES_H */
-- 
1.7.10.rc2

[RFC PATCH 02/11] mm: Add page swapping markers to memory management

[Venkatraman S]

From: Ilan Smith <ilan.smith@sandisk.com>

Add attribute to identify swap page read requests.
Mark swap read requests with swapin attributes.

Signed-off-by: Ilan Smith <ilan.smith@sandisk.com>
Signed-off-by: Alex Lemberg <alex.lemberg@sandisk.com>
Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 include/linux/bio.h       |    1 +
 include/linux/blk_types.h |    2 ++
 mm/page_io.c              |    3 ++-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/linux/bio.h b/include/linux/bio.h
index 264e0ef..8494b2f 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -63,6 +63,7 @@ static inline bool bio_rw_flagged(struct bio *bio, unsigned long flag)
 }
 
 #define bio_dmpg(bio)	bio_rw_flagged(bio, REQ_RW_DMPG)
+#define bio_swapin(bio)	bio_rw_flagged(bio, REQ_RW_SWAPIN)
 
 /*
  * various member access, note that bio_data should of course not be used
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 87feb80..df2b9ea 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -151,6 +151,7 @@ enum rq_flag_bits {
 	__REQ_IO_STAT,		/* account I/O stat */
 	__REQ_MIXED_MERGE,	/* merge of different types, fail separately */
 	__REQ_RW_DMPG,
+	__REQ_RW_SWAPIN,
 	__REQ_NR_BITS,		/* stops here */
 };
 
@@ -193,5 +194,6 @@ enum rq_flag_bits {
 #define REQ_MIXED_MERGE		(1 << __REQ_MIXED_MERGE)
 #define REQ_SECURE		(1 << __REQ_SECURE)
 #define REQ_RW_DMPG		(1 << __REQ_RW_DMPG)
+#define REQ_RW_SWAPIN		(1 << __REQ_RW_SWAPIN)
 
 #endif /* __LINUX_BLK_TYPES_H */
diff --git a/mm/page_io.c b/mm/page_io.c
index dc76b4d..a148bea 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -128,8 +128,9 @@ int swap_readpage(struct page *page)
 		ret = -ENOMEM;
 		goto out;
 	}
+	bio->bi_rw |= REQ_RW_SWAPIN;
 	count_vm_event(PSWPIN);
-	submit_bio(READ, bio);
+	submit_bio(READ | REQ_RW_SWAPIN, bio);
 out:
 	return ret;
 }
-- 
1.7.10.rc2

[RFC PATCH 03/11] block: Add queue attributes to manage dpmg and swapin requests

[Venkatraman S]

Add block queue properties to identify and manage demand paging
and swap read requests differently. sysfs entries are included
to enable or disable this feature at runtime.

Signed-off-by: Venkatraman S <svenkatr@ti.com>
Signed-off-by: Ilan Smith <ilan.smith@sandisk.com>
Signed-off-by: Alex Lemberg <alex.lemberg@sandisk.com>
---
 block/blk-sysfs.c      |   16 ++++++++++++++++
 include/linux/blkdev.h |    8 ++++++++
 2 files changed, 24 insertions(+)

diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index cf15001..764de9f 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -213,6 +213,8 @@ queue_store_##name(struct request_queue *q, const char *page, size_t count) \
 }
 
 QUEUE_SYSFS_BIT_FNS(nonrot, NONROT, 1);
+QUEUE_SYSFS_BIT_FNS(expedite_dmpg, EXP_DMPG, 0);
+QUEUE_SYSFS_BIT_FNS(expedite_swapin, EXP_SWAPIN, 0);
 QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0);
 QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0);
 #undef QUEUE_SYSFS_BIT_FNS
@@ -387,6 +389,18 @@ static struct queue_sysfs_entry queue_random_entry = {
 	.store = queue_store_random,
 };
 
+static struct queue_sysfs_entry queue_dmpg_entry = {
+	.attr = {.name = "expedite_demandpaging", .mode = S_IRUGO | S_IWUSR },
+	.show = queue_show_expedite_dmpg,
+	.store = queue_store_expedite_dmpg,
+};
+
+static struct queue_sysfs_entry queue_swapin_entry = {
+	.attr = {.name = "expedite_swapping", .mode = S_IRUGO | S_IWUSR },
+	.show = queue_show_expedite_swapin,
+	.store = queue_store_expedite_swapin,
+};
+
 static struct attribute *default_attrs[] = {
 	&queue_requests_entry.attr,
 	&queue_ra_entry.attr,
@@ -409,6 +423,8 @@ static struct attribute *default_attrs[] = {
 	&queue_rq_affinity_entry.attr,
 	&queue_iostats_entry.attr,
 	&queue_random_entry.attr,
+	&queue_dmpg_entry.attr,
+	&queue_swapin_entry.attr,
 	NULL,
 };
 
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 606cf33..fd620b9 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -420,6 +420,8 @@ struct request_queue {
 #define QUEUE_FLAG_ADD_RANDOM  16	/* Contributes to random pool */
 #define QUEUE_FLAG_SECDISCARD  17	/* supports SECDISCARD */
 #define QUEUE_FLAG_SAME_FORCE  18	/* force complete on same CPU */
+#define QUEUE_FLAG_EXP_DMPG    19	/* Expedite Demand paging requests */
+#define QUEUE_FLAG_EXP_SWAPIN  20	/* Expedit page swapping */
 
 #define QUEUE_FLAG_DEFAULT	((1 << QUEUE_FLAG_IO_STAT) |		\
 				 (1 << QUEUE_FLAG_STACKABLE)	|	\
@@ -506,6 +508,12 @@ static inline void queue_flag_clear(unsigned int flag, struct request_queue *q)
 #define blk_queue_secdiscard(q)	(blk_queue_discard(q) && \
 	test_bit(QUEUE_FLAG_SECDISCARD, &(q)->queue_flags))
 
+#define blk_queue_exp_dmpg(q) \
+	test_bit(QUEUE_FLAG_EXP_DMPG, &(q)->queue_flags)
+
+#define blk_queue_exp_swapin(q) \
+	test_bit(QUEUE_FLAG_EXP_SWAPIN, &(q)->queue_flags)
+
 #define blk_noretry_request(rq) \
 	((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \
 			     REQ_FAILFAST_DRIVER))
-- 
1.7.10.rc2

[RFC PATCH 04/11] block: Expedite DMPG and SWAPIN requests ahead of the queue

[Venkatraman S]

From: Ilan Smith <ilan.smith@sandisk.com>

When expedite_swapin and expedite_dmpg are set, move the
requests ahead of the queue.

Signed-off-by: Ilan Smith <ilan.smith@sandisk.com>
Signed-off-by: Alex Lemberg <alex.lemberg@sandisk.com>
Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 block/blk-core.c |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/block/blk-core.c b/block/blk-core.c
index 3a78b00..0b54ee9 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1305,6 +1305,12 @@ void init_request_from_bio(struct request *req, struct bio *bio)
 	if (bio->bi_rw & REQ_RAHEAD)
 		req->cmd_flags |= REQ_FAILFAST_MASK;
 
+	if (bio_swapin(bio) && blk_queue_exp_swapin(req->q))
+		req->cmd_flags |= REQ_RW_SWAPIN;
+
+	if (bio_dmpg(bio) && blk_queue_exp_dmpg(req->q))
+		req->cmd_flags |= REQ_RW_DMPG;
+
 	req->errors = 0;
 	req->__sector = bio->bi_sector;
 	req->ioprio = bio_prio(bio);
@@ -1332,6 +1338,18 @@ void blk_queue_bio(struct request_queue *q, struct bio *bio)
 		goto get_rq;
 	}
 
+	if (bio_swapin(bio) && blk_queue_exp_swapin(q)) {
+		spin_lock_irq(q->queue_lock);
+		where = ELEVATOR_INSERT_FRONT;
+		goto get_rq;
+	}
+
+	if (bio_dmpg(bio) && blk_queue_exp_dmpg(q)) {
+		spin_lock_irq(q->queue_lock);
+		where = ELEVATOR_INSERT_FRONT;
+		goto get_rq;
+	}
+
 	/*
 	 * Check if we can merge with the plugged list before grabbing
 	 * any locks.
-- 
1.7.10.rc2

[RFC PATCH 05/11] mmc: Add BKOPS field offsets

[Venkatraman S]

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 include/linux/mmc/mmc.h |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/linux/mmc/mmc.h b/include/linux/mmc/mmc.h
index b822a2c..ec2f195 100644
--- a/include/linux/mmc/mmc.h
+++ b/include/linux/mmc/mmc.h
@@ -280,6 +280,8 @@ struct _mmc_csd {
 #define EXT_CSD_PARTITION_SUPPORT	160	/* RO */
 #define EXT_CSD_HPI_MGMT		161	/* R/W */
 #define EXT_CSD_RST_N_FUNCTION		162	/* R/W */
+#define EXT_CSD_BKOPS_EN		163	/* R/W */
+#define EXT_CSD_BKOPS_START		164	/* W */
 #define EXT_CSD_SANITIZE_START		165     /* W */
 #define EXT_CSD_WR_REL_PARAM		166	/* RO */
 #define EXT_CSD_BOOT_WP			173	/* R/W */
-- 
1.7.10.rc2

[RFC PATCH 06/11] mmc: core: Helper function for finding preemptible command

[Venkatraman S]

According to Table 30 in eMMC spec, only some commands
can be preempted by HPI. Provide a helper function for
the HPI procedure to identify if the command is
preemptible.

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 include/linux/mmc/core.h |   13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/include/linux/mmc/core.h b/include/linux/mmc/core.h
index 1b431c7..680e256 100644
--- a/include/linux/mmc/core.h
+++ b/include/linux/mmc/core.h
@@ -10,6 +10,7 @@
 
 #include <linux/interrupt.h>
 #include <linux/completion.h>
+#include <linux/mmc/mmc.h>
 
 struct request;
 struct mmc_data;
@@ -192,6 +193,18 @@ static inline void mmc_claim_host(struct mmc_host *host)
 	__mmc_claim_host(host, NULL);
 }
 
+static inline bool mmc_is_preemptible_command(struct mmc_command *cmd)
+{
+	if ((cmd->opcode == MMC_SWITCH && (cmd->arg == EXT_CSD_BKOPS_START ||
+		cmd->arg == EXT_CSD_SANITIZE_START ||
+		cmd->arg == EXT_CSD_FLUSH_CACHE))
+		|| (cmd->opcode == MMC_ERASE)
+		|| (cmd->opcode == MMC_WRITE_MULTIPLE_BLOCK)
+		|| (cmd->opcode == MMC_WRITE_BLOCK))
+		return true;
+	return false;
+}
+
 extern u32 mmc_vddrange_to_ocrmask(int vdd_min, int vdd_max);
 
 #endif /* LINUX_MMC_CORE_H */
-- 
1.7.10.rc2

[RFC PATCH 07/11] mmc: core: add preemptibility tracking fields to mmc command

[Venkatraman S]

Set a preemptibility command atrribute to MMC commands. This
can be later used by write (multi block), trim etc for
evaluating if a HPI is applicable.

Note the starting time of executing a command so a decision
can be made if it is too late for preemption.

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 drivers/mmc/core/core.c  |    5 +++++
 include/linux/mmc/core.h |    4 ++++
 2 files changed, 9 insertions(+)

diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 027c579..6cefa01 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -258,6 +258,11 @@ static int __mmc_start_req(struct mmc_host *host, struct mmc_request *mrq)
 		complete(&mrq->completion);
 		return -ENOMEDIUM;
 	}
+	if (mmc_is_preemptible_command(mrq->cmd))
+		mrq->cmd->cmd_attr |= MMC_CMD_PREEMPTIBLE;
+	else
+		mrq->cmd->cmd_attr &= ~MMC_CMD_PREEMPTIBLE;
+	mrq->cmd->started_time = jiffies;
 	mmc_start_request(host, mrq);
 	return 0;
 }
diff --git a/include/linux/mmc/core.h b/include/linux/mmc/core.h
index 680e256..d86144e 100644
--- a/include/linux/mmc/core.h
+++ b/include/linux/mmc/core.h
@@ -76,6 +76,10 @@ struct mmc_command {
  */
 #define mmc_cmd_type(cmd)	((cmd)->flags & MMC_CMD_MASK)
 
+	unsigned int		cmd_attr; /*Runtime attributes of the command */
+#define MMC_CMD_PREEMPTIBLE	BIT(0)
+#define MMC_CMD_PREEMPTED	BIT(1)
+	unsigned long		started_time;
 	unsigned int		retries;	/* max number of retries */
 	unsigned int		error;		/* command error */
 
-- 
1.7.10.rc2

[RFC PATCH 08/11] mmc: core: Add MMC abort interface

[Venkatraman S]

HPI (and possibly other) procedures require that an ongoing
mmc request issued to a controller be aborted in the middle
of a transaction. Define a abort interface function to the
controller so that individual host controllers can safely
abort a request, stop the dma and cleanup their statemachine
etc. The implementation is controller dependant

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 drivers/mmc/core/core.c  |    8 ++++++++
 include/linux/mmc/host.h |    1 +
 2 files changed, 9 insertions(+)

diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 6cefa01..1963305 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -328,6 +328,14 @@ static void mmc_post_req(struct mmc_host *host, struct mmc_request *mrq,
 	}
 }
 
+static int mmc_abort_req(struct mmc_host *host, struct mmc_request *req)
+{
+	if (host->ops->abort_req)
+		return host->ops->abort_req(host, req);
+
+	return -ENOSYS;
+}
+
 /**
  *	mmc_start_req - start a non-blocking request
  *	@host: MMC host to start command
diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h
index cbde4b7..826e34d 100644
--- a/include/linux/mmc/host.h
+++ b/include/linux/mmc/host.h
@@ -98,6 +98,7 @@ struct mmc_host_ops {
 			    int err);
 	void	(*pre_req)(struct mmc_host *host, struct mmc_request *req,
 			   bool is_first_req);
+	int	(*abort_req)(struct mmc_host *host, struct mmc_request *req);
 	void	(*request)(struct mmc_host *host, struct mmc_request *req);
 	/*
 	 * Avoid calling these three functions too often or in a "fast path",
-- 
1.7.10.rc2

[RFC PATCH 09/11] mmc: block: Detect HPI support in card and host controller

[Venkatraman S]

If both the card and host controller support HPI related
operations, set a flag in MMC queue to remember it.

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 drivers/mmc/card/block.c |   12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/mmc/card/block.c b/drivers/mmc/card/block.c
index dabec55..11833e4 100644
--- a/drivers/mmc/card/block.c
+++ b/drivers/mmc/card/block.c
@@ -88,6 +88,7 @@ struct mmc_blk_data {
 	unsigned int	flags;
 #define MMC_BLK_CMD23	(1 << 0)	/* Can do SET_BLOCK_COUNT for multiblock */
 #define MMC_BLK_REL_WR	(1 << 1)	/* MMC Reliable write support */
+#define MMC_HPI_SUPPORT	(1 << 2)
 
 	unsigned int	usage;
 	unsigned int	read_only;
@@ -1548,12 +1549,15 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card,
 			md->flags |= MMC_BLK_CMD23;
 	}
 
-	if (mmc_card_mmc(card) &&
-	    md->flags & MMC_BLK_CMD23 &&
+	if (mmc_card_mmc(card)) {
+		if (md->flags & MMC_BLK_CMD23 &&
 	    ((card->ext_csd.rel_param & EXT_CSD_WR_REL_PARAM_EN) ||
 	     card->ext_csd.rel_sectors)) {
-		md->flags |= MMC_BLK_REL_WR;
-		blk_queue_flush(md->queue.queue, REQ_FLUSH | REQ_FUA);
+			md->flags |= MMC_BLK_REL_WR;
+			blk_queue_flush(md->queue.queue, REQ_FLUSH | REQ_FUA);
+		}
+		if (card->host->ops->abort_req && card->ext_csd.hpi_en)
+			md->flags |= MMC_HPI_SUPPORT;
 	}
 
 	return md;
-- 
1.7.10.rc2

[RFC PATCH 10/11] mmc: core: Utility function for mmc preemption sequence

[Venkatraman S]

Add a routine to invoke hpi command after invoking host
controller abort.

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 drivers/mmc/core/core.c  |   15 +++++++++++++++
 include/linux/mmc/core.h |    2 ++
 2 files changed, 17 insertions(+)

diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index 1963305..7230a26 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -467,6 +467,21 @@ out:
 }
 EXPORT_SYMBOL(mmc_interrupt_hpi);
 
+int mmc_preempt_foreground_request(struct mmc_card *card,
+	struct mmc_request *req)
+{
+	int ret;
+
+	ret = mmc_abort_req(card->host, req);
+	if (ret)
+		pr_err("%s: Host Abort failed %d\n",
+			mmc_hostname(card->host), ret);
+	else
+		ret = mmc_interrupt_hpi(card);
+	return ret;
+}
+EXPORT_SYMBOL(mmc_preempt_foreground_request);
+
 /**
  *	mmc_wait_for_cmd - start a command and wait for completion
  *	@host: MMC host to start command
diff --git a/include/linux/mmc/core.h b/include/linux/mmc/core.h
index d86144e..e2d55c6 100644
--- a/include/linux/mmc/core.h
+++ b/include/linux/mmc/core.h
@@ -144,6 +144,8 @@ extern struct mmc_async_req *mmc_start_req(struct mmc_host *,
 extern int mmc_interrupt_hpi(struct mmc_card *);
 extern void mmc_wait_for_req(struct mmc_host *, struct mmc_request *);
 extern int mmc_wait_for_cmd(struct mmc_host *, struct mmc_command *, int);
+extern int mmc_preempt_foreground_request(struct mmc_card *card,
+	struct mmc_request *req);
 extern int mmc_app_cmd(struct mmc_host *, struct mmc_card *);
 extern int mmc_wait_for_app_cmd(struct mmc_host *, struct mmc_card *,
 	struct mmc_command *, int);
-- 
1.7.10.rc2

[RFC PATCH 11/11] mmc: block: Implement HPI invocation and handling for foreground requests

[Venkatraman S]

Intercept requests which require high priority treatment.
An ongoing command can be preempted if
a) it is one of CMD 24, 25, 38 or CMD6(with BKOPS START)
b) a significant amount of time isn't elapsed since the
   start of the command that it is worthwhile to abort it
c) the next request is a DMPG or SWAPIN request

For practical reasons, HPI would be invoked for host controllers
 which support the abort_req interface. Technically, the
host controller driver can implement it as a no op. But the
abort would usually be required to clean up the context
variables and bypass the statemachine which requires it to
wait till the card is back to transfer state after completing
the write.

Signed-off-by: Venkatraman S <svenkatr@ti.com>
---
 drivers/mmc/card/block.c |  126 ++++++++++++++++++++++++++++++++++++++++++----
 drivers/mmc/card/queue.h |    1 +
 2 files changed, 117 insertions(+), 10 deletions(-)

diff --git a/drivers/mmc/card/block.c b/drivers/mmc/card/block.c
index 11833e4..de0634f 100644
--- a/drivers/mmc/card/block.c
+++ b/drivers/mmc/card/block.c
@@ -57,6 +57,9 @@ MODULE_ALIAS("mmc:block");
 #define INAND_CMD38_ARG_SECERASE 0x80
 #define INAND_CMD38_ARG_SECTRIM1 0x81
 #define INAND_CMD38_ARG_SECTRIM2 0x88
+/* TODO: Convert the below macro to sysfs to tune this dynamically */
+#define MMC_HPI_PREEMPT_TIME_THRESHOLD 10 /* in msec */
+
 
 static DEFINE_MUTEX(block_mutex);
 
@@ -1276,7 +1279,7 @@ static int mmc_blk_cmd_err(struct mmc_blk_data *md, struct mmc_card *card,
 	return ret;
 }
 
-static int mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *rqc)
+static int mmc_blk_execute_rw_rq(struct mmc_queue *mq, struct request *rqc)
 {
 	struct mmc_blk_data *md = mq->data;
 	struct mmc_card *card = md->queue.card;
@@ -1285,22 +1288,31 @@ static int mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *rqc)
 	enum mmc_blk_status status;
 	struct mmc_queue_req *mq_rq;
 	struct request *req;
-	struct mmc_async_req *areq;
+	struct mmc_async_req *prev_req, *cur_req;
 
 	if (!rqc && !mq->mqrq_prev->req)
 		return 0;
 
+	mq->mqrq_interrupted = NULL;
 	do {
 		if (rqc) {
 			mmc_blk_rw_rq_prep(mq->mqrq_cur, card, 0, mq);
-			areq = &mq->mqrq_cur->mmc_active;
-		} else
-			areq = NULL;
-		areq = mmc_start_req(card->host, areq, (int *) &status);
-		if (!areq)
+			cur_req = &mq->mqrq_cur->mmc_active;
+		} else {
+			cur_req = NULL;
+		}
+		prev_req = mmc_start_req(card->host, cur_req, (int *) &status);
+		if (!prev_req)
 			return 0;
 
-		mq_rq = container_of(areq, struct mmc_queue_req, mmc_active);
+		if (cur_req &&
+			cur_req->mrq->cmd->cmd_attr & MMC_CMD_PREEMPTIBLE) {
+			mq->mqrq_interrupted = mq->mqrq_cur;
+			pr_info("%x can be preempted\n", mq->mqrq_interrupted);
+		}
+
+		mq_rq = container_of(prev_req,
+			struct mmc_queue_req, mmc_active);
 		brq = &mq_rq->brq;
 		req = mq_rq->req;
 		type = rq_data_dir(req) == READ ? MMC_BLK_READ : MMC_BLK_WRITE;
@@ -1406,6 +1418,100 @@ static int mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *rqc)
 	return 0;
 }
 
+#define HPI_CHECK  (REQ_RW_SWAPIN | REQ_RW_DMPG)
+
+static bool mmc_can_do_foreground_hpi(struct mmc_queue *mq,
+					struct request *req)
+{
+
+	/* If some time has elapsed since the issuing of previous write
+	command, or if the size of the request was too small, there's
+	no point in preempting it. Check if it's worthwhile to preempt */
+	int time_elapsed = jiffies_to_msecs(jiffies -
+			mq->mqrq_cur->mmc_active.mrq->cmd->started_time);
+
+	if ((time_elapsed < MMC_HPI_PREEMPT_TIME_THRESHOLD))
+			return true;
+
+	return false;
+}
+
+/*  When a HPI command had been given for a foreground
+ *  request, the host controller will finish the request,
+ *  the completion request has to be handled differently
+ */
+
+static struct mmc_async_req *mmc_handle_aborted_request(struct mmc_queue *mq,
+	int hpi_err)
+{
+	struct mmc_async_req *areq;
+	struct mmc_request *mrq;
+	struct mmc_queue_req *mq_rq;
+	struct mmc_blk_data *md = mq->data;
+	struct request *req;
+
+	BUG_ON(!mq->mqrq_interrupted);
+
+	areq = &mq->mqrq_interrupted->mmc_active;
+	mrq = areq->mrq;
+	wait_for_completion(&mrq->completion);
+	/* Error checking is TBD
+	err = areq->err_check(card, areq);  */
+	mq_rq = container_of(areq, struct mmc_queue_req, mmc_active);
+	req = mq_rq->req;
+	mmc_queue_bounce_post(mq_rq);
+
+	spin_lock_irq(&md->lock);
+	/* TODO. Collect the correctly_programmed_sectors_num from card
+	and then pass it as parameter for __blk_end_request */
+	__blk_end_request(req, -EIO, 0);
+	spin_unlock_irq(&md->lock);
+	return areq;
+}
+
+static int mmc_blk_issue_rw_rq(struct mmc_queue *mq, struct request *req)
+{
+	int ret;
+	struct mmc_blk_data *md = mq->data;
+	struct mmc_card *card = md->queue.card;
+	struct mmc_async_req *areq;
+
+	if (req && md->flags & MMC_HPI_SUPPORT) {
+		if (!((req->cmd_flags & HPI_CHECK) && mq->mqrq_interrupted))
+			goto no_preempt;
+		if (!mmc_can_do_foreground_hpi(mq, req))
+			goto no_preempt;
+
+		ret = mmc_preempt_foreground_request(card,
+			 mq->mqrq_interrupted->mmc_active.mrq);
+		if (ret)
+			/* Couldn't execute HPI, or the request could
+			 * have been completed already. Go through
+			 * the normal route */
+			goto no_preempt;
+
+		areq = mmc_handle_aborted_request(mq, ret);
+		/* Remove the request from the host controller's
+		 * request queue. This prevents normal error handling
+		 * and retry procedures from executing (we know the
+		 * request has been aborted anyway). This also helps to start
+		 * the urgent requests without doing the post processing
+		 * of the aborted request
+		 */
+		 card->host->areq = NULL;
+
+		/* Now the decks are clear to send the most urgent command.
+		As we've preempted the ongoing one already, the urgent
+		one can go through the normal queue and it won't face much
+		resistance - hence the intentional fall through */
+		BUG_ON(areq != &mq->mqrq_interrupted->mmc_active);
+	}
+
+no_preempt:
+	ret = mmc_blk_execute_rw_rq(mq, req);
+	return ret;
+}
+
 static int mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
 {
 	int ret;
@@ -1430,7 +1536,7 @@ static int mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
 	if (req && req->cmd_flags & REQ_DISCARD) {
 		/* complete ongoing async transfer before issuing discard */
 		if (card->host->areq)
-			mmc_blk_issue_rw_rq(mq, NULL);
+			mmc_blk_execute_rw_rq(mq, NULL);
 		if (req->cmd_flags & REQ_SECURE)
 			ret = mmc_blk_issue_secdiscard_rq(mq, req);
 		else
@@ -1438,7 +1544,7 @@ static int mmc_blk_issue_rq(struct mmc_queue *mq, struct request *req)
 	} else if (req && req->cmd_flags & REQ_FLUSH) {
 		/* complete ongoing async transfer before issuing flush */
 		if (card->host->areq)
-			mmc_blk_issue_rw_rq(mq, NULL);
+			mmc_blk_execute_rw_rq(mq, NULL);
 		ret = mmc_blk_issue_flush(mq, req);
 	} else {
 		ret = mmc_blk_issue_rw_rq(mq, req);
diff --git a/drivers/mmc/card/queue.h b/drivers/mmc/card/queue.h
index d2a1eb4..7bd599e 100644
--- a/drivers/mmc/card/queue.h
+++ b/drivers/mmc/card/queue.h
@@ -33,6 +33,7 @@ struct mmc_queue {
 	struct mmc_queue_req	mqrq[2];
 	struct mmc_queue_req	*mqrq_cur;
 	struct mmc_queue_req	*mqrq_prev;
+	struct mmc_queue_req	*mqrq_interrupted;
 };
 
 extern int mmc_init_queue(struct mmc_queue *, struct mmc_card *, spinlock_t *,
-- 
1.7.10.rc2

Re: [RFC PATCH 00/11] [FS, MM, block, MMC]: eMMC High Priority Interrupt Feature

[Konstantin Dorfman]

On Wed, April 18, 2012 9:25 am, Venkatraman S wrote:
Hello,

> a) At the top level, some policy decisions have to be made on what is
> worth preempting for.
> 	This implementation uses the demand paging requests and swap
> read requests as potential reads worth preempting an ongoing long write.
> 	This is expected to provide improved responsiveness for smarphones
> with multitasking capabilities - example would be launch a email
> application
> while a video capture session (which causes long writes) is ongoing.
> b) At the block handler, the higher priority request should be queued
>   ahead of the pending requests in the elevator
> c) At the MMC block and core level, transactions have to executed to
> enforce the rules of the MMC spec and make a reasonable tradeoff if the
> ongoing command is really worth preempting. (For example, is it too close
> to completing already ?).

Do you have some profiling information (on real or synthetic scenarios)
that may prove/show improvement in read latency for your design?

It could be useful to use blktrace engine with some post processing to get
per request (or per block) latency.

Also you can gather some statistics about how often demand paging and swap
read requests occurs during typical user scenarios.

Without such statistical analysis we are risking to do hard work with zero
benefits.
Does this make sense?

Thanks,
Kostya
Consultant for Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum

Re: [RFC PATCH 00/11] [FS, MM, block, MMC]: eMMC High Priority Interrupt Feature

[Sergey Priporov]

Hi Venkat,

I was looking for some explanations of eMMC behavior in products we  are 
developing now and found your comments around eMMC here. Our observation shows 
that that eMMC performance vary significantly and depends on volume of data to 
write during time slot and temperature. After a heavy write test was run for 
some time for a good performance eMMC (showing 10-12MB/s of the write 
performance on big blocks) its write performance degraded to the 4-5MB/s.
We even observed performance lowest level as 65KB/s! 
There are a lot of Write type of operations necessary into dedicated eMMC 
partition by requirements.
The side effect is that long write operations block  Application UI to redraw 
the  views  because it requires Read operations.
Visually we see “Blank Dark Screen” for few seconds and even ANRs. So HMI may 
help us.

Question:  Did you finish research with HPI to try ?  Which type of eMMC did 
you use?

P.S. We use TI BSP for OMAP3630 for Android 

Re: [RFC PATCH 00/11] [FS, MM, block, MMC]: eMMC High Priority Interrupt Feature

[Konstantin Dorfman]

Hello Sergey,

I'm working on the same flow - to reduce read latency hit (as a result 
of big aggregated writes). I plan to send all relevant patches to the 
mailing list soon and it will be great if you can test them on your system.

There are requirements for this flow is:
  - eMMC 4.5 supported by card (this means HPI)
  - host controller shoud implement stop request api (that is be able 
correctly stop DMA & all internal state and be ready for next read 
transaction.

Right now, please look at this patches as reference:

1. [RFC/PATCH 0/2] Handling urgent and new request notifications.
2. [RFC/PATCH 1/2] mmc: Urgent data request flow.

This is old changes, just for the solution overview.

Thanks,

> Hi Venkat,
>
> I was looking for some explanations of eMMC behavior in products we  are
> developing now and found your comments around eMMC here. Our observation
> shows
> that that eMMC performance vary significantly and depends on volume of
> data to
> write during time slot and temperature. After a heavy write test was run
> for
> some time for a good performance eMMC (showing 10-12MB/s of the write
> performance on big blocks) its write performance degraded to the 4-5MB/s.
> We even observed performance lowest level as 65KB/s!
> There are a lot of Write type of operations necessary into dedicated eMMC
> partition by requirements.
> The side effect is that long write operations block  Application UI to
> redraw
> the  views  because it requires Read operations.
> Visually we see “Blank Dark Screen” for few seconds and even ANRs. So
> HMI may
> help us.
>
> Question:  Did you finish research with HPI to try ?  Which type of eMMC
> did
> you use?
>
> P.S. We use TI BSP for OMAP3630 for Android
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-mmc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [RFC PATCH 00/11] [FS, MM, block, MMC]: eMMC High Priority Interrupt Feature

[Sergey Priporov]

Konstantin,

Thank you for update. We also are reviewing file IO operations
scheduler . Looks like for eMMC  current scheduler is not good
tailored (since eMMC is not hard drive :) )

Sergey

On Wed, Jan 9, 2013 at 12:15 AM, Konstantin Dorfman
<kdorfman@codeaurora.org> wrote:
> Hello Sergey,
>
> I'm working on the same flow - to reduce read latency hit (as a result of
> big aggregated writes). I plan to send all relevant patches to the mailing
> list soon and it will be great if you can test them on your system.
>
> There are requirements for this flow is:
>  - eMMC 4.5 supported by card (this means HPI)
>  - host controller shoud implement stop request api (that is be able
> correctly stop DMA & all internal state and be ready for next read
> transaction.
>
> Right now, please look at this patches as reference:
>
> 1. [RFC/PATCH 0/2] Handling urgent and new request notifications.
> 2. [RFC/PATCH 1/2] mmc: Urgent data request flow.
>
> This is old changes, just for the solution overview.
>
> Thanks,
>
>> Hi Venkat,
>>
>> I was looking for some explanations of eMMC behavior in products we  are
>> developing now and found your comments around eMMC here. Our observation
>> shows
>> that that eMMC performance vary significantly and depends on volume of
>> data to
>> write during time slot and temperature. After a heavy write test was run
>> for
>> some time for a good performance eMMC (showing 10-12MB/s of the write
>> performance on big blocks) its write performance degraded to the 4-5MB/s.
>> We even observed performance lowest level as 65KB/s!
>> There are a lot of Write type of operations necessary into dedicated eMMC
>> partition by requirements.
>> The side effect is that long write operations block  Application UI to
>> redraw
>> the  views  because it requires Read operations.
>> Visually we see “Blank Dark Screen† for few seconds and even ANRs. So
>> HMI may
>> help us.
>>
>> Question:  Did you finish research with HPI to try ?  Which type of eMMC
>> did
>> you use?
>>
>> P.S. We use TI BSP for OMAP3630 for Android
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-mmc" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>