[PATCH V2] block: avoid io timeout in case of sync polled dio

[Ming Lei <ming.lei@redhat.com>]

For POLLED bio, bio_poll() should be called for reaping io since
there isn't irq for completing the request, so we can't call into
blk_io_schedule() in case that bio_poll() returns zero, otherwise
io timeout is easily triggered.

Also before calling bio_poll(), current->plug should be flushed out,
otherwise the bio may not be issued to driver, and ->bi_cookie won't
be set.

CC: linux-mm@kvack.org
Cc: linux-xfs@vger.kernel.org
Reported-by: Changhui Zhong <czhong@redhat.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
V2:
	- as pointed out by Christoph, iomap & mm code should be covered

 block/fops.c         | 7 ++++++-
 fs/iomap/direct-io.c | 7 +++++--
 mm/page_io.c         | 8 +++++++-
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/block/fops.c b/block/fops.c
index 9f2ecec406b0..c9bac700e072 100644
--- a/block/fops.c
+++ b/block/fops.c
@@ -101,11 +101,16 @@ static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb,
 		bio_set_polled(&bio, iocb);
 
 	submit_bio(&bio);
+	/* make sure the bio is issued before polling */
+	if (bio.bi_opf & REQ_POLLED)
+		blk_flush_plug(current->plug, false);
 	for (;;) {
 		set_current_state(TASK_UNINTERRUPTIBLE);
 		if (!READ_ONCE(bio.bi_private))
 			break;
-		if (!(iocb->ki_flags & IOCB_HIPRI) || !bio_poll(&bio, NULL, 0))
+		if (bio.bi_opf & REQ_POLLED)
+			bio_poll(&bio, NULL, 0);
+		else
 			blk_io_schedule();
 	}
 	__set_current_state(TASK_RUNNING);
diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index b08f5dc31780..e67d2f63a163 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -654,8 +654,11 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
 			if (!READ_ONCE(dio->submit.waiter))
 				break;
 
-			if (!dio->submit.poll_bio ||
-			    !bio_poll(dio->submit.poll_bio, NULL, 0))
+			if (dio->submit.poll_bio &&
+					(dio->submit.poll_bio->bi_opf &
+						REQ_POLLED))
+				bio_poll(dio->submit.poll_bio, NULL, 0);
+			else
 				blk_io_schedule();
 		}
 		__set_current_state(TASK_RUNNING);
diff --git a/mm/page_io.c b/mm/page_io.c
index b417f000b49e..16f2a63e2524 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -421,12 +421,18 @@ int swap_readpage(struct page *page, bool synchronous)
 	count_vm_event(PSWPIN);
 	bio_get(bio);
 	submit_bio(bio);
+
+	/* make sure the bio is issued before polling */
+	if (bio->bi_opf & REQ_POLLED)
+		blk_flush_plug(current->plug, false);
 	while (synchronous) {
 		set_current_state(TASK_UNINTERRUPTIBLE);
 		if (!READ_ONCE(bio->bi_private))
 			break;
 
-		if (!bio_poll(bio, NULL, 0))
+		if (bio->bi_opf & REQ_POLLED)
+			bio_poll(bio, NULL, 0);
+		else
 			blk_io_schedule();
 	}
 	__set_current_state(TASK_RUNNING);
-- 
2.31.1