From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55EF9C43387 for ; Fri, 21 Dec 2018 19:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1B6C921928 for ; Fri, 21 Dec 2018 19:22:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="x9QRHI+6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391657AbeLUTWv (ORCPT ); Fri, 21 Dec 2018 14:22:51 -0500 Received: from mail-io1-f67.google.com ([209.85.166.67]:46034 "EHLO mail-io1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391644AbeLUTWu (ORCPT ); Fri, 21 Dec 2018 14:22:50 -0500 Received: by mail-io1-f67.google.com with SMTP id p7so1308361iog.12 for ; Fri, 21 Dec 2018 11:22:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=N5ta7ic2B2jMS544hhhxgjM1PLBspMmNNpvDXhuFWys=; b=x9QRHI+6RryVJJ8r7pl2DLpCYWvY8IJPzcROUOZUBqiITkW5Jrwjbx6YglS5lBNYd4 zOR76ZZ/30gZD5ej7BBKr6LiCBZlY6ybWoyesXXH3LDyPy+mTj4kON+ZnJwHW1RCo9rR f2gQ8JeNyuw9gr1H32WHpHqMBoOpZwS3A6R9jO2t/StOCucU8nMA08Yetba2qSAzjSj1 q2ENypufDGHGyJGs1PitjF6VwFsGnmVJnA2hbGIjYsjwXbV4N4CUyttNT+VhsVIuUcYH CLUye2KZ3pEzeks7TFUxjvSbd0zzN8OPB4akKGg0fltqibKdOVFAuzMs3sINBP+AIUnt +mFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=N5ta7ic2B2jMS544hhhxgjM1PLBspMmNNpvDXhuFWys=; b=T5XAnqO0Dy3ciQ292geCnyVAbH+6McOn7vJ0SQrSXKr3qbYCITxiX4NAmpaSBuwZ4e KtxU9+aCZQ+urd3owkdfBPLrNtHvGuHeCFryti6V9vVhq+a8mWfI1bup3n5wkCaJ1dnj oLfOOwLZvzD7GNguXRHi79lqUnL4kznfJujZD4Gqz1U6frIKOu1PIgi9tUaQLxn7UmIz 4cPTRGS0f8x7XhLdRlgND0y4GjkfZYLWj83KU/F7DFvbVIF7jk8ig+dh48s5bsT+5KIN OC8i71TWugRUK2vD1SkUvOxO4pAuBr6uJafxVsdDG/QUAl9T47u19c4bzEkG4dlGawQc /m7Q== X-Gm-Message-State: AJcUukdKUht5eTXYsGC/maiItFTLTIxUahkBCLKnkPonOsxiX/vYJKyi Y06lL6cyiiB/hxOyV01u+PDsRQ== X-Google-Smtp-Source: ALg8bN4zgDz6FTuGfm34H3gFzgVz5wGCMIKgZEHLzCKpoE+SzNkoGxxB92JI4BxUJbYShoxcoB24Qw== X-Received: by 2002:a6b:919:: with SMTP id t25mr2414900ioi.207.1545420169249; Fri, 21 Dec 2018 11:22:49 -0800 (PST) Received: from localhost.localdomain ([216.160.245.98]) by smtp.gmail.com with ESMTPSA id t1sm12456290iol.85.2018.12.21.11.22.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Dec 2018 11:22:48 -0800 (PST) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-aio@kvack.org, linux-block@vger.kernel.org Cc: hch@lst.de, viro@zeniv.linux.org.uk, Jens Axboe Subject: [PATCH 06/22] iomap: wire up the iopoll method Date: Fri, 21 Dec 2018 12:22:20 -0700 Message-Id: <20181221192236.12866-7-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181221192236.12866-1-axboe@kernel.dk> References: <20181221192236.12866-1-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Christoph Hellwig Store the request queue the last bio was submitted to in the iocb private data in addition to the cookie so that we find the right block device. Also refactor the common direct I/O bio submission code into a nice little helper. Signed-off-by: Christoph Hellwig Modified to use bio_set_polled(). Signed-off-by: Jens Axboe --- fs/gfs2/file.c | 2 ++ fs/iomap.c | 43 ++++++++++++++++++++++++++++--------------- fs/xfs/xfs_file.c | 1 + include/linux/iomap.h | 1 + 4 files changed, 32 insertions(+), 15 deletions(-) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 45a17b770d97..358157efc5b7 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -1280,6 +1280,7 @@ const struct file_operations gfs2_file_fops = { .llseek = gfs2_llseek, .read_iter = gfs2_file_read_iter, .write_iter = gfs2_file_write_iter, + .iopoll = iomap_dio_iopoll, .unlocked_ioctl = gfs2_ioctl, .mmap = gfs2_mmap, .open = gfs2_open, @@ -1310,6 +1311,7 @@ const struct file_operations gfs2_file_fops_nolock = { .llseek = gfs2_llseek, .read_iter = gfs2_file_read_iter, .write_iter = gfs2_file_write_iter, + .iopoll = iomap_dio_iopoll, .unlocked_ioctl = gfs2_ioctl, .mmap = gfs2_mmap, .open = gfs2_open, diff --git a/fs/iomap.c b/fs/iomap.c index 99f545e0641f..46f4cb687f6f 100644 --- a/fs/iomap.c +++ b/fs/iomap.c @@ -1448,6 +1448,28 @@ struct iomap_dio { }; }; +int iomap_dio_iopoll(struct kiocb *kiocb, bool spin) +{ + struct request_queue *q = READ_ONCE(kiocb->private); + + if (!q) + return 0; + return blk_poll(q, READ_ONCE(kiocb->ki_cookie), spin); +} +EXPORT_SYMBOL_GPL(iomap_dio_iopoll); + +static void iomap_dio_submit_bio(struct iomap_dio *dio, struct iomap *iomap, + struct bio *bio) +{ + atomic_inc(&dio->ref); + + if (dio->iocb->ki_flags & IOCB_HIPRI) + bio_set_polled(bio, dio->iocb); + + dio->submit.last_queue = bdev_get_queue(iomap->bdev); + dio->submit.cookie = submit_bio(bio); +} + static ssize_t iomap_dio_complete(struct iomap_dio *dio) { struct kiocb *iocb = dio->iocb; @@ -1560,7 +1582,7 @@ static void iomap_dio_bio_end_io(struct bio *bio) } } -static blk_qc_t +static void iomap_dio_zero(struct iomap_dio *dio, struct iomap *iomap, loff_t pos, unsigned len) { @@ -1574,15 +1596,10 @@ iomap_dio_zero(struct iomap_dio *dio, struct iomap *iomap, loff_t pos, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - if (dio->iocb->ki_flags & IOCB_HIPRI) - flags |= REQ_HIPRI; - get_page(page); __bio_add_page(bio, page, len, 0); bio_set_op_attrs(bio, REQ_OP_WRITE, flags); - - atomic_inc(&dio->ref); - return submit_bio(bio); + iomap_dio_submit_bio(dio, iomap, bio); } static loff_t @@ -1685,9 +1702,6 @@ iomap_dio_bio_actor(struct inode *inode, loff_t pos, loff_t length, bio_set_pages_dirty(bio); } - if (dio->iocb->ki_flags & IOCB_HIPRI) - bio->bi_opf |= REQ_HIPRI; - iov_iter_advance(dio->submit.iter, n); dio->size += n; @@ -1695,11 +1709,7 @@ iomap_dio_bio_actor(struct inode *inode, loff_t pos, loff_t length, copied += n; nr_pages = iov_iter_npages(&iter, BIO_MAX_PAGES); - - atomic_inc(&dio->ref); - - dio->submit.last_queue = bdev_get_queue(iomap->bdev); - dio->submit.cookie = submit_bio(bio); + iomap_dio_submit_bio(dio, iomap, bio); } while (nr_pages); /* @@ -1910,6 +1920,9 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, if (dio->flags & IOMAP_DIO_WRITE_FUA) dio->flags &= ~IOMAP_DIO_NEED_SYNC; + WRITE_ONCE(iocb->ki_cookie, dio->submit.cookie); + WRITE_ONCE(iocb->private, dio->submit.last_queue); + if (!atomic_dec_and_test(&dio->ref)) { if (!dio->wait_for_completion) return -EIOCBQUEUED; diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index e47425071e65..60c2da41f0fc 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1203,6 +1203,7 @@ const struct file_operations xfs_file_operations = { .write_iter = xfs_file_write_iter, .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, + .iopoll = iomap_dio_iopoll, .unlocked_ioctl = xfs_file_ioctl, #ifdef CONFIG_COMPAT .compat_ioctl = xfs_file_compat_ioctl, diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 9a4258154b25..0fefb5455bda 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -162,6 +162,7 @@ typedef int (iomap_dio_end_io_t)(struct kiocb *iocb, ssize_t ret, unsigned flags); ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, iomap_dio_end_io_t end_io); +int iomap_dio_iopoll(struct kiocb *kiocb, bool spin); #ifdef CONFIG_SWAP struct file; -- 2.17.1