From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C5BECD129A for ; Mon, 8 Apr 2024 01:41:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MEf5VmBE1vRtPrVDuuwxCTTEDYixXy+WxyWA+7yrZKE=; b=W/kGMeaNdw82zxL+v7pUSE2rg8 pRbOTjmCtw8LmCV7ciOqCPSL50LX295NxPl6e+w9chfgrFM6bC8VRCDvcKfNNuEISMQtFLaJ2FhNf m13Qmbs+fX68x7OME68it3D+J2+0oTuhoJssjMTQK7e7Dk8YO6kAchJUjfDvcKn8PEVT2KtthrWXw ayX85IDuly80jfJwOPUNia5ZGes+JWYd0nt/NMgGPRB4jZkeoHAzMEfqKqDH3PKMNFb6PqrY5NvU8 xLEbpumvY/StC6gMw4PA5UUFS33bJNYnN+f77J1gjbNGjqK+wtSPM5zizt9hO+B7ttBzWXMWn0ib4 JwAOLLFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rte0z-0000000DvGl-16Pc; Mon, 08 Apr 2024 01:41:41 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rte0t-0000000DvEB-291n for linux-nvme@lists.infradead.org; Mon, 08 Apr 2024 01:41:37 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id EF5BD60DCA; Mon, 8 Apr 2024 01:41:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87E08C43394; Mon, 8 Apr 2024 01:41:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712540494; bh=tVm7PssY2TPyBnuGPMiPzuiIeLZJRpmAqG68eKRZ1Ak=; h=From:To:Subject:Date:In-Reply-To:References:From; b=JTAlHaR6gdl/j1uS/K+kOawzaytSV3rcrcrEaNeR9YvH5qtIrXOpeHXO0KjNkNNGb qitTI/Q9pIqCbw4paqcNS9nOMRM8sl48SuxNGXuwXg5fUXdQrC5G3KIU46RsJJ9ali C/At0fmgH4VZSbXkuYf7u+hWthP626FSN7tICtcE+A1TYyTM7gvMSJbW8tb9a+leIJ PvcmOmKBHZ6wMTRB6MMoHP2bh/1y7Hj88d7H8t1E0XBOxe192w96m0bAJW2+Nkfly6 Av0XAQ6a0ZtgH0o1T81IJHcEtc4HUUJ6NDpUEJUqsR4kQwZVMTyHXGHIuuMI4gwWr5 HMPxvz4XDZQ+g== From: Damien Le Moal To: linux-block@vger.kernel.org, Jens Axboe , linux-scsi@vger.kernel.org, "Martin K . Petersen" , dm-devel@lists.linux.dev, Mike Snitzer , linux-nvme@lists.infradead.org, Keith Busch , Christoph Hellwig Subject: [PATCH v7 02/28] block: Remove req_bio_endio() Date: Mon, 8 Apr 2024 10:41:02 +0900 Message-ID: <20240408014128.205141-3-dlemoal@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240408014128.205141-1-dlemoal@kernel.org> References: <20240408014128.205141-1-dlemoal@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240407_184135_665588_3C24B71E X-CRM114-Status: GOOD ( 15.90 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org Moving req_bio_endio() code into its only caller, blk_update_request(), allows reducing accesses to and tests of bio and request fields. Also, given that partial completions of zone append operations is not possible and that zone append operations cannot be merged, the update of the BIO sector using the request sector for these operations can be moved directly before the call to bio_endio(). Signed-off-by: Damien Le Moal Reviewed-by: Christoph Hellwig Reviewed-by: Bart Van Assche Reviewed-by: Hannes Reinecke Tested-by: Hans Holmberg Tested-by: Dennis Maisenbacher Reviewed-by: Martin K. Petersen --- block/blk-mq.c | 58 ++++++++++++++++++++++++-------------------------- 1 file changed, 28 insertions(+), 30 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index b8dbfed8b28b..fcbf0953a179 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -762,31 +762,6 @@ void blk_dump_rq_flags(struct request *rq, char *msg) } EXPORT_SYMBOL(blk_dump_rq_flags); -static void req_bio_endio(struct request *rq, struct bio *bio, - unsigned int nbytes, blk_status_t error) -{ - if (unlikely(error)) { - bio->bi_status = error; - } else if (req_op(rq) == REQ_OP_ZONE_APPEND) { - /* - * Partial zone append completions cannot be supported as the - * BIO fragments may end up not being written sequentially. - */ - if (bio->bi_iter.bi_size != nbytes) - bio->bi_status = BLK_STS_IOERR; - else - bio->bi_iter.bi_sector = rq->__sector; - } - - bio_advance(bio, nbytes); - - if (unlikely(rq->rq_flags & RQF_QUIET)) - bio_set_flag(bio, BIO_QUIET); - /* don't actually finish bio if it's part of flush sequence */ - if (bio->bi_iter.bi_size == 0 && !(rq->rq_flags & RQF_FLUSH_SEQ)) - bio_endio(bio); -} - static void blk_account_io_completion(struct request *req, unsigned int bytes) { if (req->part && blk_do_io_stat(req)) { @@ -890,6 +865,8 @@ static void blk_complete_request(struct request *req) bool blk_update_request(struct request *req, blk_status_t error, unsigned int nr_bytes) { + bool is_flush = req->rq_flags & RQF_FLUSH_SEQ; + bool quiet = req->rq_flags & RQF_QUIET; int total_bytes; trace_block_rq_complete(req, error, nr_bytes); @@ -910,9 +887,8 @@ bool blk_update_request(struct request *req, blk_status_t error, if (blk_crypto_rq_has_keyslot(req) && nr_bytes >= blk_rq_bytes(req)) __blk_crypto_rq_put_keyslot(req); - if (unlikely(error && !blk_rq_is_passthrough(req) && - !(req->rq_flags & RQF_QUIET)) && - !test_bit(GD_DEAD, &req->q->disk->state)) { + if (unlikely(error && !blk_rq_is_passthrough(req) && !quiet) && + !test_bit(GD_DEAD, &req->q->disk->state)) { blk_print_req_error(req, error); trace_block_rq_error(req, error, nr_bytes); } @@ -924,12 +900,34 @@ bool blk_update_request(struct request *req, blk_status_t error, struct bio *bio = req->bio; unsigned bio_bytes = min(bio->bi_iter.bi_size, nr_bytes); - if (bio_bytes == bio->bi_iter.bi_size) + if (unlikely(error)) + bio->bi_status = error; + + if (bio_bytes == bio->bi_iter.bi_size) { req->bio = bio->bi_next; + } else if (req_op(req) == REQ_OP_ZONE_APPEND && + error == BLK_STS_OK) { + /* + * Partial zone append completions cannot be supported + * as the BIO fragments may end up not being written + * sequentially. + */ + bio->bi_status = BLK_STS_IOERR; + } /* Completion has already been traced */ bio_clear_flag(bio, BIO_TRACE_COMPLETION); - req_bio_endio(req, bio, bio_bytes, error); + if (unlikely(quiet)) + bio_set_flag(bio, BIO_QUIET); + + bio_advance(bio, bio_bytes); + + /* Don't actually finish bio if it's part of flush sequence */ + if (!bio->bi_iter.bi_size && !is_flush) { + if (req_op(req) == REQ_OP_ZONE_APPEND) + bio->bi_iter.bi_sector = req->__sector; + bio_endio(bio); + } total_bytes += bio_bytes; nr_bytes -= bio_bytes; -- 2.44.0