From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2ED31C07E9D for ; Sat, 24 Sep 2022 01:00:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231253AbiIXA76 (ORCPT ); Fri, 23 Sep 2022 20:59:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229495AbiIXA75 (ORCPT ); Fri, 23 Sep 2022 20:59:57 -0400 Received: from esa2.hgst.iphmx.com (esa2.hgst.iphmx.com [68.232.143.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09DD02ED48 for ; Fri, 23 Sep 2022 17:59:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1663981194; x=1695517194; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=H7T05O47XpwCtajDZM5zYG5x8fGP2AaZXpX62UZ1rNk=; b=i1TaqOOk9zEuM54pvA7LhdfnIomT2UETeuwPHqllaZOZlIlhBh85wAU+ GLEYWoyxuoWjFmhv5h4PUrc2v7+CypOBNm7HlwOnEgLv+bmXA+Z1OjH6c 1h+a53Nutu03Gr/Q9GbNPUOkbs5wSXIp/ck1ScsWo5GiyvTpo9pvQpqSH S+B65PWxiEns/7/AkucCroet3xMUaOUZqMgm+5pMgKOAmBE4tfHd4qXWY 4a2RnWygPviYQsKjuKJ3fV2BRjEDhhoMJm5J63nJhBLxJF8+XD0qcs6IH uY/R0fdc8Tn/8OSLtGNi5uwK7bQiF1aRiczgej9vFY2LNJqaG6DCyqBw5 w==; X-IronPort-AV: E=Sophos;i="5.93,340,1654531200"; d="scan'208";a="316437420" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 24 Sep 2022 08:59:53 +0800 IronPort-SDR: dRBCB5eOH7VAz/cSAybuNC/kExPhRhYLSwaFiF9B/77tjeWneFXhwY0ARei48eLygGbl2RKl0c jS8wpzlHuYS/BCExpicLHCeqNTKB3LnS/ddT4sitkBvWn42w2JNhkL/JZF9RPYoLG5gMAiSF8b ehtJcW2Cso37d2Q7Dn1lbCzntC6I+q9zx7kiR8/oriP842CIblZNiRt9dycs1lCX5yJ4RXndXB lCPwL2KO0MAMv7sooamXtNupKVw8jkVUIQVYQryfVAdEQ5jHhev4P4PpJhw91tG3KjW0Z9n3qF tYSZ4BqGclrllArQenNbtAnx Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 23 Sep 2022 17:14:24 -0700 IronPort-SDR: c0KzVvjyB0xREcXEMK9soCqyGBhCj+3iBw9NtgAtjadFX/pPRbboNa1o6Kfm+wipY5ps3mog0e kD9+3818EkwVxhJ394cX7dvOUIv/j09ICKPNtyR/zfTDDmm7dOfzta5KMlbv1uXjKnCBlFKuI1 GpH9+Yv3GBOAPpOoHrAqmZUSedRjSd9KwGk1QCEAmctbnjnoi0CJiHmoX3NhyTZRp6LwwINXnj sRbsBHZ+WgEOL0+FI6jj5/br0bglBRYBwC5CVIPEBvmXB4wfBTNWmmLKHU2Q6riZuafbFzWCSV QsY= WDCIronportException: Internal Received: from usg-ed-osssrv.wdc.com ([10.3.10.180]) by uls-op-cesaip02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 23 Sep 2022 17:59:54 -0700 Received: from usg-ed-osssrv.wdc.com (usg-ed-osssrv.wdc.com [127.0.0.1]) by usg-ed-osssrv.wdc.com (Postfix) with ESMTP id 4MZ9fK0Ct1z1Rwrq for ; Fri, 23 Sep 2022 17:59:52 -0700 (PDT) Authentication-Results: usg-ed-osssrv.wdc.com (amavisd-new); dkim=pass reason="pass (just generated, assumed good)" header.d=opensource.wdc.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= opensource.wdc.com; h=content-transfer-encoding:content-type :in-reply-to:organization:from:references:to:content-language :subject:user-agent:mime-version:date:message-id; s=dkim; t= 1663981192; x=1666573193; bh=H7T05O47XpwCtajDZM5zYG5x8fGP2AaZXpX 62UZ1rNk=; b=hpOSGwBu3+pGwvslvSOU82CjqOVm+828jdvQKOAZABYd3fzCvQY ouLJ/WGwyxeIeFP0C8Rput+7taAjNJlRPtjcwC3puYtjgrU1Rxq4mYbrQw64z2cq gdvj9h7eBYnkc+2JfgjyEs2yrljdBEOveeH0xBmS5TSRqhoHRj0wwSpqxcMWKYYH CgynmOIrOQyHTFQOrPjk1Wc1sF+s6N2jZNCEla7W6jcmQC9H72pmIJupLqEaI5gb bzYRyBXVOdmqXfUZWEbY+iu2YsSIRvD2RpAvQCUO/drepsrNZMK+boSyBHjKv3Bv bUX3/70Ag45jGKnIx/GffRovaLeHs7U5g3w== X-Virus-Scanned: amavisd-new at usg-ed-osssrv.wdc.com Received: from usg-ed-osssrv.wdc.com ([127.0.0.1]) by usg-ed-osssrv.wdc.com (usg-ed-osssrv.wdc.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id T1GYRHmHByKw for ; Fri, 23 Sep 2022 17:59:52 -0700 (PDT) Received: from [10.225.163.88] (unknown [10.225.163.88]) by usg-ed-osssrv.wdc.com (Postfix) with ESMTPSA id 4MZ9fH0Vlsz1RvLy; Fri, 23 Sep 2022 17:59:50 -0700 (PDT) Message-ID: Date: Sat, 24 Sep 2022 09:59:49 +0900 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 Subject: Re: [PATCH 1/5] block: enable batched allocation for blk_mq_alloc_request() Content-Language: en-US To: Jens Axboe , Pankaj Raghav Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, linux-nvme@lists.infradead.org, joshi.k@samsung.com, Pankaj Raghav , Bart Van Assche References: <20220922182805.96173-1-axboe@kernel.dk> <20220922182805.96173-2-axboe@kernel.dk> <20220923145236.pr7ssckko4okklo2@quentin> <2e484ccb-b65b-2991-e259-d3f7be6ad1a6@kernel.dk> From: Damien Le Moal Organization: Western Digital Research In-Reply-To: <2e484ccb-b65b-2991-e259-d3f7be6ad1a6@kernel.dk> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 9/24/22 05:54, Jens Axboe wrote: > On 9/23/22 9:13 AM, Pankaj Raghav wrote: >> On 2022-09-23 16:52, Pankaj Raghav wrote: >>> On Thu, Sep 22, 2022 at 12:28:01PM -0600, Jens Axboe wrote: >>>> The filesystem IO path can take advantage of allocating batches of >>>> requests, if the underlying submitter tells the block layer about it >>>> through the blk_plug. For passthrough IO, the exported API is the >>>> blk_mq_alloc_request() helper, and that one does not allow for >>>> request caching. >>>> >>>> Wire up request caching for blk_mq_alloc_request(), which is generally >>>> done without having a bio available upfront. >>>> >>>> Signed-off-by: Jens Axboe >>>> --- >>>> block/blk-mq.c | 80 ++++++++++++++++++++++++++++++++++++++++++++------ >>>> 1 file changed, 71 insertions(+), 9 deletions(-) >>>> >>> I think we need this patch to ensure correct behaviour for passthrough: >>> >>> diff --git a/block/blk-mq.c b/block/blk-mq.c >>> index c11949d66163..840541c1ab40 100644 >>> --- a/block/blk-mq.c >>> +++ b/block/blk-mq.c >>> @@ -1213,7 +1213,7 @@ void blk_execute_rq_nowait(struct request *rq, bool at_head) >>> WARN_ON(!blk_rq_is_passthrough(rq)); >>> >>> blk_account_io_start(rq); >>> - if (current->plug) >>> + if (blk_mq_plug(rq->bio)) >>> blk_add_rq_to_plug(current->plug, rq); >>> else >>> blk_mq_sched_insert_request(rq, at_head, true, false); >>> >>> As the passthrough path can now support request caching via blk_mq_alloc_request(), >>> and it uses blk_execute_rq_nowait(), bad things can happen at least for zoned >>> devices: >>> >>> static inline struct blk_plug *blk_mq_plug( struct bio *bio) >>> { >>> /* Zoned block device write operation case: do not plug the BIO */ >>> if (bdev_is_zoned(bio->bi_bdev) && op_is_write(bio_op(bio))) >>> return NULL; >>> .. >> >> Thinking more about it, even this will not fix it because op is >> REQ_OP_DRV_OUT if it is a NVMe write for passthrough requests. >> >> @Damien Should the condition in blk_mq_plug() be changed to: >> >> static inline struct blk_plug *blk_mq_plug( struct bio *bio) >> { >> /* Zoned block device write operation case: do not plug the BIO */ >> if (bdev_is_zoned(bio->bi_bdev) && !op_is_read(bio_op(bio))) >> return NULL; > > That looks reasonable to me. It'll prevent plug optimizations even > for passthrough on zoned devices, but that's probably fine. Could do: if (blk_op_is_passthrough(bio_op(bio)) || (bdev_is_zoned(bio->bi_bdev) && op_is_write(bio_op(bio)))) return NULL; Which I think is way cleaner. No ? Unless you want to preserve plugging with passthrough commands on regular (not zoned) drives ? -- Damien Le Moal Western Digital Research