From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5F0DC43219 for ; Thu, 25 Apr 2019 14:32:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AE1762067D for ; Thu, 25 Apr 2019 14:32:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726155AbfDYOcs (ORCPT ); Thu, 25 Apr 2019 10:32:48 -0400 Received: from mx2.suse.de ([195.135.220.15]:59546 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725900AbfDYOcs (ORCPT ); Thu, 25 Apr 2019 10:32:48 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 95BDAAEE8; Thu, 25 Apr 2019 14:32:47 +0000 (UTC) Subject: Re: [PATCH] block: use static bio_set for bio_split() calls To: Ming Lei , Sagi Grimberg Cc: Jens Axboe , Hannes Reinecke , Bart van Assche , Ming Lei , neilb@suse.com, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Christoph Hellwig , Kent Overstreet References: <20190418140632.60606-1-hare@suse.de> <20190418143429.GA19175@ming.t460p> <98d8549a-2663-b404-e38a-6f55dfb575bf@grimberg.me> <20190424221422.GA21351@ming.t460p> <20190425004106.GA22683@ming.t460p> From: Hannes Reinecke Message-ID: Date: Thu, 25 Apr 2019 16:32:42 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190425004106.GA22683@ming.t460p> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 4/25/19 2:41 AM, Ming Lei wrote: > On Thu, Apr 25, 2019 at 06:14:22AM +0800, Ming Lei wrote: >> On Wed, Apr 24, 2019 at 10:20:46AM -0700, Sagi Grimberg wrote: >>> >>>> per-queue bioset is used originally for avoiding deadlock, are you >>>> sure the static bioset is safe? >>> >>> Can you explain this? I didn't find any indication of that in the change >>> log history... >>> >>> Originally introduced by Kent: >> >> bio split can be run from stacking drivers, for example, MD over NVMe, >> if the global reserved mempool is consumed by MD bio splitting, then >> no any progress can be made when splitting on bio submitted to NVMe. >> >> Kent may have more details... > > I guess it might be fine to use one shared global bio_set for all > lowest underlying queues, could be all queues except for loop, dm, md > , drbd, bcache, ... > But wasn't the overall idea of stacking drivers that we propagate the queue limits up to the uppermost drivers, so that we have to do a split only at the upper layers? Furthermore, it's not every bio which needs to be split, only those which straddle some device limitations. The only ones not being able to propagate the queue limits is MD, and that is already using a private bio_set here. I've worked on a patchset to provide a separate bio set for each of the stacking drivers, but the interface I've been able to come up with is not very nice, and I really doubt it's worth it. So I'd advocate to test with this patch first, and only provide individual biosets once we find it's an issue. Cheers, Hannes -- Dr. Hannes Reinecke Teamlead Storage & Networking hare@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnberg) From mboxrd@z Thu Jan 1 00:00:00 1970 From: hare@suse.de (Hannes Reinecke) Date: Thu, 25 Apr 2019 16:32:42 +0200 Subject: [PATCH] block: use static bio_set for bio_split() calls In-Reply-To: <20190425004106.GA22683@ming.t460p> References: <20190418140632.60606-1-hare@suse.de> <20190418143429.GA19175@ming.t460p> <98d8549a-2663-b404-e38a-6f55dfb575bf@grimberg.me> <20190424221422.GA21351@ming.t460p> <20190425004106.GA22683@ming.t460p> Message-ID: On 4/25/19 2:41 AM, Ming Lei wrote: > On Thu, Apr 25, 2019@06:14:22AM +0800, Ming Lei wrote: >> On Wed, Apr 24, 2019@10:20:46AM -0700, Sagi Grimberg wrote: >>> >>>> per-queue bioset is used originally for avoiding deadlock, are you >>>> sure the static bioset is safe? >>> >>> Can you explain this? I didn't find any indication of that in the change >>> log history... >>> >>> Originally introduced by Kent: >> >> bio split can be run from stacking drivers, for example, MD over NVMe, >> if the global reserved mempool is consumed by MD bio splitting, then >> no any progress can be made when splitting on bio submitted to NVMe. >> >> Kent may have more details... > > I guess it might be fine to use one shared global bio_set for all > lowest underlying queues, could be all queues except for loop, dm, md > , drbd, bcache, ... > But wasn't the overall idea of stacking drivers that we propagate the queue limits up to the uppermost drivers, so that we have to do a split only at the upper layers? Furthermore, it's not every bio which needs to be split, only those which straddle some device limitations. The only ones not being able to propagate the queue limits is MD, and that is already using a private bio_set here. I've worked on a patchset to provide a separate bio set for each of the stacking drivers, but the interface I've been able to come up with is not very nice, and I really doubt it's worth it. So I'd advocate to test with this patch first, and only provide individual biosets once we find it's an issue. Cheers, Hannes -- Dr. Hannes Reinecke Teamlead Storage & Networking hare at suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg GF: Felix Imend?rffer, Mary Higgins, Sri Rasiah HRB 21284 (AG N?rnberg)