From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80FEDECAAD3 for ; Mon, 19 Sep 2022 14:36:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230040AbiISOgE (ORCPT ); Mon, 19 Sep 2022 10:36:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229540AbiISOgB (ORCPT ); Mon, 19 Sep 2022 10:36:01 -0400 Received: from verein.lst.de (verein.lst.de [213.95.11.211]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47F64B85A for ; Mon, 19 Sep 2022 07:36:00 -0700 (PDT) Received: by verein.lst.de (Postfix, from userid 2407) id 44D1568BEB; Mon, 19 Sep 2022 16:35:56 +0200 (CEST) Date: Mon, 19 Sep 2022 16:35:56 +0200 From: Christoph Hellwig To: Jens Axboe Cc: Liu Song , kbusch@kernel.org, hch@lst.de, sagi@grimberg.me, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH] nvme: request remote is usually not involved for nvme devices Message-ID: <20220919143556.GA28122@lst.de> References: <1663432858-99743-1-git-send-email-liusong@linux.alibaba.com> <7b28925a-cbee-620f-fde7-d16f256836cc@linux.alibaba.com> <894e18a4-4504-df48-6429-a04c222ca064@kernel.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <894e18a4-4504-df48-6429-a04c222ca064@kernel.dk> User-Agent: Mutt/1.5.17 (2007-11-01) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Sep 19, 2022 at 08:10:31AM -0600, Jens Axboe wrote: > I'm not disagreeing with any of that, my point is just that you're > hacking around this in the nvme driver. This is problematic whenever > core changes happen, because now we have to touch individual drivers. > While the expectation is that there are no remote IPI completions for > NVMe, queue starved devices do exist and those do see remote > completions. > > This optimization belongs in the blk-mq core, not in nvme. I do think it > makes sense, you just need to solve it in blk-mq rather than in the nvme > driver. I'd also really see solid numbers to justify it. And btw, having more than one core per queue is quite common in nvme. Even many enterprise SSDs only have 64 queues, and some of the low-end consumers ones have very low number of queues that are not enough for the number of cores in even mid-end desktop CPUs.