From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keith Busch Subject: Re: [PATCH] nvme: Acknowledge completion queue on each iteration Date: Tue, 18 Jul 2017 17:26:32 -0400 Message-ID: <20170718212631.GB1202@localhost.localdomain> References: <1500330983-27501-1-git-send-email-okaya@codeaurora.org> <20170717224551.GA1496@localhost.localdomain> <6d10032c-35ec-978c-6b8f-1ab9c07adf7f@codeaurora.org> <20170717225615.GB1496@localhost.localdomain> <79413407294645f0e1252112c3435a29@codeaurora.org> <20170718143617.GA7613@localhost.localdomain> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from mga01.intel.com ([192.55.52.88]:25368 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751513AbdGRVUd (ORCPT ); Tue, 18 Jul 2017 17:20:33 -0400 Content-Disposition: inline In-Reply-To: Sender: linux-arm-msm-owner@vger.kernel.org List-Id: linux-arm-msm@vger.kernel.org To: Sinan Kaya Cc: linux-nvme@lists.infradead.org, timur@codeaurora.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-kernel@vger.kernel.org On Tue, Jul 18, 2017 at 02:52:26PM -0400, Sinan Kaya wrote: > On 7/18/2017 10:36 AM, Keith Busch wrote: > > I do see that the NVMe driver is creating a completion interrupt on > each CPU core for the completions. No problems with that. > > However, I don't think you can guarantee that there will always be a single > CPU core targeting one submission queue especially with asynchronous IO. > > Lock contention counters from CONFIG_LOCK_STAT are pointing to nvmeq->lock > in my FIO tests. > > Did I miss something? I think that must mean your machine has many more CPUs than your nvme controller has IO queues.