From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35658) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZzMbM-0002qw-Ew for qemu-devel@nongnu.org; Thu, 19 Nov 2015 05:38:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZzMbI-0003TD-Av for qemu-devel@nongnu.org; Thu, 19 Nov 2015 05:38:04 -0500 Received: from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d]:37764) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZzMbI-0003T6-3o for qemu-devel@nongnu.org; Thu, 19 Nov 2015 05:38:00 -0500 Received: by wmww144 with SMTP id w144so110297795wmw.0 for ; Thu, 19 Nov 2015 02:37:59 -0800 (PST) Sender: Paolo Bonzini References: <1447825624-17011-1-git-send-email-mlin@kernel.org> <1447825624-17011-3-git-send-email-mlin@kernel.org> From: Paolo Bonzini Message-ID: <564DA682.8050706@redhat.com> Date: Thu, 19 Nov 2015 11:37:54 +0100 MIME-Version: 1.0 In-Reply-To: <1447825624-17011-3-git-send-email-mlin@kernel.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH -qemu] nvme: support Google vendor extension List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Ming Lin , linux-nvme@lists.infradead.org, qemu-devel@nongnu.org Cc: fes@google.com, axboe@fb.com, Rob Nelson , virtualization@lists.linux-foundation.org, keith.busch@intel.com, tytso@mit.edu, Christoph Hellwig , Mihai Rusu On 18/11/2015 06:47, Ming Lin wrote: > @@ -726,7 +798,11 @@ static void nvme_process_db(NvmeCtrl *n, hwaddr addr, int val) > } > > start_sqs = nvme_cq_full(cq) ? 1 : 0; > - cq->head = new_head; > + /* When the mapped pointer memory area is setup, we don't rely on > + * the MMIO written values to update the head pointer. */ > + if (!cq->db_addr) { > + cq->head = new_head; > + } You are still checking if (new_head >= cq->size) { return; } above. I think this is incorrect when the extension is present, and furthermore it's the only case where val is being used. If you're not using val, you could use ioeventfd for the MMIO. An ioeventfd cuts the MMIO cost by at least 55% and up to 70%. Here are quick and dirty measurements from kvm-unit-tests's vmexit.flat benchmark, on two very different machines: Haswell-EP Ivy Bridge i7 MMIO memory write 5100 -> 2250 (55%) 7000 -> 3000 (58%) I/O port write 3800 -> 1150 (70%) 4100 -> 1800 (57%) You would need to allocate two eventfds for each qid, one for the sq and one for the cq. Also, processing the queues is now bounced to the QEMU iothread, so you can probably get rid of sq->timer and cq->timer. Paolo