From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:48423) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UQJ7m-0000Q7-CO for qemu-devel@nongnu.org; Thu, 11 Apr 2013 11:09:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UQJ7d-0004mb-SV for qemu-devel@nongnu.org; Thu, 11 Apr 2013 11:09:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:61820) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UQJ7d-0004mN-KI for qemu-devel@nongnu.org; Thu, 11 Apr 2013 11:09:09 -0400 Message-ID: <5166D1DA.3050804@redhat.com> Date: Thu, 11 Apr 2013 17:08:10 +0200 From: Paolo Bonzini MIME-Version: 1.0 References: <1365632901-15470-1-git-send-email-mrhines@linux.vnet.ibm.com> <1365632901-15470-11-git-send-email-mrhines@linux.vnet.ibm.com> <20130411073843.GB19601@redhat.com> <51667FEE.903@redhat.com> <5166B9A9.9070904@linux.vnet.ibm.com> <5166C59A.4010904@redhat.com> <5166CF56.2060105@linux.vnet.ibm.com> In-Reply-To: <5166CF56.2060105@linux.vnet.ibm.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC PATCH RDMA support v1: 10/13] introduce new command migrate_check_for_zero List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael R. Hines" Cc: aliguori@us.ibm.com, "Michael S. Tsirkin" , qemu-devel@nongnu.org, owasserm@redhat.com, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com Il 11/04/2013 16:57, Michael R. Hines ha scritto: > We have hardware already with front side bus speeds of 13 GB/s. > > We also already have 5 GB/s RDMA hardware, and we will likely > have even faster RDMA hardware in the future. > > This analysis is not factoring into account the cycles it takes to > map the pages before they are checked for duplicate bytes, Do you mean the TLB misses? > regardless whether or not very little of the page is actually > cached on the processor. > > This analysis is also not taking into account the possibility that the > VM may be CPU-bound at the same time that QEMU is competing > to execute is_dup_page(). is_dup_page() is memory-bound, not CPU-bound. Note that is_dup_page only needs 1% of the bandwidth it scans (32 bytes for a cache line out of 4096 bytes/page). Scanning 30 GB/s only requires reading 250 MB/s from memory to the FSB. > Thus, as you mentioned, a worst-case 5 GB/s memory bandwidth > for is_dup_page() could be very easily reached given the right > conditions - and we do have many workloads both HPC and Multi-tier > which can easily cause QEMU's zero scanning performance to suffer. These are the real world scenarios that I was talking about. Do you have profiles of these, with the latest QEMU code, that show is_dup_page() to be expensive? We could try prefetching the first cache line *of the next page* before running is_dup_page. There's a lot of things to test before giving up and inventing a new API. Paolo