From mboxrd@z Thu Jan 1 00:00:00 1970 From: Takuya Yoshikawa Subject: Re: [PATCH 0/4] KVM: Dirty logging optimization using rmap Date: Sat, 3 Dec 2011 13:37:47 +0900 Message-ID: <20111203133747.7994cd7a0a69aebe7a33e9f2@gmail.com> References: <20111114182041.43570cdf.yoshikawa.takuya@oss.ntt.co.jp> <4EC0EC90.1090202@redhat.com> <4EC0F3D3.9090907@oss.ntt.co.jp> <4EC10BFE.7050704@redhat.com> <4EC33C0B.1060807@oss.ntt.co.jp> <4EC37D18.4010609@redhat.com> <4ED4AF43.2040003@linux.vnet.ibm.com> <4ED4B574.8090907@oss.ntt.co.jp> <4ED4BFEB.5010600@redhat.com> <4ED4C85A.5020509@linux.vnet.ibm.com> <4ED4C9A3.50504@redhat.com> <4ED4E626.5010507@redhat.com> <4ED5B8F2.2090804@oss.ntt.co.jp> <4ED5BC03.5000901@oss.ntt.co.jp> <4ED79AAE.8000201@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: KVM , quintela@redhat.com, Marcelo Tosatti , qemu-devel@nongnu.org, Xiao Guangrong , Takuya Yoshikawa To: Avi Kivity Return-path: In-Reply-To: <4ED79AAE.8000201@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org Avi Kivity wrote: > That's true. But some applications do require low latency, and the > current code can impose a lot of time with the mmu spinlock held. > > The total amount of work actually increases slightly, from O(N) to O(N > log N), but since the tree is so wide, the overhead is small. > Controlling the latency can be achieved by making the user space limit the number of dirty pages to scan without hacking the core mmu code. The fact that we cannot transfer so many pages on the network at once suggests this is reasonable. With the rmap write protection method in KVM, the only thing we need is a new GET_DIRTY_LOG api which takes the [gfn_start, gfn_end] to scan, or max_write_protections optionally. I remember that someone suggested splitting the slot at KVM forum. Same effect with less effort. QEMU can also avoid unwanted page faults by using this api wisely. E.g. you can use this for "Interactivity improvements" TODO on KVM wiki, I think. Furthermore, QEMU may be able to use multiple threads for the memory copy task. Each thread has its own range of memory to copy, and does GET_DIRTY_LOG independently. This will make things easy to add further optimizations in QEMU. In summary, my impression is that the main cause of the current latency problem is not the write protection of KVM but the strategy which tries to cook the large slot in one hand. What do you think? Takuya From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:48895) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RWhMI-0000XQ-Sp for qemu-devel@nongnu.org; Fri, 02 Dec 2011 23:37:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RWhMH-00089N-QG for qemu-devel@nongnu.org; Fri, 02 Dec 2011 23:37:54 -0500 Received: from mail-gx0-f173.google.com ([209.85.161.173]:43490) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RWhMH-00089G-My for qemu-devel@nongnu.org; Fri, 02 Dec 2011 23:37:53 -0500 Received: by ggnk1 with SMTP id k1so2462850ggn.4 for ; Fri, 02 Dec 2011 20:37:52 -0800 (PST) Date: Sat, 3 Dec 2011 13:37:47 +0900 From: Takuya Yoshikawa Message-Id: <20111203133747.7994cd7a0a69aebe7a33e9f2@gmail.com> In-Reply-To: <4ED79AAE.8000201@redhat.com> References: <20111114182041.43570cdf.yoshikawa.takuya@oss.ntt.co.jp> <4EC0EC90.1090202@redhat.com> <4EC0F3D3.9090907@oss.ntt.co.jp> <4EC10BFE.7050704@redhat.com> <4EC33C0B.1060807@oss.ntt.co.jp> <4EC37D18.4010609@redhat.com> <4ED4AF43.2040003@linux.vnet.ibm.com> <4ED4B574.8090907@oss.ntt.co.jp> <4ED4BFEB.5010600@redhat.com> <4ED4C85A.5020509@linux.vnet.ibm.com> <4ED4C9A3.50504@redhat.com> <4ED4E626.5010507@redhat.com> <4ED5B8F2.2090804@oss.ntt.co.jp> <4ED5BC03.5000901@oss.ntt.co.jp> <4ED79AAE.8000201@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH 0/4] KVM: Dirty logging optimization using rmap List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: KVM , quintela@redhat.com, Marcelo Tosatti , qemu-devel@nongnu.org, Xiao Guangrong , Takuya Yoshikawa Avi Kivity wrote: > That's true. But some applications do require low latency, and the > current code can impose a lot of time with the mmu spinlock held. > > The total amount of work actually increases slightly, from O(N) to O(N > log N), but since the tree is so wide, the overhead is small. > Controlling the latency can be achieved by making the user space limit the number of dirty pages to scan without hacking the core mmu code. The fact that we cannot transfer so many pages on the network at once suggests this is reasonable. With the rmap write protection method in KVM, the only thing we need is a new GET_DIRTY_LOG api which takes the [gfn_start, gfn_end] to scan, or max_write_protections optionally. I remember that someone suggested splitting the slot at KVM forum. Same effect with less effort. QEMU can also avoid unwanted page faults by using this api wisely. E.g. you can use this for "Interactivity improvements" TODO on KVM wiki, I think. Furthermore, QEMU may be able to use multiple threads for the memory copy task. Each thread has its own range of memory to copy, and does GET_DIRTY_LOG independently. This will make things easy to add further optimizations in QEMU. In summary, my impression is that the main cause of the current latency problem is not the write protection of KVM but the strategy which tries to cook the large slot in one hand. What do you think? Takuya