From mboxrd@z Thu Jan 1 00:00:00 1970 From: Xiao Guangrong Subject: Re: dirty page tracking in kvm/qemu -- page faults inevitable? Date: Wed, 30 Jul 2014 14:09:26 +0800 Message-ID: <53D88C16.5090006@linux.vnet.ibm.com> References: <53D81C40.9020800@mail.usask.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit To: Chris Friesen , avi@redhat.com, mtosatti@redhat.com, kvm@vger.kernel.org Return-path: Received: from e28smtp02.in.ibm.com ([122.248.162.2]:53216 "EHLO e28smtp02.in.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754603AbaG3GJj (ORCPT ); Wed, 30 Jul 2014 02:09:39 -0400 Received: from /spool/local by e28smtp02.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 30 Jul 2014 11:39:36 +0530 Received: from d28relay05.in.ibm.com (d28relay05.in.ibm.com [9.184.220.62]) by d28dlp02.in.ibm.com (Postfix) with ESMTP id DD17A394005E for ; Wed, 30 Jul 2014 11:39:31 +0530 (IST) Received: from d28av05.in.ibm.com (d28av05.in.ibm.com [9.184.220.67]) by d28relay05.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s6U69qhq59834538 for ; Wed, 30 Jul 2014 11:39:53 +0530 Received: from d28av05.in.ibm.com (localhost [127.0.0.1]) by d28av05.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s6U69U4a004798 for ; Wed, 30 Jul 2014 11:39:30 +0530 In-Reply-To: <53D81C40.9020800@mail.usask.ca> Sender: kvm-owner@vger.kernel.org List-ID: On 07/30/2014 06:12 AM, Chris Friesen wrote: > Hi, > > I've got an issue where we're hitting major performance penalties while doing live migration, and it seems like it might > be due to page faults triggering hypervisor exits, and then we get stuck waiting for the iothread lock which is held by > the qemu dirty page scanning code. I am afraid that using dirty-bit instead of write-protection may cause the case even more worse for iothread-lock because we need to walk whole sptes to get dirty-set pages, however currently we only need to walk the page set in the bitmap. > > Accordingly, I'm trying to figure out the actual mechanism whereby dirty pages are tracked in qemu/kvm. I've got an Ivy Bridge CPU, a 3.4 kernel on the host, and qemu 1.4. > > Looking at the qemu code, it seems to be calling down into kvm to get the dirty page information. > > Looking at kvm, most of what I read seems to be doing the usual "mark it read-only and then when we take a page fault mark it as dirty" trick. > > However, I read something about Intel EPT having hardware support for tracking dirty pages. It seems like this might avoid the need for a page fault, but might only be available on Haswell or later CPUs--is that correct? Is it supported in kvm? If so, when was support added? Actually, i have implemented the prototype long time ago, maybe it's the time to benchmark it and post it out.