From mboxrd@z Thu Jan 1 00:00:00 1970 From: Takuya Yoshikawa Date: Tue, 01 Jun 2010 12:05:38 +0000 Subject: Re: Any comments? Re: [RFC][PATCH 0/12] KVM, x86, ppc, asm-generic: Message-Id: <4C04F792.10207@oss.ntt.co.jp> List-Id: References: <4BFA2539.3030709@oss.ntt.co.jp> In-Reply-To: <4BFA2539.3030709@oss.ntt.co.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: kvm-ia64@vger.kernel.org (2010/06/01 19:55), Marcelo Tosatti wrote: >>> Sorry but I have to say that mmu_lock spin_lock problem was completely >>> out of >>> my mind. Although I looked through the code, it seems not easy to move the >>> set_bit_user to outside of spinlock section without breaking the >>> semantics of >>> its protection. >>> >>> So this may take some time to solve. >>> >>> But personally, I want to do something for x86's "vmallc() every time" >>> problem >>> even though moving dirty bitmaps to user space cannot be achieved soon. >>> >>> In that sense, do you mind if we do double buffering without moving >>> dirty bitmaps to >>> user space? >> >> So I would be happy if you give me any comments about this kind of other >> options. > > What if you pin the bitmaps? Yes, pinning bitmaps works. The small problem is that we need to hold the dirty_bitmap_pages[] array for every slot, the size of this array depends on the slot length, and of course pinning itself. In the performance point of view, having double sized vmalloc'ed area may be better. > > The alternative to that is to move mark_page_dirty(gfn) before acquision > of mmu_lock, in the page fault paths. The downside of that is a > potentially (large?) number of false positives in the dirty bitmap. > Interesting, but probably dangerous. From my experience, though this includes my personal view, removing vmalloc currently used by x86 is the most simple and effective change. So if you don't mind, I want to double the size of vmalloc'ed area for x86 without changing other parts. => if this one more bitmap is problematic, dirty logging itself would be in danger of failure: we need to have the same size in the timing of switch. Make sense? We can consider moving dirty bitmaps to user space later. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Takuya Yoshikawa Date: Tue, 01 Jun 2010 12:05:38 +0000 Subject: Re: Any comments? Re: [RFC][PATCH 0/12] KVM, x86, ppc, asm-generic: Message-Id: <4C04F792.10207@oss.ntt.co.jp> List-Id: References: <20100504215645.6448af8f.takuya.yoshikawa@gmail.com> <4BE7F6D7.3060005@redhat.com> <4BE7FB7B.5010600@oss.ntt.co.jp> <4BEBE6D0.8020000@redhat.com> <4BF1070B.4000507@oss.ntt.co.jp> <4BFA2539.3030709@oss.ntt.co.jp> <20100601105549.GA1975@amt.cnet> In-Reply-To: <20100601105549.GA1975-I4X2Mt4zSy4@public.gmane.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Marcelo Tosatti Cc: Avi Kivity , Takuya Yoshikawa , agraf-l3A5Bk7waGM@public.gmane.org, fernando-gVGce1chcLdL9jVzuh4AOg@public.gmane.org, kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kvm-ppc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kvm-ia64-u79uwXL29TY76Z2rM5mHXA@public.gmane.org (2010/06/01 19:55), Marcelo Tosatti wrote: >>> Sorry but I have to say that mmu_lock spin_lock problem was completely >>> out of >>> my mind. Although I looked through the code, it seems not easy to move the >>> set_bit_user to outside of spinlock section without breaking the >>> semantics of >>> its protection. >>> >>> So this may take some time to solve. >>> >>> But personally, I want to do something for x86's "vmallc() every time" >>> problem >>> even though moving dirty bitmaps to user space cannot be achieved soon. >>> >>> In that sense, do you mind if we do double buffering without moving >>> dirty bitmaps to >>> user space? >> >> So I would be happy if you give me any comments about this kind of other >> options. > > What if you pin the bitmaps? Yes, pinning bitmaps works. The small problem is that we need to hold the dirty_bitmap_pages[] array for every slot, the size of this array depends on the slot length, and of course pinning itself. In the performance point of view, having double sized vmalloc'ed area may be better. > > The alternative to that is to move mark_page_dirty(gfn) before acquision > of mmu_lock, in the page fault paths. The downside of that is a > potentially (large?) number of false positives in the dirty bitmap. > Interesting, but probably dangerous. From my experience, though this includes my personal view, removing vmalloc currently used by x86 is the most simple and effective change. So if you don't mind, I want to double the size of vmalloc'ed area for x86 without changing other parts. => if this one more bitmap is problematic, dirty logging itself would be in danger of failure: we need to have the same size in the timing of switch. Make sense? We can consider moving dirty bitmaps to user space later. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Takuya Yoshikawa Subject: Re: Any comments? Re: [RFC][PATCH 0/12] KVM, x86, ppc, asm-generic: moving dirty bitmaps to user space Date: Tue, 01 Jun 2010 21:05:38 +0900 Message-ID: <4C04F792.10207@oss.ntt.co.jp> References: <20100504215645.6448af8f.takuya.yoshikawa@gmail.com> <4BE7F6D7.3060005@redhat.com> <4BE7FB7B.5010600@oss.ntt.co.jp> <4BEBE6D0.8020000@redhat.com> <4BF1070B.4000507@oss.ntt.co.jp> <4BFA2539.3030709@oss.ntt.co.jp> <20100601105549.GA1975@amt.cnet> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Avi Kivity , Takuya Yoshikawa , agraf-l3A5Bk7waGM@public.gmane.org, fernando-gVGce1chcLdL9jVzuh4AOg@public.gmane.org, kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kvm-ppc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kvm-ia64-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Marcelo Tosatti Return-path: In-Reply-To: <20100601105549.GA1975-I4X2Mt4zSy4@public.gmane.org> Sender: kvm-ppc-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: kvm.vger.kernel.org (2010/06/01 19:55), Marcelo Tosatti wrote: >>> Sorry but I have to say that mmu_lock spin_lock problem was completely >>> out of >>> my mind. Although I looked through the code, it seems not easy to move the >>> set_bit_user to outside of spinlock section without breaking the >>> semantics of >>> its protection. >>> >>> So this may take some time to solve. >>> >>> But personally, I want to do something for x86's "vmallc() every time" >>> problem >>> even though moving dirty bitmaps to user space cannot be achieved soon. >>> >>> In that sense, do you mind if we do double buffering without moving >>> dirty bitmaps to >>> user space? >> >> So I would be happy if you give me any comments about this kind of other >> options. > > What if you pin the bitmaps? Yes, pinning bitmaps works. The small problem is that we need to hold the dirty_bitmap_pages[] array for every slot, the size of this array depends on the slot length, and of course pinning itself. In the performance point of view, having double sized vmalloc'ed area may be better. > > The alternative to that is to move mark_page_dirty(gfn) before acquision > of mmu_lock, in the page fault paths. The downside of that is a > potentially (large?) number of false positives in the dirty bitmap. > Interesting, but probably dangerous. From my experience, though this includes my personal view, removing vmalloc currently used by x86 is the most simple and effective change. So if you don't mind, I want to double the size of vmalloc'ed area for x86 without changing other parts. ==> if this one more bitmap is problematic, dirty logging itself would be in danger of failure: we need to have the same size in the timing of switch. Make sense? We can consider moving dirty bitmaps to user space later.