From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Paul E. McKenney" Subject: Re: qemu-kvm.git build problem Date: Tue, 12 Jan 2010 05:50:18 -0800 Message-ID: <20100112135018.GE6807@linux.vnet.ibm.com> References: <1262025646.2774.16.camel@localhost.localdomain> <1263014581.2439.207.camel@localhost.localdomain> <4B4AF9D2.3080100@siemens.com> <4B4AFAD9.3010604@redhat.com> <4B4AFBC7.1020801@siemens.com> <4B4BC0F3.2010507@web.de> <20100112005118.GO6632@linux.vnet.ibm.com> <4B4C329F.4010301@web.de> Reply-To: paulmck@linux.vnet.ibm.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Avi Kivity , Lucas Meneghel Rodrigues , KVM mailing list , Michael Goldish , Eduardo Habkost , Dor Laor To: Jan Kiszka Return-path: Received: from e6.ny.us.ibm.com ([32.97.182.146]:35284 "EHLO e6.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751651Ab0ALOws (ORCPT ); Tue, 12 Jan 2010 09:52:48 -0500 Received: from d01relay03.pok.ibm.com (d01relay03.pok.ibm.com [9.56.227.235]) by e6.ny.us.ibm.com (8.14.3/8.13.1) with ESMTP id o0CEmonO015595 for ; Tue, 12 Jan 2010 09:48:50 -0500 Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay03.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o0CEqloE140508 for ; Tue, 12 Jan 2010 09:52:47 -0500 Received: from d01av04.pok.ibm.com (loopback [127.0.0.1]) by d01av04.pok.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o0CEqld4013286 for ; Tue, 12 Jan 2010 09:52:47 -0500 Content-Disposition: inline In-Reply-To: <4B4C329F.4010301@web.de> Sender: kvm-owner@vger.kernel.org List-ID: On Tue, Jan 12, 2010 at 09:28:15AM +0100, Jan Kiszka wrote: > Paul E. McKenney wrote: > > On Tue, Jan 12, 2010 at 01:23:15AM +0100, Jan Kiszka wrote: > >> Jan Kiszka wrote: > >>> Avi Kivity wrote: > >>>> On 01/11/2010 12:13 PM, Jan Kiszka wrote: > >>>>> BTW, does anybody know how to back-port synchronize_srcu_expedited best? > >>>>> It looked like a simple mapping to synchronize_srcu was not sufficient > >>>>> to achieve the same performance as with the pre-srcu locking (e.g. > >>>>> guest&host stalled during guest's framebuffer setup). > >>>>> > >>>> Isn't it sufficient to backport kernel/srcu.c? I thought no sched.c > >>>> changes were necessary. > >>> Haven't looked yet, but if that's the case, it would indeed be > >>> straightforward. > >> It's far away from being straightforward: synchronize_rcu_expedited is > >> based on synchronize_sched_expedited, introduced to 2.6.32. But that > >> services is hooked deep into the scheduler, fiddling directly with > >> runqueues (which are completely private to sched.c). This path looks > >> like a dead end, specifically when its about supporting ~8 major Linux > >> releases backwards. > >> > >> Paul, we have a problem here on the KVM-for-older-kernels front: We need > >> synchronize_rcu_expedited for acceptable write-side performance (there > >> are certain phases with lots of changes, plain synchronize_rcu just > >> stalls both guest and host for several seconds). Our target kernels > >> (down to 2.6.27, unofficially even 2.6.24) do not have the expedited > >> service. Can you think of a poor man's solution for those kernels? > >> > >> Unfortunately, I don't think there is mechanical patching possible to > >> role-back our srcu use to a rw-sem. But I will check this once again > >> tomorrow. > > > > Would it help if I backported the RCU expedited stuff? Yes, it does > > involve kernel/sched.c, but it is reasonably hands-off. > > > > If this would help, please let me know to which kernel version you need > > the backport. > > Thanks a lot. Mmm, it's probably already sufficient when you tell me if > I'm now on the right tracks: > > The idea behind synchronize_sched_expedited is, instead of waiting for > random task switches on all CPUs, to enforce them. It now just reuses > the migration_thread, locks and request queues from the rqs for this > purpose. So it should have the same effect if we create our own > high-prio srcu_expedited kthreads that include the srcu logic which just > complete() the queued requests of some compat > kvm_synchronize_sched_expedited. Did I get it? Yep. There are any number of ways to force context switches on all CPUs, and pretty much any will work for synchronize_sched_expedited(). Once you have something that does synchronize_sched_expedited(), synchronize_srcu_expedited() can just invoke it. > If so, I will try to write something like this the next days. Will > surely appreciate your review afterwards! Sounds good! Thanx, Paul