From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michal Novotny Subject: Re: xen_sched_clock problem in RHEL6b2, Was: [PATCH] x86: unconditionally mark TSC unstable under Xen Date: Mon, 19 Jul 2010 08:00:12 +0200 Message-ID: <4C43E9EC.9030007@redhat.com> References: <002b7702-f446-427f-8048-27d4edd7a51e@default> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <002b7702-f446-427f-8048-27d4edd7a51e@default> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dan Magenheimer Cc: Jeremy Fitzhardinge , xen-devel@lists.xensource.com, Andrew Jones , Jed Smith , Jan Beulich List-Id: xen-devel@lists.xenproject.org Hi Dan, since it looks like this is the kernel/hypervisor bug I'm not really the right one to ask. You should better as Andrew Jones - CCed now (his e-mail is drjones@redhat.com just for sure) - since he's working in the kernel-xen area and therefore even the RHEL-6 kernel so I guess he's a good guy to ask about this one. Also, according to the links it seems you're running RHEL-6 beta 2 as PV guest, right ? Are you running this on Xen-4.1-unstable or what version? Andrew, could you please have a look this one? I don't know whether it's RHEL-6 kernel bug itself or whether there's some bug in the hypervisor, nevetheless both parts belongs to the same component - kernel-xen - you're working on so I guess you're a good guy to ask. Please, leave me in the CC list as I'm interested as well. Thanks, Michal On 07/17/2010 12:32 AM, Dan Magenheimer wrote: > Hi Michael -- > > Not sure if you are the right person for this, but FYI > it appears that this bug is also in RHEL6 beta 2 and > might account for some weird issues I've seen running > RHEL6 betas as Xen guests. > > Problem description: > http://lists.xensource.com/archives/html/xen-devel/2010-07/msg00241.html > Jeremy's solution (forwarded below): > http://lists.xensource.com/archives/html/xen-devel/2010-07/msg00749.html > > Dan > > >> -----Original Message----- >> From: Jeremy Fitzhardinge [mailto:jeremy@goop.org] >> Sent: Wednesday, July 14, 2010 3:36 PM >> To: Jed Smith >> Cc: xen-devel@lists.xensource.com; Jan Beulich >> Subject: [Xen-devel] Re: [PATCH] x86: unconditionally mark TSC unstable >> under Xen >> >> On 07/14/2010 12:24 PM, Jed Smith wrote: >> >>> Jeremy, Jan - what do you think? Is this a bad move? I feel like >>> >> there >> >>> is a consequence to this that I am unaware of, but it fixes my issue. >>> >>> >> Ah, well that's interesting. >> >> There's a couple of comments: >> >> 1. you can't do this with just a compile-time test, since the same >> kernel can also boot native >> 2. nothing in a Xen PV domU environment should be using the tsc >> directly, so this shouldn't have an effect. If something is >> using >> the tsc we should track it down. >> >> I wonder, however, if you're getting the same result as Jan's >> suggestion >> of making sched_clock unstable by making the tsc unstable. >> >> In that case, this patch may help: >> >> Subject: [PATCH] xen: disable xen_sched_clock by default >> >> xen_sched_clock only counts unstolen time. In principle this should >> be useful to the Linux scheduler so that it knows how much time a >> process >> actually consumed. But in practice this doesn't work very well as the >> scheduler expects the sched_clock time to be synchronized between >> cpus. It also uses sched_clock to measure the time a task spends >> sleeping, in which case "unstolen time" isn't meaningful. >> >> So just use plain xen_clocksource_read to return wallclock nanoseconds >> for sched_clock. >> >> Signed-off-by: Jeremy Fitzhardinge >> >> diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig >> index b83e119..6a09f01 100644 >> --- a/arch/x86/xen/Kconfig >> +++ b/arch/x86/xen/Kconfig >> @@ -29,6 +29,10 @@ config XEN_SAVE_RESTORE >> depends on XEN&& PM >> default y >> >> +config XEN_SCHED_CLOCK >> + bool >> + default n >> + >> config XEN_DEBUG_FS >> bool "Enable Xen debug and tuning parameters in debugfs" >> depends on XEN&& DEBUG_FS >> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c >> index 4b57c0b..242a230 100644 >> --- a/arch/x86/xen/enlighten.c >> +++ b/arch/x86/xen/enlighten.c >> @@ -939,7 +939,11 @@ static const struct pv_time_ops xen_time_ops >> __initdata = { >> .set_wallclock = xen_set_wallclock, >> .get_wallclock = xen_get_wallclock, >> .get_tsc_khz = xen_tsc_khz, >> +#ifdef CONFIG_XEN_SCHED_CLOCK >> .sched_clock = xen_sched_clock, >> +#else >> + .sched_clock = xen_clocksource_read, >> +#endif >> }; >> >> static const struct pv_cpu_ops xen_cpu_ops __initdata = { >> diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c >> index 9d1f853..32dc3dc 100644 >> --- a/arch/x86/xen/time.c >> +++ b/arch/x86/xen/time.c >> @@ -154,6 +154,7 @@ static void do_stolen_accounting(void) >> account_idle_ticks(ticks); >> } >> >> +#ifdef CONFIG_XEN_SCHED_CLOCK >> /* >> * Xen sched_clock implementation. Returns the number of unstolen >> * nanoseconds, which is nanoseconds the VCPU spent in RUNNING+BLOCKED >> @@ -191,7 +192,7 @@ unsigned long long xen_sched_clock(void) >> >> return ret; >> } >> - >> +#endif >> >> /* Get the TSC speed from Xen */ >> unsigned long xen_tsc_khz(void) >> >> Thanks, >> J >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> -- Michal Novotny, RHCE Virtualization Team (xen userspace), Red Hat