From: Dor Laor <dor.laor@gmail.com>
To: Amit Shah <amit.shah@qumranet.com>
Cc: kvm-devel@lists.sourceforge.net, Avi Kivity <avi@qumranet.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
Avi Kivity <avi@argo.co.il>, Gerd Hoffmann <kraxel@redhat.com>
Subject: Re: [kvm-devel] Guest kernel hangs in smp kvm for older kernelsprior to tsc sync cleanup
Date: Wed, 19 Dec 2007 19:04:53 +0200 [thread overview]
Message-ID: <47694F35.6070401@qumranet.com> (raw)
In-Reply-To: <200712192225.53748.amit.shah@qumranet.com>
Amit Shah wrote:
>
> On Wednesday 19 December 2007 21:02:06 Glauber de Oliveira Costa wrote:
> > On Dec 19, 2007 12:27 PM, Avi Kivity <avi@argo.co.il> wrote:
> > > Ingo Molnar wrote:
> > > > * Avi Kivity <avi@qumranet.com> wrote:
> > > >> Avi Kivity wrote:
> > > >>> Testing shows wrmsr and rdtsc function normally.
> > > >>>
> > > >>> I'll try pinning the vcpus to cpus and see if that helps.
> > > >>
> > > >> It does.
> > > >
> > > > do we let the guest read the physical CPU's TSC? That would be
> trouble.
> > >
> > > vmx (and svm) allow us to add an offset to the physical tsc. We
> set it
> > > on startup to -tsc (so that an rdtsc on boot would return 0), and
> > > massage it on vcpu migration so that guest rdtsc is monotonic.
> > >
> > > The net effect is that tsc on a vcpu can experience large forward
> jumps
> > > and changes in rate, but no negative jumps.
> >
> > Changes in rate does not sound good. It's possibly what's screwing up
> > my paravirt clock implementation in smp.
>
> Do you mean in the case of VM migration, or just starting them on a single
> host?
>
It's the cpu preemption stuff on local host and not VM migration
>
> > Since the host updates guest time prior to putting vcpu to run, two
> > vcpus that start running at different times will have different system
> > values.
> >
> > Now if the vcpu that started running later probes the time first,
> > we'll se the time going backwards. A constant tsc rate is the only way
> > around
> > my limited mind sees around the problem (besides, obviously, _not_
> > making the system time per-vcpu).
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> kvm-devel mailing list
> kvm-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>
WARNING: multiple messages have this Message-ID (diff)
From: Dor Laor <dor.laor-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
To: Amit Shah <amit.shah-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org,
Avi Kivity <avi-7k6+44Jx4zn6gbPvEgmw2w@public.gmane.org>,
linux-kernel
<linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org>,
Gerd Hoffmann <kraxel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Subject: Re: Guest kernel hangs in smp kvm for older kernelsprior to tsc sync cleanup
Date: Wed, 19 Dec 2007 19:04:53 +0200 [thread overview]
Message-ID: <47694F35.6070401@qumranet.com> (raw)
In-Reply-To: <200712192225.53748.amit.shah-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
Amit Shah wrote:
>
> On Wednesday 19 December 2007 21:02:06 Glauber de Oliveira Costa wrote:
> > On Dec 19, 2007 12:27 PM, Avi Kivity <avi-7k6+44Jx4zn6gbPvEgmw2w@public.gmane.org> wrote:
> > > Ingo Molnar wrote:
> > > > * Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> > > >> Avi Kivity wrote:
> > > >>> Testing shows wrmsr and rdtsc function normally.
> > > >>>
> > > >>> I'll try pinning the vcpus to cpus and see if that helps.
> > > >>
> > > >> It does.
> > > >
> > > > do we let the guest read the physical CPU's TSC? That would be
> trouble.
> > >
> > > vmx (and svm) allow us to add an offset to the physical tsc. We
> set it
> > > on startup to -tsc (so that an rdtsc on boot would return 0), and
> > > massage it on vcpu migration so that guest rdtsc is monotonic.
> > >
> > > The net effect is that tsc on a vcpu can experience large forward
> jumps
> > > and changes in rate, but no negative jumps.
> >
> > Changes in rate does not sound good. It's possibly what's screwing up
> > my paravirt clock implementation in smp.
>
> Do you mean in the case of VM migration, or just starting them on a single
> host?
>
It's the cpu preemption stuff on local host and not VM migration
>
> > Since the host updates guest time prior to putting vcpu to run, two
> > vcpus that start running at different times will have different system
> > values.
> >
> > Now if the vcpu that started running later probes the time first,
> > we'll se the time going backwards. A constant tsc rate is the only way
> > around
> > my limited mind sees around the problem (besides, obviously, _not_
> > making the system time per-vcpu).
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>
-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
next prev parent reply other threads:[~2007-12-19 17:05 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-12-18 17:20 Guest kernel hangs in smp kvm for older kernels prior to tsc sync cleanup Avi Kivity
2007-12-18 17:20 ` Avi Kivity
2007-12-18 22:19 ` Ingo Molnar
2007-12-18 22:19 ` Ingo Molnar
2007-12-19 6:33 ` Avi Kivity
2007-12-19 6:33 ` Avi Kivity
2007-12-19 11:19 ` [kvm-devel] " Avi Kivity
2007-12-19 11:19 ` Avi Kivity
2007-12-19 11:39 ` [kvm-devel] " Avi Kivity
2007-12-19 14:06 ` Ingo Molnar
2007-12-19 14:06 ` Ingo Molnar
2007-12-19 14:27 ` [kvm-devel] " Avi Kivity
2007-12-19 15:32 ` Glauber de Oliveira Costa
2007-12-19 15:32 ` Glauber de Oliveira Costa
2007-12-19 15:41 ` [kvm-devel] " Avi Kivity
2007-12-20 10:34 ` Glauber de Oliveira Costa
2007-12-20 10:34 ` Glauber de Oliveira Costa
2007-12-19 16:55 ` [kvm-devel] " Amit Shah
2007-12-19 16:55 ` Amit Shah
2007-12-19 17:04 ` Dor Laor [this message]
2007-12-19 17:04 ` Guest kernel hangs in smp kvm for older kernelsprior " Dor Laor
2007-12-19 17:09 ` [kvm-devel] " Avi Kivity
2007-12-19 17:09 ` Avi Kivity
2007-12-19 14:53 ` [kvm-devel] Guest kernel hangs in smp kvm for older kernels prior " Avi Kivity
2007-12-19 14:53 ` Avi Kivity
2007-12-19 15:09 ` [kvm-devel] " Ingo Molnar
2007-12-19 15:21 ` Avi Kivity
2007-12-19 15:21 ` Avi Kivity
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=47694F35.6070401@qumranet.com \
--to=dor.laor@gmail.com \
--cc=amit.shah@qumranet.com \
--cc=avi@argo.co.il \
--cc=avi@qumranet.com \
--cc=dor.laor@qumranet.com \
--cc=kraxel@redhat.com \
--cc=kvm-devel@lists.sourceforge.net \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.