From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: RE: rdtsc strangeness on upstream kernel Date: Mon, 19 Jul 2010 16:55:26 -0700 (PDT) Message-ID: <681683ff-677b-4709-911e-3d90d7dbed7c@default> References: <0784c7e7-1e9c-4abd-8e90-837051fe6b8b@default 4C44E123.3030308@goop.org 441eecef-5758-4c3f-924e-74823b7cb776@default> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <441eecef-5758-4c3f-924e-74823b7cb776@default> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jeremy Fitzhardinge Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org > With vcpus=3D2, timer0 goes up to about 2000 irq/sec on CPU0 > and timer1 on CPU1 is the same while rdtsc/sec is about 20000. No, wait, correct that...timer0 and timer1 are 1000 irq/sec. (I was using "watch -d" which defaults to updating every two seconds, not every second.) > Is it possible that the timer interrupt is using rdtsc > five times each interrupt? TEN times each interrupt. > -----Original Message----- > From: Dan Magenheimer > Sent: Monday, July 19, 2010 5:51 PM > To: Jeremy Fitzhardinge > Cc: xen-devel@lists.xensource.com > Subject: RE: rdtsc strangeness on upstream kernel >=20 > > > The number of rdtsc/second goes up dramatically when > > > there is CPU-intensive load on an upstream kernel! > > > > Are you looking at rdtsc emulation traps when running a PV domain? >=20 > Yes, with "for i in {0..99}; do xm debug-key s; sleep 1; done". >=20 > > > I know we both observed some cases where rdtsc/sec > > > was very high, but I don't think we ever were able to > > > reproduce this consistently. > > > > It would be interesting to compare that to the context switch rate > (cs > > column in vmstat output) to see if they correlate. >=20 > Doesn't appear to be related. Vmstat continues to show > cs at about 40/sec while the rdtsc/sec is very high. >=20 > > Also, how does it relate to timer interrupts? >=20 > With vcpus=3D2, timer0 goes up to about 2000 irq/sec on CPU0 > and timer1 on CPU1 is the same while rdtsc/sec is about 20000. > Is it possible that the timer interrupt is using rdtsc > five times each interrupt? >=20 > > > First, this is a single vcpu, 64-bit 2.6.32 (RHEL6b2) > > > kernel. > > > > > > I am observing ~300 rdtsc/sec on an idle VM. When > > > I run a load of: > > > > > > main() {while(1);} > > > > > > I am observing about 10000 rdtsc/sec!! > > > > > > This is a CONFIG_HZ_1000=3Dy kernel, so I would > > > expect 1000 rdtsc/sec, or maybe 2000 rdtsc/sec, > > > but 10000 makes me wonder if there is some hidden > > > bug. > > > > Do you have preemption running? And why HZ=3D1000? >=20 > CONFIG_PREEMPT is off, though there are some other > PREEMPT-related configs that are on. Are any others > relevant? >=20 > HZ=3D1000 is because this is a straight RHEL6b2 kernel, > but running as a PV guest. >=20 > Dan