All of lore.kernel.org
 help / color / mirror / Atom feed
* CPU enumeration in Xen
@ 2007-08-22 10:34 Dominik Klein
  2007-08-22 12:34 ` Keir Fraser
  0 siblings, 1 reply; 9+ messages in thread
From: Dominik Klein @ 2007-08-22 10:34 UTC (permalink / raw)
  To: xen-devel

http://lists.xensource.com/archives/html/xen-users/2007-08/msg00507.html

In this thread I posted some interesting experiment results which lead 
to the assumption that Xenified Linux does not enumerate CPUs the same 
way as a non-xen-Linux does.

Can someone confirm this?

Regards
Dominik

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 10:34 CPU enumeration in Xen Dominik Klein
@ 2007-08-22 12:34 ` Keir Fraser
  2007-08-22 13:39   ` Dominik Klein
  2007-08-22 13:48   ` Daniel P. Berrange
  0 siblings, 2 replies; 9+ messages in thread
From: Keir Fraser @ 2007-08-22 12:34 UTC (permalink / raw)
  To: Dominik Klein, xen-devel

It seems a bit unlikely, since the enumeration code is from Linux. Perhaps
the enumeration order has changed in Linux?

To my knowledge the enumeration order is determined by the order we find
CPUs in the BIOS tables, and can hence vary across different systems, and
even across BIOS upgrades.

 -- Keir

On 22/8/07 11:34, "Dominik Klein" <dk@in-telegence.net> wrote:

> http://lists.xensource.com/archives/html/xen-users/2007-08/msg00507.html
> 
> In this thread I posted some interesting experiment results which lead
> to the assumption that Xenified Linux does not enumerate CPUs the same
> way as a non-xen-Linux does.
> 
> Can someone confirm this?
> 
> Regards
> Dominik
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 12:34 ` Keir Fraser
@ 2007-08-22 13:39   ` Dominik Klein
  2007-08-22 13:48   ` Daniel P. Berrange
  1 sibling, 0 replies; 9+ messages in thread
From: Dominik Klein @ 2007-08-22 13:39 UTC (permalink / raw)
  To: xen-devel

Keir Fraser schrieb:
> It seems a bit unlikely, since the enumeration code is from Linux. Perhaps
> the enumeration order has changed in Linux?

Here's what I see without Xen:

# uname -r
2.6.18.2-34-default
# grep -E "processor|physical" /proc/cpuinfo
processor       : 0
physical id     : 0
processor       : 1
physical id     : 0
processor       : 2
physical id     : 3
processor       : 3
physical id     : 3

>> http://lists.xensource.com/archives/html/xen-users/2007-08/msg00507.html

This was tested on the exact machine the output above is from. The 
described tests were made with 2.6.18 and xen 3.1.0 compiled from 
source. No redhat patches or something like that.

 From what I saw and described, it looks pretty clear that the 
numeration in Xen is different, but I cannot see the physical ID in 
/proc/cpuinfo to confirm that.

Maybe there is another way to see that?

Regards
Dominik

ps. i'll be on holiday for 3 days from now but i'll be happy to supply 
more information when i am back

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 12:34 ` Keir Fraser
  2007-08-22 13:39   ` Dominik Klein
@ 2007-08-22 13:48   ` Daniel P. Berrange
  2007-08-22 13:53     ` Keir Fraser
  1 sibling, 1 reply; 9+ messages in thread
From: Daniel P. Berrange @ 2007-08-22 13:48 UTC (permalink / raw)
  To: Keir Fraser; +Cc: xen-devel, Dominik Klein

On Wed, Aug 22, 2007 at 01:34:47PM +0100, Keir Fraser wrote:
> It seems a bit unlikely, since the enumeration code is from Linux. Perhaps
> the enumeration order has changed in Linux?
>
> To my knowledge the enumeration order is determined by the order we find
> CPUs in the BIOS tables, and can hence vary across different systems, and
> even across BIOS upgrades.

The HV may enumerate CPUs in the same way as Linux baremetal, but Dom0 sees
VCPUs not physical CPUs. So /proc/cpuinfo in Dom0 will bear little resemblance
to the same file on a bare metal machine. You'd need to compare the HV's view
of physical CPUs to the Linux bare metal view, which isn't something you can
do  by merely comparing /proc/cpuinfo AFAICT.

> On 22/8/07 11:34, "Dominik Klein" <dk@in-telegence.net> wrote:
> 
> > http://lists.xensource.com/archives/html/xen-users/2007-08/msg00507.html
> > 
> > In this thread I posted some interesting experiment results which lead
> > to the assumption that Xenified Linux does not enumerate CPUs the same
> > way as a non-xen-Linux does.
> > 
> > Can someone confirm this?

Also, the Dom0 sees no (NUMA) topology - all CPUs are considered equal, so 
Dom0 doesn't see any of the hyperthread, cores, sockets, nodes relationship
that a baremetal kernel sees - the HV of course knows about this topology
but doesn't expose it to the guest kernels when assigning them vCPUs.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 13:48   ` Daniel P. Berrange
@ 2007-08-22 13:53     ` Keir Fraser
  2007-08-22 14:04       ` Dominik Klein
  0 siblings, 1 reply; 9+ messages in thread
From: Keir Fraser @ 2007-08-22 13:53 UTC (permalink / raw)
  To: Daniel P. Berrange; +Cc: xen-devel, Dominik Klein

On 22/8/07 14:48, "Daniel P. Berrange" <berrange@redhat.com> wrote:

>> To my knowledge the enumeration order is determined by the order we find
>> CPUs in the BIOS tables, and can hence vary across different systems, and
>> even across BIOS upgrades.
> 
> The HV may enumerate CPUs in the same way as Linux baremetal, but Dom0 sees
> VCPUs not physical CPUs. So /proc/cpuinfo in Dom0 will bear little resemblance
> to the same file on a bare metal machine. You'd need to compare the HV's view
> of physical CPUs to the Linux bare metal view, which isn't something you can
> do  by merely comparing /proc/cpuinfo AFAICT.

Oh yes, I missed this stunningly obvious point. :-)

If you want to tie dom0 VCPUs 1:1 with physical CPUs then you can put
dom0_vcpus_pin on your Xen command line.

To see how Xen enumerates the CPUs: look at 'xm dmesg', or look at Xen boot
output, to see APIC IDs of processors as they are booted.

 -- Keir

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 13:53     ` Keir Fraser
@ 2007-08-22 14:04       ` Dominik Klein
  2007-08-22 14:15         ` Keir Fraser
  0 siblings, 1 reply; 9+ messages in thread
From: Dominik Klein @ 2007-08-22 14:04 UTC (permalink / raw)
  To: xen-devel

> If you want to tie dom0 VCPUs 1:1 with physical CPUs then you can put
> dom0_vcpus_pin on your Xen command line.

So then, when I do "xm vcpu-pin my-domain 0 1", it will pin VCPU0 in 
my-domain to the second HT of physical CPU 1?

That would be great as it would be straight forward.

Regards
Dominik

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 14:04       ` Dominik Klein
@ 2007-08-22 14:15         ` Keir Fraser
  2007-08-27  6:09           ` Dominik Klein
  0 siblings, 1 reply; 9+ messages in thread
From: Keir Fraser @ 2007-08-22 14:15 UTC (permalink / raw)
  To: Dominik Klein, xen-devel

On 22/8/07 15:04, "Dominik Klein" <dk@in-telegence.net> wrote:

>> If you want to tie dom0 VCPUs 1:1 with physical CPUs then you can put
>> dom0_vcpus_pin on your Xen command line.
> 
> So then, when I do "xm vcpu-pin my-domain 0 1", it will pin VCPU0 in
> my-domain to the second HT of physical CPU 1?
> 
> That would be great as it would be straight forward.

Enumeration order is not guaranteed across systems. This could be fixed by
sorting the processor info records after we've found them in the BIOS
tables.

 -- Keir

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-22 14:15         ` Keir Fraser
@ 2007-08-27  6:09           ` Dominik Klein
  2007-08-27  6:35             ` Dominik Klein
  0 siblings, 1 reply; 9+ messages in thread
From: Dominik Klein @ 2007-08-27  6:09 UTC (permalink / raw)
  To: xen-devel

>>> If you want to tie dom0 VCPUs 1:1 with physical CPUs then you can put
>>> dom0_vcpus_pin on your Xen command line.
>> So then, when I do "xm vcpu-pin my-domain 0 1", it will pin VCPU0 in
>> my-domain to the second HT of physical CPU 1?
>>
>> That would be great as it would be straight forward.
> 
> Enumeration order is not guaranteed across systems. This could be fixed by
> sorting the processor info records after we've found them in the BIOS
> tables.

Well I meant on the system I mentioned, which showed
CPU1=HT1+HT2
CPU2=HT3+HT4
without Xen.

I'll give it a try today.

Regards
Dominik

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: CPU enumeration in Xen
  2007-08-27  6:09           ` Dominik Klein
@ 2007-08-27  6:35             ` Dominik Klein
  0 siblings, 0 replies; 9+ messages in thread
From: Dominik Klein @ 2007-08-27  6:35 UTC (permalink / raw)
  To: xen-devel

Dominik Klein schrieb:
>>>> If you want to tie dom0 VCPUs 1:1 with physical CPUs then you can put
>>>> dom0_vcpus_pin on your Xen command line.
>>> So then, when I do "xm vcpu-pin my-domain 0 1", it will pin VCPU0 in
>>> my-domain to the second HT of physical CPU 1?
>>>
>>> That would be great as it would be straight forward.
>>
>> Enumeration order is not guaranteed across systems. This could be 
>> fixed by
>> sorting the processor info records after we've found them in the BIOS
>> tables.
> 
> Well I meant on the system I mentioned, which showed
> CPU1=HT1+HT2
> CPU2=HT3+HT4
> without Xen.
> 
> I'll give it a try today.

Well, it doesn't do what I would have wanted it to. It just does
something like
xm vcpu-pin 0 0 0
xm vcpu-pin 0 1 1
xm vcpu-pin 0 2 2
xm vcpu-pin 0 3 3

So is there currently a way to see the processors in the same numeration 
as in Linux without Xen?

Regards
Dominik

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2007-08-27  6:35 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-08-22 10:34 CPU enumeration in Xen Dominik Klein
2007-08-22 12:34 ` Keir Fraser
2007-08-22 13:39   ` Dominik Klein
2007-08-22 13:48   ` Daniel P. Berrange
2007-08-22 13:53     ` Keir Fraser
2007-08-22 14:04       ` Dominik Klein
2007-08-22 14:15         ` Keir Fraser
2007-08-27  6:09           ` Dominik Klein
2007-08-27  6:35             ` Dominik Klein

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.