Is QoS of virtual disk not necessary?

Is QoS of virtual disk not necessary?

[Satoshi Uchida]

Hi,

I sent RFC and a patch-set of I/O framework for virtual disk in order to introduce QoS in end of July.
 > http://lists.xensource.com/archives/html/xen-devel/2007-07/msg00863.html
But no one responded to it.

I think that when many virtual machines run on one physical machine, QoS of machine resources will be necessary.
For example, in server consolidation, integrated servers have various roles and are different in the desired resources.
So, distribution of resources is very important for stabilization.
As other example, it will be avoided to be affected by jobs of other people in consolidation of office platform.

CPU and memory are controlled (allocated) at Xen hypervisor. (This is OS agnostic)
Is similar controller for I/O not necessary?

Linux adopts CFQ which is fair scheduler which is suitable for desktop system.
But, privileged domain will be not only Linux in future.
I think that such function should be developed in Xen hypervisor or virtual block driver framework.

Does anyone have opinions or idea?
Please reply comments.

 --------------------------------------------------
 Satoshi UCHIDA 

Re: Is QoS of virtual disk not necessary?

[Keir Fraser]

First impression was a lot of plumbing and not much in the way of moving
parts. Also of course we have CFQ on Linux -- I suspect the kinds of changes
you are suggesting would not be popular with kernel maintainers since they
will argue there is already an I/O scheduling subsystem. :-)

On the other hand, if you want to run a block driver in a driver domain (and
so outside dom0) then having a programmatic scheduling interface via
xenstore is quite nice...

 -- Keir

On 22/8/07 10:32, "Satoshi Uchida" <s-uchida@ap.jp.nec.com> wrote:

> Hi,
> 
> I sent RFC and a patch-set of I/O framework for virtual disk in order to
> introduce QoS in end of July.
>> http://lists.xensource.com/archives/html/xen-devel/2007-07/msg00863.html
> But no one responded to it.
> 
> I think that when many virtual machines run on one physical machine, QoS of
> machine resources will be necessary.
> For example, in server consolidation, integrated servers have various roles
> and are different in the desired resources.
> So, distribution of resources is very important for stabilization.
> As other example, it will be avoided to be affected by jobs of other people in
> consolidation of office platform.
> 
> CPU and memory are controlled (allocated) at Xen hypervisor. (This is OS
> agnostic)
> Is similar controller for I/O not necessary?
> 
> Linux adopts CFQ which is fair scheduler which is suitable for desktop system.
> But, privileged domain will be not only Linux in future.
> I think that such function should be developed in Xen hypervisor or virtual
> block driver framework.
> 
> Does anyone have opinions or idea?
> Please reply comments.
> 
>  --------------------------------------------------
>  Satoshi UCHIDA 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: Is QoS of virtual disk not necessary?

[Satoshi Uchida]

[-- Attachment #1.1: Type: text/plain, Size: 1848 bytes --]

Hi, Keir. 

Thanks for comment.

> First impression was a lot of plumbing and not much in the way of 
> moving parts.

What does this means?

Are "iomgr_xx" procedures too much for the turn-based control module?
E.g. the iomgr_oo_abort_request_fn procedure is not used in turn-based control module.


> Also of course we have CFQ on Linux -- I suspect the kinds of changes 
> you are suggesting would not be popular with kernel maintainers since 
> they will argue there is already an I/O scheduling subsystem. :-)

I think that using I/O scheduling subsystem is one of solutions.
But, I fear two things.

  One is that CFQ can be used only if a driver domain (or dom0) is only Linux.
  In future, driver domains will be choose any types of OS, and they don't have same I/O scheduling mechanism.
  And there are also many I/O schedulers in Linux, and driver domains will be choose some schedulers.
  Therefore, I/O control is not dependent with I/O scheduler in specific type of OS.

  Another is that CFQ is developed for desktop system and for private environments.
  So, this may not be suitable in virtualization environments.
  And, a setting parameter of CFQ is too simple, namely it have only 8-level priority ranks.
  Therefore, it is difficult to apply CFQ into huge virtualization system.
  E.g. for many domains, it is difficult to set them by a percentage.

Therefore, I think that it is better to develop OS-agnostic I/O control.


> On the other hand, if you want to run a block driver in a driver 
> domain (and so outside dom0) then having a programmatic scheduling 
> interface via xenstore is quite nice...

Does this mean that interfaces should not be implemented by insmod or rewriting sysfs entries, but should be implemented by xenstore and xm commands?


Thanks.
---------------------------------------------
 Satoshi UCHIDA

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 4036 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Is QoS of virtual disk not necessary?

[Keir Fraser]

On 24/8/07 09:27, "Satoshi Uchida" <s-uchida@ap.jp.nec.com> wrote:

>   Another is that CFQ is developed for desktop system and for private
> environments.
>   So, this may not be suitable in virtualization environments.
>   And, a setting parameter of CFQ is too simple, namely it have only 8-level
> priority ranks.
>   Therefore, it is difficult to apply CFQ into huge virtualization system.
>   E.g. for many domains, it is difficult to set them by a percentage.
> 
> Therefore, I think that it is better to develop OS-agnostic I/O control.

Another nice thing would be that if we do not use CFQ then we do not need a
kernel thread per VBD. We could support one kernel thread per blkback and
one kernel thread per VBD.

I don't know if both these models can be neatly supported by a single
consistent set of iomgr hooks.

>> On the other hand, if you want to run a block driver in a driver
>> domain (and so outside dom0) then having a programmatic scheduling
>> interface via xenstore is quite nice...
> 
> Does this mean that interfaces should not be implemented by insmod or
> rewriting sysfs entries, but should be implemented by xenstore and xm
> commands?

Hmmm... Well actually all the blkdev stats are exported thru sysfs right
now, so already there are things that do not work well when blkback is in a
driver domain (e.g., xentop). Perhaps we should export everything thru
sysfs, but then provide a way to proxy that info through xenstore too, as an
optional extra (either a user-space daemon, or we could make it a kernel
driver option).

 -- Keir

Re: Is QoS of virtual disk not necessary?

[Andi Kleen]

Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > Therefore, I think that it is better to develop OS-agnostic I/O control.
> 
> Another nice thing would be that if we do not use CFQ then we do not need a
> kernel thread per VBD.

It would be probably easy to extend CFQ2 to use an user passed identifier
instead of per task for sharing if that's your goal. For a kernel driver
like blkback you could just switch around multiple current->io_contexts.

And if CFQ2 can manage hundreds of processes I don't see why it
couldn't manage hundreds of guests. Although you probably should not
put that many on a single device anyways; or rather if it's a single
device for that many it's likely a RAM backed big storage box that
doesn't need much scheduling anyways. 

The claim that CFQ2 is desktop oriented in this thread earlier is also
not true. The desktop oriented scheduler is more AS and CFQ2 is widely
and successfully used in server applications.

-Andi

Re: Is QoS of virtual disk not necessary?

[Keir Fraser]

On 24/8/07 18:26, "Andi Kleen" <andi@firstfloor.org> wrote:

>> Another nice thing would be that if we do not use CFQ then we do not need a
>> kernel thread per VBD.
> 
> It would be probably easy to extend CFQ2 to use an user passed identifier
> instead of per task for sharing if that's your goal. For a kernel driver
> like blkback you could just switch around multiple current->io_contexts.

Does current->io_context get latched before submit_bh() (or whatever similar
function it is we use) returns? Is this true even in older 2.6 kernels with
old CFQ?

 -- Keir

Re: Is QoS of virtual disk not necessary?

[Andi Kleen]

On Fri, Aug 24, 2007 at 06:40:54PM +0100, Keir Fraser wrote:
> On 24/8/07 18:26, "Andi Kleen" <andi@firstfloor.org> wrote:
> 
> >> Another nice thing would be that if we do not use CFQ then we do not need a
> >> kernel thread per VBD.
> > 
> > It would be probably easy to extend CFQ2 to use an user passed identifier
> > instead of per task for sharing if that's your goal. For a kernel driver
> > like blkback you could just switch around multiple current->io_contexts.
> 
> Does current->io_context get latched before submit_bh() (or whatever similar
> function it is we use) returns? Is this true even in older 2.6 kernels with
> old CFQ?

It obviously has to because you could exit in the next call.
The context is reference counted and lazily freed.

-Andi

Re: Is QoS of virtual disk not necessary?

[Andi Kleen]

> CFQ2 is CFQ version 2 or new CFQ?
> In linux documentation ( Documentation/block/ioprio.txt ), I see that CFQ is version 3.
>  > With the introduction of cfq v3
> If CFQ2 is CFQ version 2, which is current CFQ version 2? or version 3?

Linux kernel subsystems rarely have fixed meaningfull major release numbers. They just
improve continuously in small steps with each kernel release. 

> However, CFQ is implemented in only Linux and not in other OSs.
> So, privileged domains or driver domains are restricted with only Linux.
> In current, privileged domain (domain 0) is only Linux, but is this situation continued in future?
> I think that users use also other OSs as driver domains.
> Therefore, I suggested IOMGR.
> By control mechanism is developed at backend drivers, I/O control setting will become OS independent.
> -- But, it is needed to implement for other OSs.

Xen depends on Dom0 IO facilities for a lot of things. Following your argument
to the logical conclusion you would need to reimplement a lot more code 
(like firewalling or network QoS) in Xen too. Just doesn't seem to be
a wise investment of precious development resources to me, especially since most Dom0 
OSes probably already have mostly equivalent facilities.

Of course without hard data it is a stab in the dark. Did you
do some benchmarking if there is a real fairness problem currently?

> > The claim that CFQ2 is desktop oriented in this thread 
> > earlier is also not true. The desktop oriented scheduler is 
> > more AS and CFQ2 is widely and successfully used in server 
> > applications.
> 
> I read on web that "CFQ is made for multimedia desktop usage only” said Andrea.
>    http://www.cs.rice.edu/~ssiyer/r/antsched/shines.html
> (Perhaps version 1 ?)

Perhaps. 

> And following document is found in Kconfig.iosched.
> 　config IOSCHED_CFQ
> 	tristate "CFQ I/O scheduler"
> 	default y
> 	---help---
> 	  The CFQ I/O scheduler tries to distribute bandwidth equally
> 	  among all processes in the system. It should provide a fair
> 	  working environment, suitable for desktop systems. 
> 	  This is the default I/O scheduler.
> 
> Therefore, I seem that "CFQ is suitable for desktop system".
> However, its report is old, and I don't know surely whether current CFQ is desktop oriented or not.

First server (and as well as desktop) workloads are quite varied. It is hard to say 
anything ultimative and there can be always special cases and exceptions.
Also desktop and server workloads can overlap. The main difference is probably
just that servers tend to have a lot more spindles, but that normally
just requires less scheduling then.

CFQ2 is the default scheduler in the main server oriented distributions
(SLES, RHEL) so it's not doing that badly. It's also default in most
desktop oriented distributions (Fedora, OpenSUSE, Ubuntu, etc.) as far
as I know.

The main reason why AS (which showing nice gains in some ways) didn't
really catch on was that it was somewhat slower in some server workloads

In some cases special workloads prefer other schedulers, but they often want
the noop scheduler then (e.g. some highend databases tend to prefer to 
be not scheduled by the OS at all while others like MySQL tend to prefer
some help from the OS)


-Andi

how many processors does xen support in hardware ?

[tgh]

hi
   how many processors does xen support in hardware for the practical 
system?
  what is the max number of processors in the practice system running 
under xen ?


Thanks in advance

Re: how many processors does xen support in hardware ?

[pradeep singh rautela]

On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote:
> hi
>    how many processors does xen support in hardware for the practical
> system?

Thats would be 255 i guess.

thanks
>   what is the max number of processors in the practice system running
> under xen ?

>
>
> Thanks in advance
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
--
pradeep singh rautela

"Genius is 1% inspiration, and 99% perspiration" - not me :)

Re: how many processors does xen support in hardware ?

[tgh]

pradeep singh rautela 写道:
> On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote:
>   
>> hi
>>    how many processors does xen support in hardware for the practical
>> system?
>>     
>
> Thats would be 255 i guess.
>   
thank you for your reply
but I want to kown the max number of hardware cpus the xen could 
support, or the xen's scalability in the aspect of cpu number,not in the 
aspect of the VM number

thanks





> thanks
>   
>>   what is the max number of processors in the practice system running
>> under xen ?
>>     
>
>   
>> Thanks in advance
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>     
>
>
>   

RE: how many processors does xen support in hardware?

[Puthiyaparambil, Aravindh]

Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and there is no easy way to increase this.

Xen x86_64 supports 64 logical CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If you want to go beyond 126 LCPUs some patching is needed. 

I guess you can theoretically go up to 255 but I think you might run out of GDT slots before that.

Aravindh Puthiyaparambil 
Virtualization Engineer
Virtual Systems Development
Unisys, Tredyffrin PA

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh
Sent: Friday, August 24, 2007 7:31 AM
To: pradeep singh rautela
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] how many processors does xen support in hardware?

pradeep singh rautela 写道:
> On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote:
>   
>> hi
>>    how many processors does xen support in hardware for the practical
>> system?
>>     
>
> Thats would be 255 i guess.
>   
thank you for your reply
but I want to kown the max number of hardware cpus the xen could 
support, or the xen's scalability in the aspect of cpu number,not in the 
aspect of the VM number

thanks





> thanks
>   
>>   what is the max number of processors in the practice system running
>> under xen ?
>>     
>
>   
>> Thanks in advance
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>     
>
>
>   


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: how many processors does xen support in hardware?

[pradeep singh rautela]

[-- Attachment #1: Type: text/plain, Size: 2090 bytes --]

On 8/24/07, Puthiyaparambil, Aravindh
<aravindh.puthiyaparambil@unisys.com> wrote:
> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and there is no easy way to increase this.
>
> Xen x86_64 supports 64 logical CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If you want to go beyond 126 LCPUs some patching is needed.
>
> I guess you can theoretically go up to 255 but I think you might run out of GDT slots before that.

Right, so why keep 255 there at all? I mean why hard code it  to a max
255 then?Put it at a number a little near or higher than what Xen is
actually capable of.

thanks
>
> Aravindh Puthiyaparambil
> Virtualization Engineer
> Virtual Systems Development
> Unisys, Tredyffrin PA
>
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh
> Sent: Friday, August 24, 2007 7:31 AM
> To: pradeep singh rautela
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] how many processors does xen support in hardware?
>
> pradeep singh rautela 写道:
> > On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote:
> >
> >> hi
> >>    how many processors does xen support in hardware for the practical
> >> system?
> >>
> >
> > Thats would be 255 i guess.
> >
> thank you for your reply
> but I want to kown the max number of hardware cpus the xen could
> support, or the xen's scalability in the aspect of cpu number,not in the
> aspect of the VM number
>
> thanks
>
>
>
>
>
> > thanks
> >
> >>   what is the max number of processors in the practice system running
> >> under xen ?
> >>
> >
> >
> >> Thanks in advance
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xensource.com
> >> http://lists.xensource.com/xen-devel
> >>
> >>
> >
> >
> >
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
--
pradeep singh rautela

"Genius is 1% inspiration, and 99% perspiration" - not me :)

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: how many processors does xen support in hardware?

[Keir Fraser]

Where did you get this number 255 from?

 K.


On 24/8/07 15:32, "pradeep singh rautela" <rautelap@gmail.com> wrote:

> On 8/24/07, Puthiyaparambil, Aravindh
<aravindh.puthiyaparambil@unisys.com>
> wrote:
> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and
> there is no easy way to increase this.
>
> Xen x86_64 supports 64 logical
> CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If
> you want to go beyond 126 LCPUs some patching is needed.
>
> I guess you can
> theoretically go up to 255 but I think you might run out of GDT slots before
> that.

Right, so why keep 255 there at all? I mean why hard code it  to a
> max
255 then?Put it at a number a little near or higher than what Xen
> is
actually capable of.

thanks
>
> Aravindh Puthiyaparambil
> Virtualization
> Engineer
> Virtual Systems Development
> Unisys, Tredyffrin PA
>
>
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh
> Sent:
> Friday, August 24, 2007 7:31 AM
> To: pradeep singh rautela
> Cc:
> xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] how many processors
> does xen support in hardware?
>
> pradeep singh rautela 写道:
> > On 8/24/07,
> tgh <tianguanhua@ncic.ac.cn> wrote:
> >
> >> hi
> >>    how many processors
> does xen support in hardware for the practical
> >> system?
> >>
> >
> > Thats
> would be 255 i guess.
> >
> thank you for your reply
> but I want to kown the
> max number of hardware cpus the xen could
> support, or the xen's scalability
> in the aspect of cpu number,not in the
> aspect of the VM number
>
>
> thanks
>
>
>
>
>
> > thanks
> >
> >>   what is the max number of processors in
> the practice system running
> >> under xen ?
> >>
> >
> >
> >> Thanks in
> advance
> >>
> >>
> >> _______________________________________________
> >>
> Xen-devel mailing list
> >> Xen-devel@lists.xensource.com
> >>
> http://lists.xensource.com/xen-devel
> >>
> >>
> >
> >
> >
>
>
>
> _______________________________________________
> Xen-devel mailing list
>
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


--
> 
--
pradeep singh rautela

"Genius is 1% inspiration, and 99% perspiration" -
> not me :)

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: how many processors does xen support in hardware?

[pradeep singh]

On Friday 24 August 2007 20:19:07 Keir Fraser wrote:
> Where did you get this number 255 from?

Sorry i meant, while i compile the xen kernel i can configure kernel upto a 
max of 255. right? 

And i guess that means the number of CPUs you can compile your SMP kernel to 
work with. 
what did i miss? :-/

thanks
--pradeep
>
>  K.
>
> On 24/8/07 15:32, "pradeep singh rautela" <rautelap@gmail.com> wrote:
> > On 8/24/07, Puthiyaparambil, Aravindh
>
> <aravindh.puthiyaparambil@unisys.com>
>
> > wrote:
> > Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and
> > there is no easy way to increase this.
> >
> > Xen x86_64 supports 64 logical
> > CPUs. You need to compile Xen with max_phys_cpus=64 to enable this
> > support. If you want to go beyond 126 LCPUs some patching is needed.
> >
> > I guess you can
> > theoretically go up to 255 but I think you might run out of GDT slots
> > before that.
>
> Right, so why keep 255 there at all? I mean why hard code it  to a
>
> > max
>
> 255 then?Put it at a number a little near or higher than what Xen
>
> > is
>
> actually capable of.
>
> thanks
>
> > Aravindh Puthiyaparambil
> > Virtualization
> > Engineer
> > Virtual Systems Development
> > Unisys, Tredyffrin PA
> >
> >
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com
> > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh
> > Sent:
> > Friday, August 24, 2007 7:31 AM
> > To: pradeep singh rautela
> > Cc:
> > xen-devel@lists.xensource.com
> > Subject: Re: [Xen-devel] how many processors
> > does xen support in hardware?
> >
> > pradeep singh rautela 写道:
> > > On 8/24/07,
> >
> > tgh <tianguanhua@ncic.ac.cn> wrote:
> > >> hi
> > >>    how many processors
> >
> > does xen support in hardware for the practical
> >
> > >> system?
> > >
> > > Thats
> >
> > would be 255 i guess.
> >
> > thank you for your reply
> > but I want to kown the
> > max number of hardware cpus the xen could
> > support, or the xen's scalability
> > in the aspect of cpu number,not in the
> > aspect of the VM number
> >
> >
> > thanks
> >
> > > thanks
> > >
> > >>   what is the max number of processors in
> >
> > the practice system running
> >
> > >> under xen ?
> > >>
> > >>
> > >>
> > >> Thanks in
> >
> > advance
> >
> > >> _______________________________________________
> >
> > Xen-devel mailing list
> >
> > >> Xen-devel@lists.xensource.com
> >
> > http://lists.xensource.com/xen-devel
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> >
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
> --
>
> --
> pradeep singh rautela
>
> "Genius is 1% inspiration, and 99% perspiration" -
>
> > not me :)
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel

Re: how many processors does xen support in hardware?

[Keir Fraser]

Oh, right, yes that is Linux's configurable maximum (itself fairly arbitrary
really). You can't actually use that number on Xen, so don't set it so high.
Simple really. :-)

 -- Keir

On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote:

> On Friday 24 August 2007 20:19:07 Keir Fraser wrote:
>> Where did you get this number 255 from?
> 
> Sorry i meant, while i compile the xen kernel i can configure kernel upto a
> max of 255. right?
> 
> And i guess that means the number of CPUs you can compile your SMP kernel to
> work with. 
> what did i miss? :-/
> 
> thanks
> --pradeep
>> 
>>  K.
>> 
>> On 24/8/07 15:32, "pradeep singh rautela" <rautelap@gmail.com> wrote:
>>> On 8/24/07, Puthiyaparambil, Aravindh
>> 
>> <aravindh.puthiyaparambil@unisys.com>
>> 
>>> wrote:
>>> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and
>>> there is no easy way to increase this.
>>> 
>>> Xen x86_64 supports 64 logical
>>> CPUs. You need to compile Xen with max_phys_cpus=64 to enable this
>>> support. If you want to go beyond 126 LCPUs some patching is needed.
>>> 
>>> I guess you can
>>> theoretically go up to 255 but I think you might run out of GDT slots
>>> before that.
>> 
>> Right, so why keep 255 there at all? I mean why hard code it  to a
>> 
>>> max
>> 
>> 255 then?Put it at a number a little near or higher than what Xen
>> 
>>> is
>> 
>> actually capable of.
>> 
>> thanks
>> 
>>> Aravindh Puthiyaparambil
>>> Virtualization
>>> Engineer
>>> Virtual Systems Development
>>> Unisys, Tredyffrin PA
>>> 
>>> 
>>> -----Original Message-----
>>> From: xen-devel-bounces@lists.xensource.com
>>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh
>>> Sent:
>>> Friday, August 24, 2007 7:31 AM
>>> To: pradeep singh rautela
>>> Cc:
>>> xen-devel@lists.xensource.com
>>> Subject: Re: [Xen-devel] how many processors
>>> does xen support in hardware?
>>> 
>>> pradeep singh rautela 写道:
>>>> On 8/24/07,
>>> 
>>> tgh <tianguanhua@ncic.ac.cn> wrote:
>>>>> hi
>>>>>    how many processors
>>> 
>>> does xen support in hardware for the practical
>>> 
>>>>> system?
>>>> 
>>>> Thats
>>> 
>>> would be 255 i guess.
>>> 
>>> thank you for your reply
>>> but I want to kown the
>>> max number of hardware cpus the xen could
>>> support, or the xen's scalability
>>> in the aspect of cpu number,not in the
>>> aspect of the VM number
>>> 
>>> 
>>> thanks
>>> 
>>>> thanks
>>>> 
>>>>>   what is the max number of processors in
>>> 
>>> the practice system running
>>> 
>>>>> under xen ?
>>>>> 
>>>>> 
>>>>> 
>>>>> Thanks in
>>> 
>>> advance
>>> 
>>>>> _______________________________________________
>>> 
>>> Xen-devel mailing list
>>> 
>>>>> Xen-devel@lists.xensource.com
>>> 
>>> http://lists.xensource.com/xen-devel
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> 
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>> 
>> --
>> 
>> --
>> pradeep singh rautela
>> 
>> "Genius is 1% inspiration, and 99% perspiration" -
>> 
>>> not me :)
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
> 
> 

Re: how many processors does xen support in hardware?

[pradeep singh]

On Friday 24 August 2007 23:34:27 Keir Fraser wrote:
> Oh, right, yes that is Linux's configurable maximum (itself fairly
> arbitrary really). You can't actually use that number on Xen, so don't set
> it so high. Simple really. :-)

Simple?...heh :-)
I'll keep that in mind.

thanks for insight.
--pradeep
>
>  -- Keir
>
> On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote:
> > On Friday 24 August 2007 20:19:07 Keir Fraser wrote:
> >> Where did you get this number 255 from?
> >
> > Sorry i meant, while i compile the xen kernel i can configure kernel upto
> > a max of 255. right?
> >
> > And i guess that means the number of CPUs you can compile your SMP kernel
> > to work with.
> > what did i miss? :-/
> >
> > thanks
> > --pradeep
> >
> >>  K.
> >>
[snip]

Re: how many processors does xen support in hardware?

[tgh]

hi
did someone make the experiment for the xen's scalability in the aspect
of the cpu number supported? and what about the performence with 16 cpus
, 32 cpus , and 64cpus, say ,comparing with the native linux?



pradeep singh 写道:
> On Friday 24 August 2007 23:34:27 Keir Fraser wrote:
>   
>> Oh, right, yes that is Linux's configurable maximum (itself fairly
>> arbitrary really). You can't actually use that number on Xen, so don't set
>> it so high. Simple really. :-)
>>     
>
> Simple?...heh :-)
> I'll keep that in mind.
>
> thanks for insight.
> --pradeep
>   
>>  -- Keir
>>
>> On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote:
>>     
>>> On Friday 24 August 2007 20:19:07 Keir Fraser wrote:
>>>       
>>>> Where did you get this number 255 from?
>>>>         
>>> Sorry i meant, while i compile the xen kernel i can configure kernel upto
>>> a max of 255. right?
>>>
>>> And i guess that means the number of CPUs you can compile your SMP kernel
>>> to work with.
>>> what did i miss? :-/
>>>
>>> thanks
>>> --pradeep
>>>
>>>       
>>>>  K.
>>>>
>>>>         
> [snip]
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
> .
>
>   

Re: how many processors does xen support in hardware?

[tgh]

Thank you for your reply
and did you take any scalability experiments for xen under hardware 
platform with 16cpus or more cpus for datacenter application or any 
other applications? and what about the performance of xen ,comparing 
with the native linux?


Thanks



Mats Petersson 写道:
> At 08:29 25/08/2007, you wrote:
>> hi
>> did someone make the experiment for the xen's scalability in the aspect
>> of the cpu number supported? and what about the performence with 16 cpus
>> , 32 cpus , and 64cpus, say ,comparing with the native linux?
>
>
> This would be very dependant on the workload type - something that 
> does a lot of page-table updates will be more affected by "xen 
> overhead" than something that does no page-table updates. Other than 
> that, a para-virtual guest should be fairly close to the native setup.
>
> Of course, finding machines with more than 32 cores is pretty 
> difficult, as 32 takes 8 sockets of quad-cores, and that's pretty much 
> the limit on "standard" motherboards - and even those are pretty darn 
> expensive.
>
> -- 
> Mats
>
>
>
>
>> pradeep singh $B<LF;(B:
>> > On Friday 24 August 2007 23:34:27 Keir Fraser wrote:
>> >
>> >> Oh, right, yes that is Linux's configurable maximum (itself fairly
>> >> arbitrary really). You can't actually use that number on Xen, so 
>> don't set
>> >> it so high. Simple really. :-)
>> >>
>> >
>> > Simple?...heh :-)
>> > I'll keep that in mind.
>> >
>> > thanks for insight.
>> > --pradeep
>> >
>> >> -- Keir
>> >>
>> >> On 24/8/07 19:00, "pradeep singh" <rautelap@gmail.com> wrote:
>> >>
>> >>> On Friday 24 August 2007 20:19:07 Keir Fraser wrote:
>> >>>
>> >>>> Where did you get this number 255 from?
>> >>>>
>> >>> Sorry i meant, while i compile the xen kernel i can configure 
>> kernel upto
>> >>> a max of 255. right?
>> >>>
>> >>> And i guess that means the number of CPUs you can compile your 
>> SMP kernel
>> >>> to work with.
>> >>> what did i miss? :-/
>> >>>
>> >>> thanks
>> >>> --pradeep
>> >>>
>> >>>
>> >>>> K.
>> >>>>
>> >>>>
>> > [snip]
>> >
>> >
>> >
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@lists.xensource.com
>> > http://lists.xensource.com/xen-devel
>> >
>> >
>> > .
>> >
>> >
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
> .
>

RE: how many processors does xen support in hardware?

[Puthiyaparambil, Aravindh]

It is not hard coded to 255. For example if you want to up the number to 96, then build Xen x86_64 with "make max_phys_cpus=96" and so on. But like I mentioned anything above 126 will require some patching.

BTW (echoing Keir's question), why do you think it is hard coded to 255?

Aravindh

-----Original Message-----
From: pradeep singh rautela [mailto:rautelap@gmail.com] 
Sent: Friday, August 24, 2007 10:33 AM
To: Puthiyaparambil, Aravindh
Cc: tgh; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] how many processors does xen support in hardware?

On 8/24/07, Puthiyaparambil, Aravindh
<aravindh.puthiyaparambil@unisys.com> wrote:
> Xen x86_32/PAE supports 32 logical CPUs. This is a hard limit and there is no easy way to increase this.
>
> Xen x86_64 supports 64 logical CPUs. You need to compile Xen with max_phys_cpus=64 to enable this support. If you want to go beyond 126 LCPUs some patching is needed.
>
> I guess you can theoretically go up to 255 but I think you might run out of GDT slots before that.

Right, so why keep 255 there at all? I mean why hard code it  to a max
255 then?Put it at a number a little near or higher than what Xen is
actually capable of.

thanks
>
> Aravindh Puthiyaparambil
> Virtualization Engineer
> Virtual Systems Development
> Unisys, Tredyffrin PA
>
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of tgh
> Sent: Friday, August 24, 2007 7:31 AM
> To: pradeep singh rautela
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] how many processors does xen support in hardware?
>
> pradeep singh rautela 写道:
> > On 8/24/07, tgh <tianguanhua@ncic.ac.cn> wrote:
> >
> >> hi
> >>    how many processors does xen support in hardware for the practical
> >> system?
> >>
> >
> > Thats would be 255 i guess.
> >
> thank you for your reply
> but I want to kown the max number of hardware cpus the xen could
> support, or the xen's scalability in the aspect of cpu number,not in the
> aspect of the VM number
>
> thanks
>
>
>
>
>
> > thanks
> >
> >>   what is the max number of processors in the practice system running
> >> under xen ?
> >>
> >
> >
> >> Thanks in advance
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xensource.com
> >> http://lists.xensource.com/xen-devel
> >>
> >>
> >
> >
> >
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
--
pradeep singh rautela

"Genius is 1% inspiration, and 99% perspiration" - not me :)