A question no one can answer

A question no one can answer

[Robert Stober]

Good Afternoon,

Is it possible to limit a guest OS to a specific memory bandwidth
allocation? The purpose is to stop one guest OS from saturating the
memory bus.

Thank you,

Robert

Re: A question no one can answer

[weiming]

[-- Attachment #1.1: Type: text/plain, Size: 739 bytes --]

You mean QoS on memory bandwidth? It's an interesting question. (there are
similar problems like QoS on shared L2)

I think it's hard to do.  Even for a native OS, it may only control this
indirectly by adjusting the CPU time slice. And I don't think any commercial
OS has implemented such feature yet.

Weiming

On Thu, Feb 14, 2008 at 9:27 PM, Robert Stober <rstober@platform.com> wrote:

> Good Afternoon,
>
> Is it possible to limit a guest OS to a specific memory bandwidth
> allocation? The purpose is to stop one guest OS from saturating the
> memory bus.
>
> Thank you,
>
> Robert
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

[-- Attachment #1.2: Type: text/html, Size: 1152 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: A question no one can answer

[Robert Stober]

[-- Attachment #1.1: Type: text/plain, Size: 1656 bytes --]

Weiming,
 
I agree that it is very hard, and that no one has done it. But
nevertheless I suggest the following question to the Xen developers:
 
Given the fact that memory bandwidth is shared amongst multiple cores on
a single die, assume that one VM is running on each core. What is to
stop one VM from saturating the memory bus, causing reduced performance
of all the other VMs? This is the general multi-core problem, not
specific to Xen. But it affects Xen greatly. What use is it to allocate
memory to a VM if it can't use the memory because a process of another
VM has saturated the memory bus?
 
Thank you,
 
Robert


  _____  

	From: weiming [mailto:zephyr.zhao@gmail.com] 
	Sent: Thursday, February 14, 2008 7:02 PM
	To: Robert Stober
	Cc: xen-devel@lists.xensource.com
	Subject: Re: [Xen-devel] A question no one can answer
	
	
	You mean QoS on memory bandwidth? It's an interesting question.
(there are similar problems like QoS on shared L2)
	
	I think it's hard to do.  Even for a native OS, it may only
control this indirectly by adjusting the CPU time slice. And I don't
think any commercial OS has implemented such feature yet.
	
	Weiming
	
	
	On Thu, Feb 14, 2008 at 9:27 PM, Robert Stober
<rstober@platform.com> wrote:
	

		Good Afternoon,
		
		Is it possible to limit a guest OS to a specific memory
bandwidth
		allocation? The purpose is to stop one guest OS from
saturating the
		memory bus.
		
		Thank you,
		
		Robert
		
		_______________________________________________
		Xen-devel mailing list
		Xen-devel@lists.xensource.com
		http://lists.xensource.com/xen-devel
		



[-- Attachment #1.2: Type: text/html, Size: 3761 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: A question no one can answer

[weiming]

[-- Attachment #1.1: Type: text/plain, Size: 2195 bytes --]

Hi Robert,

The memory bus controller like Memory Controller Hub (MCH) on Intel's north
bridge should arbitrate the traffic from different cores/processors. I guess
the hardware can guarantee some kind of fairness so that no core would be
starving and the other one is busy. (just my guess, I don't know the
details)

However, control the bandwidth with priorities is difficult. Any one has
ideas to this problem?

Weiming

On Thu, Feb 14, 2008 at 10:24 PM, Robert Stober <rstober@platform.com>
wrote:

>  Weiming,
>
> I agree that it is very hard, and that no one has done it. But nevertheless
> I suggest the following question to the Xen developers:
>
> Given the fact that memory bandwidth is shared amongst multiple cores on a
> single die, assume that one VM is running on each core. What is to stop one
> VM from saturating the memory bus, causing reduced performance of all the
> other VMs? This is the general multi-core problem, not specific to Xen. But
> it affects Xen greatly. What use is it to allocate memory to a VM if it
> can't use the memory because a process of another VM has saturated the
> memory bus?
>
> Thank you,
>
> Robert
>
>  ------------------------------
> *From:* weiming [mailto:zephyr.zhao@gmail.com]
> *Sent:* Thursday, February 14, 2008 7:02 PM
> *To:* Robert Stober
> *Cc:* xen-devel@lists.xensource.com
> *Subject:* Re: [Xen-devel] A question no one can answer
>
> You mean QoS on memory bandwidth? It's an interesting question. (there are
> similar problems like QoS on shared L2)
>
> I think it's hard to do.  Even for a native OS, it may only control this
> indirectly by adjusting the CPU time slice. And I don't think any commercial
> OS has implemented such feature yet.
>
> Weiming
>
> On Thu, Feb 14, 2008 at 9:27 PM, Robert Stober <rstober@platform.com>
> wrote:
>
> > Good Afternoon,
> >
> > Is it possible to limit a guest OS to a specific memory bandwidth
> > allocation? The purpose is to stop one guest OS from saturating the
> > memory bus.
> >
> > Thank you,
> >
> > Robert
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
>
>

[-- Attachment #1.2: Type: text/html, Size: 4282 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: A question no one can answer

[Keir Fraser]

Without hardware support to help with this (which does not exist) the
hypervisor is restricted to crude methods like monitoring cache-miss
performance counters and preempting an overly-aggressive guest. It is rather
likely that to get smooth enough throttling of bandwidth you would incur
significant monitoring and scheduling overhead. It's not even certain that
this approach would work at all!

 -- Keir

On 15/2/08 02:27, "Robert Stober" <rstober@platform.com> wrote:

> Good Afternoon,
> 
> Is it possible to limit a guest OS to a specific memory bandwidth
> allocation? The purpose is to stop one guest OS from saturating the
> memory bus.
> 
> Thank you,
> 
> Robert
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: A question no one can answer

[Alan Cox]

On Thu, 14 Feb 2008 22:24:18 -0500
"Robert Stober" <rstober@platform.com> wrote:

> Weiming,
>  
> I agree that it is very hard, and that no one has done it. But
> nevertheless I suggest the following question to the Xen developers:
>  
> Given the fact that memory bandwidth is shared amongst multiple cores on
> a single die, assume that one VM is running on each core. What is to
> stop one VM from saturating the memory bus, causing reduced performance
> of all the other VMs? This is the general multi-core problem, not
> specific to Xen. But it affects Xen greatly. What use is it to allocate
> memory to a VM if it can't use the memory because a process of another
> VM has saturated the memory bus?

Its perfectly doable on modern x86 - you use the profiling registers and
set them up so you get a threshold interrupt when too much main memory
traffic is counted (which you use to reschedule punishing the memory
user). There are research papers on it from quite a long time back but
afaik nobody ever implemented it in production although its not too hard
to do so might be an interesting project.

Alan

Re: A question no one can answer

[Dan Doucette]

[-- Attachment #1.1: Type: text/plain, Size: 1818 bytes --]

Hello Robert and others,

Below is a link to a presentation given at a conference last summer, where
the authors used cache colour allocation schemes to limit L2 cache overuse
by an application.
If your goal is to create a fair memory share per-guest (bandwidth or
otherwise), you may want to consider the possibility of managing it from a
level closer to the CPU.

http://www.ideal.ece.ufl.edu/workshops/wiosca07/P4Slides.pdf

Dan.




On Fri, Feb 15, 2008 at 5:05 AM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> On Thu, 14 Feb 2008 22:24:18 -0500
> "Robert Stober" <rstober@platform.com> wrote:
>
> > Weiming,
> >
> > I agree that it is very hard, and that no one has done it. But
> > nevertheless I suggest the following question to the Xen developers:
> >
> > Given the fact that memory bandwidth is shared amongst multiple cores on
> > a single die, assume that one VM is running on each core. What is to
> > stop one VM from saturating the memory bus, causing reduced performance
> > of all the other VMs? This is the general multi-core problem, not
> > specific to Xen. But it affects Xen greatly. What use is it to allocate
> > memory to a VM if it can't use the memory because a process of another
> > VM has saturated the memory bus?
>
> Its perfectly doable on modern x86 - you use the profiling registers and
> set them up so you get a threshold interrupt when too much main memory
> traffic is counted (which you use to reschedule punishing the memory
> user). There are research papers on it from quite a long time back but
> afaik nobody ever implemented it in production although its not too hard
> to do so might be an interesting project.
>
> Alan
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

[-- Attachment #1.2: Type: text/html, Size: 2651 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel