TMEM + pv_ops kernel?

TMEM + pv_ops kernel?

[Rick Boone]

[-- Attachment #1.1: Type: text/plain, Size: 322 bytes --]

Hey,

I'm building Jeremy's pv_ops dom0 kernel (2.6.32.12) for use with Xen 
4.0 and it looks like the Transcendant Memory options arent in the 
kernel config. Is TMEM not available in pv_ops? I've been looking 
forward to using it with Xen4, but also really wanna work with pv_ops 
moving forward.

Thanks alot,
-- Rick


[-- Attachment #1.2: Type: text/html, Size: 608 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: TMEM + pv_ops kernel?

[Dan Magenheimer]

[-- Attachment #1.1: Type: text/plain, Size: 1353 bytes --]

Hi Rick -

 

Tmem is primarily of use in PV guests, less so in dom0.  Are you putting pv_ops kernels in your PV guests (domUs)?

 

If so, Jeremy has been a big proponent of tmem and we can work together to get it into his pv_ops tree.   But I think most of the focus for the pv_ops work has been to provide dom0 support.  Upstream kernels already provide domU support, so I've been instead working on the linux-kernel-mailing-list toward getting tmem patches accepted upstream.

 

If not, just using a tmem-modified dom0 will probably not help you.   I have been working on tmem-modified rpm's for various RedHat-ish kernels and I know Jan Beulich has been working on putting tmem support into various SuSE kernels.

 

I'd also be interested in more information about your planned usage model for tmem.

 

Thanks,

Dan

 

From: Rick Boone [mailto:rick@buzz-media.com] 
Sent: Thursday, May 13, 2010 4:17 PM
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] TMEM + pv_ops kernel?

 

Hey, 

I'm building Jeremy's pv_ops dom0 kernel (2.6.32.12) for use with Xen 4.0 and it looks like the Transcendant Memory options arent in the kernel config. Is TMEM not available in pv_ops? I've been looking forward to using it with Xen4, but also really wanna work with pv_ops moving forward. 

Thanks alot,  
-- Rick

[-- Attachment #1.2: Type: text/html, Size: 5602 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: TMEM + pv_ops kernel?

[Rick Boone]

[-- Attachment #1.1: Type: text/plain, Size: 2447 bytes --]

Hey Dan,

Yeah, I'm planning on using pv_ops for both the dom0 and domU kernels.

Any movement towards making TMEM available would be great; it's one of 
the key things we're looking forward to testing and using (along with Remus)

I'm systems engineer at a company running about 50 blogs/social 
networking websites (http://buzz-media.com) , all on top of a heavily 
virtualized infrastructure (Xen [v3.2] is our sole virt platform) . 
We've got about 80 dom0's, holding about 125 domU's and on a few of 
those boxes, especially those holding one or more of our web server 
domU's, we run into "out of memory" crashes from time to time. So, we're 
hoping to see if TMEM helps reduce our memory woes. We tend to mix and 
match types of domU's within dom0's, so some boxes will have 1 
memory-hungry web server, alongside a number of smaller, low-usage, 
low-memory, developer boxes. From what we've read, TMEM might be perfect 
for our setup.

-- Rick


On 05/14/2010 08:26 AM, Dan Magenheimer wrote:
>
> Hi Rick –
>
> Tmem is primarily of use in PV guests, less so in dom0.  Are you 
> putting pv_ops kernels in your PV guests (domUs)?
>
> If so, Jeremy has been a big proponent of tmem and we can work 
> together to get it into his pv_ops tree.   But I think most of the 
> focus for the pv_ops work has been to provide dom0 support.  Upstream 
> kernels already provide domU support, so I’ve been instead working on 
> the linux-kernel-mailing-list toward getting tmem patches accepted 
> upstream.
>
> If not, just using a tmem-modified dom0 will probably not help you.  
>  I have been working on tmem-modified rpm’s for various RedHat-ish 
> kernels and I know Jan Beulich has been working on putting tmem 
> support into various SuSE kernels.
>
> I’d also be interested in more information about your planned usage 
> model for tmem.
>
> Thanks,
>
> Dan
>
> *From:* Rick Boone [mailto:rick@buzz-media.com]
> *Sent:* Thursday, May 13, 2010 4:17 PM
> *To:* xen-devel@lists.xensource.com
> *Subject:* [Xen-devel] TMEM + pv_ops kernel?
>
> Hey,
>
> I'm building Jeremy's pv_ops dom0 kernel (2.6.32.12) for use with Xen 
> 4.0 and it looks like the Transcendant Memory options arent in the 
> kernel config. Is TMEM not available in pv_ops? I've been looking 
> forward to using it with Xen4, but also really wanna work with pv_ops 
> moving forward.
>
> Thanks alot,
> -- Rick
>

[-- Attachment #1.2: Type: text/html, Size: 7591 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: TMEM + pv_ops kernel?

[Dan Magenheimer]

[-- Attachment #1.1: Type: text/plain, Size: 2841 bytes --]

Hi Rick -


Which exact pv_ops tree do you expect to use?  The Linux patch is slightly different on each Linux version and I don't know if/how patches such as tmem will get transferred across Jeremy's and Konrad's different pv_ops trees.


Dan

 

From: Rick Boone [mailto:rick@buzz-media.com] 
Sent: Friday, May 14, 2010 4:38 PM
To: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] TMEM + pv_ops kernel?

 

Hey Dan, 

Yeah, I'm planning on using pv_ops for both the dom0 and domU kernels. 

Any movement towards making TMEM available would be great; it's one of the key things we're looking forward to testing and using (along with Remus)

I'm systems engineer at a company running about 50 blogs/social networking websites (http://buzz-media.com) , all on top of a heavily virtualized infrastructure (Xen [v3.2] is our sole virt platform) . We've got about 80 dom0's, holding about 125 domU's and on a few of those boxes, especially those holding one or more of our web server domU's, we run into "out of memory" crashes from time to time. So, we're hoping to see if TMEM helps reduce our memory woes. We tend to mix and match types of domU's within dom0's, so some boxes will have 1 memory-hungry web server, alongside a number of smaller, low-usage, low-memory, developer boxes. From what we've read, TMEM might be perfect for our setup. 

-- Rick


On 05/14/2010 08:26 AM, Dan Magenheimer wrote: 

Hi Rick -

 

Tmem is primarily of use in PV guests, less so in dom0.  Are you putting pv_ops kernels in your PV guests (domUs)?

 

If so, Jeremy has been a big proponent of tmem and we can work together to get it into his pv_ops tree.   But I think most of the focus for the pv_ops work has been to provide dom0 support.  Upstream kernels already provide domU support, so I've been instead working on the linux-kernel-mailing-list toward getting tmem patches accepted upstream.

 

If not, just using a tmem-modified dom0 will probably not help you.   I have been working on tmem-modified rpm's for various RedHat-ish kernels and I know Jan Beulich has been working on putting tmem support into various SuSE kernels.

 

I'd also be interested in more information about your planned usage model for tmem.

 

Thanks,

Dan

 

From: Rick Boone [mailto:rick@buzz-media.com] 
Sent: Thursday, May 13, 2010 4:17 PM
To: HYPERLINK "mailto:xen-devel@lists.xensource.com"xen-devel@lists.xensource.com
Subject: [Xen-devel] TMEM + pv_ops kernel?

 

Hey, 

I'm building Jeremy's pv_ops dom0 kernel (2.6.32.12) for use with Xen 4.0 and it looks like the Transcendant Memory options arent in the kernel config. Is TMEM not available in pv_ops? I've been looking forward to using it with Xen4, but also really wanna work with pv_ops moving forward. 

Thanks alot,  
-- Rick

[-- Attachment #1.2: Type: text/html, Size: 8557 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: TMEM + pv_ops kernel?

[Rick Boone]

[-- Attachment #1.1: Type: text/plain, Size: 3307 bytes --]

Hey Dan,

I'm currently working with the xen/stable-2.6.32.x branch (2.6.32.12 in 
particular), from Jeremy's repo. I figured we would try to standardize 
around that, since the xen wiki says it's gonna be the branch thats 
maintained long term.

-- Rick

On 05/14/2010 03:59 PM, Dan Magenheimer wrote:
>
> Hi Rick –
>
>
> Which exact pv_ops tree do you expect to use?  The Linux patch is 
> slightly different on each Linux version and I don’t know if/how 
> patches such as tmem will get transferred across Jeremy’s and Konrad’s 
> different pv_ops trees.
>
>
> Dan
>
> *From:* Rick Boone [mailto:rick@buzz-media.com]
> *Sent:* Friday, May 14, 2010 4:38 PM
> *To:* xen-devel@lists.xensource.com
> *Subject:* Re: [Xen-devel] TMEM + pv_ops kernel?
>
> Hey Dan,
>
> Yeah, I'm planning on using pv_ops for both the dom0 and domU kernels.
>
> Any movement towards making TMEM available would be great; it's one of 
> the key things we're looking forward to testing and using (along with 
> Remus)
>
> I'm systems engineer at a company running about 50 blogs/social 
> networking websites (http://buzz-media.com) , all on top of a heavily 
> virtualized infrastructure (Xen [v3.2] is our sole virt platform) . 
> We've got about 80 dom0's, holding about 125 domU's and on a few of 
> those boxes, especially those holding one or more of our web server 
> domU's, we run into "out of memory" crashes from time to time. So, 
> we're hoping to see if TMEM helps reduce our memory woes. We tend to 
> mix and match types of domU's within dom0's, so some boxes will have 1 
> memory-hungry web server, alongside a number of smaller, low-usage, 
> low-memory, developer boxes. From what we've read, TMEM might be 
> perfect for our setup.
>
> -- Rick
>
>
> On 05/14/2010 08:26 AM, Dan Magenheimer wrote:
>
> Hi Rick –
>
> Tmem is primarily of use in PV guests, less so in dom0.  Are you 
> putting pv_ops kernels in your PV guests (domUs)?
>
> If so, Jeremy has been a big proponent of tmem and we can work 
> together to get it into his pv_ops tree.   But I think most of the 
> focus for the pv_ops work has been to provide dom0 support.  Upstream 
> kernels already provide domU support, so I’ve been instead working on 
> the linux-kernel-mailing-list toward getting tmem patches accepted 
> upstream.
>
> If not, just using a tmem-modified dom0 will probably not help you.  
>  I have been working on tmem-modified rpm’s for various RedHat-ish 
> kernels and I know Jan Beulich has been working on putting tmem 
> support into various SuSE kernels.
>
> I’d also be interested in more information about your planned usage 
> model for tmem.
>
> Thanks,
>
> Dan
>
> *From:* Rick Boone [mailto:rick@buzz-media.com]
> *Sent:* Thursday, May 13, 2010 4:17 PM
> *To:* xen-devel@lists.xensource.com <mailto:xen-devel@lists.xensource.com>
> *Subject:* [Xen-devel] TMEM + pv_ops kernel?
>
> Hey,
>
> I'm building Jeremy's pv_ops dom0 kernel (2.6.32.12) for use with Xen 
> 4.0 and it looks like the Transcendant Memory options arent in the 
> kernel config. Is TMEM not available in pv_ops? I've been looking 
> forward to using it with Xen4, but also really wanna work with pv_ops 
> moving forward.
>
> Thanks alot,
> -- Rick
>

[-- Attachment #1.2: Type: text/html, Size: 10013 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: TMEM + pv_ops kernel?

[Pasi Kärkkäinen]

On Fri, May 14, 2010 at 03:38:24PM -0700, Rick Boone wrote:
>    Hey Dan,
> 
>    Yeah, I'm planning on using pv_ops for both the dom0 and domU kernels.
> 
>    Any movement towards making TMEM available would be great; it's one of the
>    key things we're looking forward to testing and using (along with Remus)
> 
>    I'm systems engineer at a company running about 50 blogs/social networking
>    websites ([1]http://buzz-media.com) , all on top of a heavily virtualized
>    infrastructure (Xen [v3.2] is our sole virt platform) . We've got about 80
>    dom0's, holding about 125 domU's and on a few of those boxes, especially
>    those holding one or more of our web server domU's, we run into "out of
>    memory" crashes from time to time. 
>

OOM in the guest, I assume? 

-- Pasi

Re: TMEM + pv_ops kernel?

[Rick Boone]

[-- Attachment #1.1: Type: text/plain, Size: 956 bytes --]

Hey,

Yeah, in the guest.

-- Rick

On 05/15/2010 03:22 AM, Pasi Kärkkäinen wrote:
> On Fri, May 14, 2010 at 03:38:24PM -0700, Rick Boone wrote:
>    
>>     Hey Dan,
>>
>>     Yeah, I'm planning on using pv_ops for both the dom0 and domU kernels.
>>
>>     Any movement towards making TMEM available would be great; it's one of the
>>     key things we're looking forward to testing and using (along with Remus)
>>
>>     I'm systems engineer at a company running about 50 blogs/social networking
>>     websites ([1]http://buzz-media.com) , all on top of a heavily virtualized
>>     infrastructure (Xen [v3.2] is our sole virt platform) . We've got about 80
>>     dom0's, holding about 125 domU's and on a few of those boxes, especially
>>     those holding one or more of our web server domU's, we run into "out of
>>     memory" crashes from time to time.
>>
>>      
> OOM in the guest, I assume?
>
> -- Pasi
>
>
>    

[-- Attachment #1.2: Type: text/html, Size: 1446 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: TMEM + pv_ops kernel?

[Jeremy Fitzhardinge]

On 05/14/2010 03:59 PM, Dan Magenheimer wrote:
>
> Hi Rick –
>
>
> Which exact pv_ops tree do you expect to use? The Linux patch is
> slightly different on each Linux version and I don’t know if/how
> patches such as tmem will get transferred across Jeremy’s and Konrad’s
> different pv_ops trees.
>

Since the tmem patches seem to be settling down, I was waiting a bit to
see if they get into an upstreamable state for this next merge window
(though that's getting pretty tight at this point). But if we can set up
a git branch based on 2.6.32 we should be able to easily merge it into
all the various pvops branches. Are there any tmem changes needed
between .32 and later kernels?

J

RE: TMEM + pv_ops kernel?

[Dan Magenheimer]

> Since the tmem patches seem to be settling down, I was waiting a bit to
> see if they get into an upstreamable state for this next merge window
> (though that's getting pretty tight at this point). But if we can set
> up
> a git branch based on 2.6.32 we should be able to easily merge it into
> all the various pvops branches. Are there any tmem changes needed
> between .32 and later kernels?

Sometimes the patches don't apply perfectly cleanly to later kernels,
but maybe git automatically handles merges better than "patch"?

Re: TMEM + pv_ops kernel?

[Jeremy Fitzhardinge]

On 05/15/2010 03:36 PM, Dan Magenheimer wrote:
>> Since the tmem patches seem to be settling down, I was waiting a bit to
>> see if they get into an upstreamable state for this next merge window
>> (though that's getting pretty tight at this point). But if we can set
>> up
>> a git branch based on 2.6.32 we should be able to easily merge it into
>> all the various pvops branches. Are there any tmem changes needed
>> between .32 and later kernels?
>>     
> Sometimes the patches don't apply perfectly cleanly to later kernels,
> but maybe git automatically handles merges better than "patch"?
>   

Yeah, that shouldn't be a problem.

    J