Re: [PATCH 0/3] KVM: PPC: Book3S HV: More flexible allocator for linear memory

Re: [PATCH 0/3] KVM: PPC: Book3S HV: More flexible allocator for linear memory

[Alexander Graf]

On 14.09.2012, at 10:11, Paul Mackerras wrote:

> On Fri, Sep 14, 2012 at 01:32:23AM +0200, Alexander Graf wrote:
>> 
>> On 12.09.2012, at 02:34, Paul Mackerras wrote:
>> 
>>> This series of 3 patches makes it possible for guests to allocate
>>> whatever size of HPT they need from linear memory preallocated at
>>> boot, rather than being restricted to a single size of HPT (by
>>> default, 16MB) and having to use the kernel page allocator for
>>> anything else -- which in practice limits them to at most 16MB given
>>> the default value for the maximum page order.  Instead of allocating
>>> many individual pieces of memory, this allocates a single contiguous
>>> area and uses a simple bitmap-based allocator to hand out pieces of it
>>> as required.
>> 
>> Have you tried to play with CMA for this? It sounds like it could buy us exactly what we need.
> 
> Interesting, I hadn't noticed that there.  I had a bit of a look at
> it, and it's certainly in the right general direction, however it
> would need some changes to do what we need.  It limits the alignment
> to at most 512 pages, i.e. 2MB with 4k pages or 32MB with 64k pages,
> but we need RMAs of 64MB to 256MB for PPC970 and they have to be
> aligned on their size, as do the HPTs for PPC970.
> 
> Secondly, it has a link with the page allocator that I don't fully
> understand, but it seems from the comments in alloc_contig_range()
> (mm/page_alloc.c) that you can allocate at most MAX_ORDER_NR_PAGES
> pages at once, and that defaults to 16MB for ppc64, which isn't nearly
> enough.  If that's true then it would make it unusable for this.

So do you think it makes more sense to reimplement a large page allocator in KVM, as this patch set does, or improve CMA to get us really big chunks of linear memory?

Let's ask the Linux mm guys too :). Maybe they have an idea.


Alex

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/3] KVM: PPC: Book3S HV: More flexible allocator for linear memory

[Paul Mackerras]

On Fri, Sep 14, 2012 at 02:13:37PM +0200, Alexander Graf wrote:

> So do you think it makes more sense to reimplement a large page allocator in KVM, as this patch set does, or improve CMA to get us really big chunks of linear memory?
> 
> Let's ask the Linux mm guys too :). Maybe they have an idea.

I asked the authors of CMA, and apparently it's not limited to
MAX_ORDER as I feared.  It has the advantage that the memory can be
used for other things such as page cache when it's not needed, but not
for immovable allocations such as kmalloc.  I'm going to try it out.
It will need a patch to increase the maximum alignment it allows.

Paul.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/3] KVM: PPC: Book3S HV: More flexible allocator for linear memory

[Alexander Graf]

On 14.09.2012, at 14:45, Paul Mackerras wrote:

> On Fri, Sep 14, 2012 at 02:13:37PM +0200, Alexander Graf wrote:
> 
>> So do you think it makes more sense to reimplement a large page allocator in KVM, as this patch set does, or improve CMA to get us really big chunks of linear memory?
>> 
>> Let's ask the Linux mm guys too :). Maybe they have an idea.
> 
> I asked the authors of CMA, and apparently it's not limited to
> MAX_ORDER as I feared.  It has the advantage that the memory can be
> used for other things such as page cache when it's not needed, but not
> for immovable allocations such as kmalloc.  I'm going to try it out.
> It will need a patch to increase the maximum alignment it allows.

Awesome. Thanks a lot. I'd really prefer if we can stick to generic Linux solutions rather than invent our own :).


Alex

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/3] KVM: PPC: Book3S HV: More flexible allocator for linear memory

[Paul Mackerras]

On Fri, Sep 14, 2012 at 03:15:32PM +0200, Alexander Graf wrote:
> 
> On 14.09.2012, at 14:45, Paul Mackerras wrote:
> 
> > On Fri, Sep 14, 2012 at 02:13:37PM +0200, Alexander Graf wrote:
> > 
> >> So do you think it makes more sense to reimplement a large page allocator in KVM, as this patch set does, or improve CMA to get us really big chunks of linear memory?
> >> 
> >> Let's ask the Linux mm guys too :). Maybe they have an idea.
> > 
> > I asked the authors of CMA, and apparently it's not limited to
> > MAX_ORDER as I feared.  It has the advantage that the memory can be
> > used for other things such as page cache when it's not needed, but not
> > for immovable allocations such as kmalloc.  I'm going to try it out.
> > It will need a patch to increase the maximum alignment it allows.
> 
> Awesome. Thanks a lot. I'd really prefer if we can stick to generic Linux solutions rather than invent our own :).

Turns out there is a difficulty with this.  When we have a guest page
that we want to pin in memory, and that page happens to have been
allocated within the CMA region, we would need to migrate it out of
the CMA region before pinning it, since otherwise it would reduce the
amount of contiguous memory available.  But it appears that there
isn't any way to do that.

Paul.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/3] KVM: PPC: Book3S HV: More flexible allocator for linear memory

[Alexander Graf]

On 26.10.2012, at 03:17, Paul Mackerras wrote:

> On Fri, Sep 14, 2012 at 03:15:32PM +0200, Alexander Graf wrote:
>> 
>> On 14.09.2012, at 14:45, Paul Mackerras wrote:
>> 
>>> On Fri, Sep 14, 2012 at 02:13:37PM +0200, Alexander Graf wrote:
>>> 
>>>> So do you think it makes more sense to reimplement a large page allocator in KVM, as this patch set does, or improve CMA to get us really big chunks of linear memory?
>>>> 
>>>> Let's ask the Linux mm guys too :). Maybe they have an idea.
>>> 
>>> I asked the authors of CMA, and apparently it's not limited to
>>> MAX_ORDER as I feared.  It has the advantage that the memory can be
>>> used for other things such as page cache when it's not needed, but not
>>> for immovable allocations such as kmalloc.  I'm going to try it out.
>>> It will need a patch to increase the maximum alignment it allows.
>> 
>> Awesome. Thanks a lot. I'd really prefer if we can stick to generic Linux solutions rather than invent our own :).
> 
> Turns out there is a difficulty with this.  When we have a guest page
> that we want to pin in memory, and that page happens to have been
> allocated within the CMA region, we would need to migrate it out of
> the CMA region before pinning it, since otherwise it would reduce the
> amount of contiguous memory available.  But it appears that there
> isn't any way to do that.

How does this work for other users of CMA? I can't possibly believe that we only ever want a static amount of contiguous memory on the system.


Alex

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>