Re: [PATCH/RFCv4 0/6] The Contiguous Memory Allocator framework

[Peter Zijlstra <peterz@infradead.org>]

On Sat, 2010-08-28 at 15:08 +0200, Hans Verkuil wrote:

> > That would be good.  Although I expect that the allocation would need
> > to be 100% rock-solid reliable, otherwise the end user has a
> > non-functioning device.
> 
> Yes, indeed. And you have to be careful as well how you move pages around.
> Say that you have a capture and an output v4l device: the first one needs
> 64 MB contiguous memory and so it allocates that amount, moving pages around
> as needed. Once allocated that memory is pinned in place since it is needed
> for DMA. So if the output device also needs 64 MB, then you must have a
> guarantee that the first allocation didn't fragment the available contiguous
> memory.

Isn't the proposed CMA thing vulnerable to the exact same problem? If
you allow sharing of regions and plug some allocator in there you get
the same problem. If you can solve it there, you can solve it for any
kind of reservation scheme.

> I also wonder how expensive it is to move all the pages around. E.g. if you
> have a digital camera and want to make a hires picture, then it wouldn't
> do if it takes a second to move all the pages around making room for the
> captured picture. The CPUs in many SoCs are not very powerful compared to
> your average desktop.

Well, that's a trade-off, if you want to have the memory be usable for
anything else (which I understood people did want) then you have to pay
for cleaning it up when you need to use it.

As for the cost of compaction vs regular page-out of random page-cache
memory, compaction is actually cheaper, since it doesn't need to write
out dirty data, and page-out driven writeback sucks due to the
non-linear nature of it.

> And how would memory allocations in specific memory ranges (e.g. memory
> banks) work?

Make sure you reserve pageblocks in the desired range.

> Note also that these issues are not limited to embedded systems, also PCI(e)
> boards can sometimes require massive amounts of DMA-able memory. I have had
> this happen in the past with the ivtv driver with customers that had 15 or so
> capture cards in one box. And I'm sure it will happen in the future as well,
> esp. with upcoming 4k video formats.

I would sincerely hope PCI(e) devices come with an IOMMU (and all memory
lines wired up), really, any hardware that doesn't isn't worth the
silicon its engraved in. Just don't buy it.