CMA question

CMA question

[Johannes Thoma]

Dear Kernel hackers,

As far as I understood CMA (contiguous memory allocation) memory is used 
for other purposes as long it isn't requested via cma_alloc() by a 
driver. cma_alloc then tries to free the memory by relocating it and 
returns the contiguous area.

I have a case where cma_alloc() sometimes fails to relocate the memory 
which causes my driver (a GPU driver) to fail starting up (the driver is 
started when the system is running for a while). Is there a way to 
prevent the CM allocator to use the memory for any other purpose? Or 
should I better use another mechanism to allocate  contiguous memory at 
boot time (is there a framework for doing so)?

PS: I once had a patch which tries harder to relocate the pages (I 
patched cma_alloc()), but it didn't really work well, blocking the 
allocation for seconds.

Thanks for your advice,

- Johannes

CMA question

[Greg KH]

On Sun, Feb 26, 2017 at 05:29:05PM +0100, Johannes Thoma wrote:
> Dear Kernel hackers,
> 
> As far as I understood CMA (contiguous memory allocation) memory is used 
> for other purposes as long it isn't requested via cma_alloc() by a 
> driver. cma_alloc then tries to free the memory by relocating it and 
> returns the contiguous area.
> 
> I have a case where cma_alloc() sometimes fails to relocate the memory 
> which causes my driver (a GPU driver) to fail starting up (the driver is 
> started when the system is running for a while). Is there a way to 
> prevent the CM allocator to use the memory for any other purpose? Or 
> should I better use another mechanism to allocate  contiguous memory at 
> boot time (is there a framework for doing so)?

If at boot time you don't have enough memory, something is really wrong
and your driver shouldn't work, that's to be expected.  Try starting it
earlier in the boot process.

Do you have a pointer to your driver anywhere so we could see if you are
doing something odd with the cma interface?

thanks,

greg k-h

CMA question

[Johannes Thoma]

Dear Greg,

Thank you for your quick response.

Am 26.02.17 um 17:31 schrieb Greg KH:
> On Sun, Feb 26, 2017 at 05:29:05PM +0100, Johannes Thoma wrote:
>> Dear Kernel hackers,
>>
>> As far as I understood CMA (contiguous memory allocation) memory is used
>> for other purposes as long it isn't requested via cma_alloc() by a
>> driver. cma_alloc then tries to free the memory by relocating it and
>> returns the contiguous area.
>>
>> I have a case where cma_alloc() sometimes fails to relocate the memory
>> which causes my driver (a GPU driver) to fail starting up (the driver is
>> started when the system is running for a while). Is there a way to
>> prevent the CM allocator to use the memory for any other purpose? Or
>> should I better use another mechanism to allocate  contiguous memory at
>> boot time (is there a framework for doing so)?
>
> If at boot time you don't have enough memory, something is really wrong
> and your driver shouldn't work, that's to be expected.  Try starting it
> earlier in the boot process.
>
Maybe that is the problem. The call to cma_alloc() happens in a
driver-specific API function which is called when my application starts
(at uptime 30 seconds it always succeeded (at least I didn't observe any
failures), however when I restart the application later and call
cma_alloc() again (the memory was freed meanwhile) it sometimes fails).

> Do you have a pointer to your driver anywhere so we could see if you are
> doing something odd with the cma interface?
>

The driver is part of the imx kernel from boundary devices and can be found here:

https://github.com/boundarydevices/linux-imx6/blob/boundary-imx_4.1.15_2.0.0_ga/drivers/mxc/gpu-viv

The call to dma_alloc_writecombine (which eventually calls cma_alloc() if I am not
completely wrong) can be found in this file:

?https://github.com/boundarydevices/linux-imx6/blob/boundary-imx_4.1.15_2.0.0_ga/drivers/mxc/gpu-viv/hal/os/linux/kernel/allocator/freescale/gc_hal_kernel_allocator_cma.c

Note that I didn't write the driver, I am just using it (it seems to be a port from
another OS to Linux to me).

As you pointed out the solution would be to allocate the memory earlier in the
boot process, by modifying the driver. I will try that in the next few days and
let you know the result.

Best,

- Johannes

CMA question

[Thomas Petazzoni]

Hello,

On Sun, 26 Feb 2017 18:14:42 +0100, Johannes Thoma wrote:

> As you pointed out the solution would be to allocate the memory earlier in the
> boot process, by modifying the driver. I will try that in the next few days and
> let you know the result.

The whole point of CMA is that allocating earlier in the boot process
should not be necessary. CMA guarantees that the memory reserved for
the CMA pool is "movable", i.e it can be discarded when a CMA
allocation is done.

So if a CMA allocation fails, I would rather suggest that the size of
your CMA pool is not large enough. Check your Device Tree and/or kernel
command (you can specify the size of the CMA pool on both if I remember
correctly).

Best regarsd,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

CMA question

[Johannes Thoma]

Hello,

Am 28.02.17 um 11:04 schrieb Thomas Petazzoni:
> Hello,
>
> On Sun, 26 Feb 2017 18:14:42 +0100, Johannes Thoma wrote:
>
>> As you pointed out the solution would be to allocate the memory earlier in the
>> boot process, by modifying the driver. I will try that in the next few days and
>> let you know the result.
>
> The whole point of CMA is that allocating earlier in the boot process
> should not be necessary. CMA guarantees that the memory reserved for
> the CMA pool is "movable", i.e it can be discarded when a CMA
> allocation is done.
>
Yes it is moveable, but it takes rather long until it is moved (up to
20 Seconds). To test this I've implemented a loop that restarts from
beginning of the CMA area as long as alloc_contig_range() returns
-EBUSY (something similar to
https://www.mail-archive.com/kernelnewbies at kernelnewbies.org/msg16956.html
). The result (on my system, which is an ARM based IMX-6 SOC) is that
after some while it always succeeds but it takes some time.

> So if a CMA allocation fails, I would rather suggest that the size of
> your CMA pool is not large enough. Check your Device Tree and/or kernel
> command (you can specify the size of the CMA pool on both if I remember
> correctly).
>
We use kernel command line to set the CMA size, so I will try as you
suggested.

Thanks a lot,

- Johannes