dma_declare_coherent_memory fails for RAM allocated memory

dma_declare_coherent_memory fails for RAM allocated memory

[Ivaylo Dimitrov]

Hi,

When trying to declare and use DT reserved memory region on ARM (OMAP3), 
dma_declare_coherent_memory() fails in memremap(). This is from today's 
master:

------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f800000 size: 0x700000
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[<c010bc30>] (unwind_backtrace) from [<c0109f70>] (show_stack+0x10/0x14)
[<c0109f70>] (show_stack) from [<c0126d80>] (__warn+0xcc/0xf8)
[<c0126d80>] (__warn) from [<c0126e40>] (warn_slowpath_fmt+0x34/0x44)
[<c0126e40>] (warn_slowpath_fmt) from [<c019c8d4>] (memremap+0x118/0x194)
[<c019c8d4>] (memremap) from [<c03b88fc>] 
(dma_init_coherent_memory+0x48/0x104)
[<c03b88fc>] (dma_init_coherent_memory) from [<c03b89e4>] 
(dma_declare_coherent_memory+0x2c/0x68)
[<c03b89e4>] (dma_declare_coherent_memory) from [<c0115c0c>] 
(rmem_omapfb_device_init+0x34/0x64)
[<c0115c0c>] (rmem_omapfb_device_init) from [<c045b0f8>] 
(of_reserved_mem_device_init+0x94/0xd8)
[<c045b0f8>] (of_reserved_mem_device_init) from [<c080aaf8>] 
(omapdss_init_of+0xe4/0x154)
[<c080aaf8>] (omapdss_init_of) from [<c08033f4>] 
(customize_machine+0x20/0x44)
[<c08033f4>] (customize_machine) from [<c01016b8>] 
(do_one_initcall+0xac/0x158)
[<c01016b8>] (do_one_initcall) from [<c0800d64>] 
(kernel_init_freeable+0xf8/0x1c8)
[<c0800d64>] (kernel_init_freeable) from [<c05519dc>] 
(kernel_init+0x8/0x110)
[<c05519dc>] (kernel_init) from [<c0107258>] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct 
device *dev)
{
	int dma;

	if (rmem->priv)
		return 0;

	dma = dma_declare_coherent_memory(&omap_fb_device.dev, rmem->base,
					  rmem->base, rmem->size,
					  DMA_MEMORY_MAP |
					  DMA_MEMORY_EXCLUSIVE);
	if (!(dma & DMA_MEMORY_MAP)) {
			pr_err("omapfb: dma_declare_coherent_memory failed\n");
			return -ENOMEM;
	}
	else
		rmem->priv = omap_fb_device.dev.dma_mem;

	return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
				       struct device *dev)
{
	dma_release_declared_memory(&omap_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
	.device_init    = rmem_omapfb_device_init,
	.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
	rmem->ops = &rmem_omapfb_ops;
	pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size, &rmem->base);

	return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with 
MEMREMAP_WC flag, which is disallowed for RAM IIUC.

I quickly hacked some code to fix the issue, but as memremap API is 
relatively new(esp to me), I wonder if this is the correct way to go:

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
  #include <linux/io.h>
  #include <linux/slab.h>
  #include <linux/kernel.h>
+#include <linux/memblock.h>
  #include <linux/module.h>
  #include <linux/dma-mapping.h>

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
         if (!size)
                 goto out;

-       if (flags & DMA_MEMORY_MAP)
-               mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+       if (flags & DMA_MEMORY_MAP) {
+               unsigned long map_type = memblock_is_map_memory(phys_addr) ?
+                                                MEMREMAP_WB : MEMREMAP_WC;
+
+               mem_base = memremap(phys_addr, size, map_type);
+       }
         else
                 mem_base = ioremap(phys_addr, size);
         if (!mem_base)

Does the above code looks sane? How to fix the problem if not?

Thanks,
Ivo

Re: dma_declare_coherent_memory fails for RAM allocated memory

[Brian Starkey]

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:
>Hi,
>
>When trying to declare and use DT reserved memory region on ARM 
>(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is 
>from today's master:
>
>------------[ cut here ]------------
>WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
>memremap attempted on ram 0x8f800000 size: 0x700000
>Modules linked in:
>CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
>Hardware name: Nokia RX-51 board
>[<c010bc30>] (unwind_backtrace) from [<c0109f70>] (show_stack+0x10/0x14)
>[<c0109f70>] (show_stack) from [<c0126d80>] (__warn+0xcc/0xf8)
>[<c0126d80>] (__warn) from [<c0126e40>] (warn_slowpath_fmt+0x34/0x44)
>[<c0126e40>] (warn_slowpath_fmt) from [<c019c8d4>] (memremap+0x118/0x194)
>[<c019c8d4>] (memremap) from [<c03b88fc>] 
>(dma_init_coherent_memory+0x48/0x104)
>[<c03b88fc>] (dma_init_coherent_memory) from [<c03b89e4>] 
>(dma_declare_coherent_memory+0x2c/0x68)
>[<c03b89e4>] (dma_declare_coherent_memory) from [<c0115c0c>] 
>(rmem_omapfb_device_init+0x34/0x64)
>[<c0115c0c>] (rmem_omapfb_device_init) from [<c045b0f8>] 
>(of_reserved_mem_device_init+0x94/0xd8)
>[<c045b0f8>] (of_reserved_mem_device_init) from [<c080aaf8>] 
>(omapdss_init_of+0xe4/0x154)
>[<c080aaf8>] (omapdss_init_of) from [<c08033f4>] 
>(customize_machine+0x20/0x44)
>[<c08033f4>] (customize_machine) from [<c01016b8>] 
>(do_one_initcall+0xac/0x158)
>[<c01016b8>] (do_one_initcall) from [<c0800d64>] 
>(kernel_init_freeable+0xf8/0x1c8)
>[<c0800d64>] (kernel_init_freeable) from [<c05519dc>] 
>(kernel_init+0x8/0x110)
>[<c05519dc>] (kernel_init) from [<c0107258>] (ret_from_fork+0x14/0x3c)
>---[ end trace 73a8c076df72166b ]---
>omapfb: dma_declare_coherent_memory failed
>
>
>The failing code looks like:
>.
>.
>.
>static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct 
>device *dev)
>{
>	int dma;
>
>	if (rmem->priv)
>		return 0;
>
>	dma = dma_declare_coherent_memory(&omap_fb_device.dev, rmem->base,
>					  rmem->base, rmem->size,
>					  DMA_MEMORY_MAP |
>					  DMA_MEMORY_EXCLUSIVE);
>	if (!(dma & DMA_MEMORY_MAP)) {
>			pr_err("omapfb: dma_declare_coherent_memory failed\n");
>			return -ENOMEM;
>	}
>	else
>		rmem->priv = omap_fb_device.dev.dma_mem;
>
>	return 0;
>}
>
>static void rmem_omapfb_device_release(struct reserved_mem *rmem,
>				       struct device *dev)
>{
>	dma_release_declared_memory(&omap_fb_device.dev);
>}
>
>static const struct reserved_mem_ops rmem_omapfb_ops = {
>	.device_init    = rmem_omapfb_device_init,
>	.device_release = rmem_omapfb_device_release,
>};
>
>static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
>{
>	rmem->ops = &rmem_omapfb_ops;
>	pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size, &rmem->base);
>
>	return 0;
>}
>
>RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);
>
>
>It turns out that dma_init_coherent_memory calls memremap with 
>MEMREMAP_WC flag, which is disallowed for RAM IIUC.
>

Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.

>I quickly hacked some code to fix the issue, but as memremap API is 
>relatively new(esp to me), I wonder if this is the correct way to go:
>
>diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
>index bdf28f7..04b1687 100644
>--- a/drivers/base/dma-coherent.c
>+++ b/drivers/base/dma-coherent.c
>@@ -5,6 +5,7 @@
> #include <linux/io.h>
> #include <linux/slab.h>
> #include <linux/kernel.h>
>+#include <linux/memblock.h>
> #include <linux/module.h>
> #include <linux/dma-mapping.h>
>
>@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
>        if (!size)
>                goto out;
>
>-       if (flags & DMA_MEMORY_MAP)
>-               mem_base = memremap(phys_addr, size, MEMREMAP_WC);
>+       if (flags & DMA_MEMORY_MAP) {
>+               unsigned long map_type = memblock_is_map_memory(phys_addr) ?
>+                                                MEMREMAP_WB : MEMREMAP_WC;
>+
>+               mem_base = memremap(phys_addr, size, map_type);
>+       }
>        else
>                mem_base = ioremap(phys_addr, size);
>        if (!mem_base)
>
>Does the above code looks sane? How to fix the problem if not?

AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.

Cheers,
-Brian

>
>Thanks,
>Ivo
>

Re: dma_declare_coherent_memory fails for RAM allocated memory

[Ivaylo Dimitrov]

Hi,

On  1.06.2016 14:18, Brian Starkey wrote:
> Hi Ivo,
>
> On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:
>> Hi,
>>
>> When trying to declare and use DT reserved memory region on ARM
>> (OMAP3), dma_declare_coherent_memory() fails in memremap(). This is
>> from today's master:
>>
>> ------------[ cut here ]------------
>> WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
>> memremap attempted on ram 0x8f800000 size: 0x700000
>> Modules linked in:
>> CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
>> Hardware name: Nokia RX-51 board
>> [<c010bc30>] (unwind_backtrace) from [<c0109f70>] (show_stack+0x10/0x14)
>> [<c0109f70>] (show_stack) from [<c0126d80>] (__warn+0xcc/0xf8)
>> [<c0126d80>] (__warn) from [<c0126e40>] (warn_slowpath_fmt+0x34/0x44)
>> [<c0126e40>] (warn_slowpath_fmt) from [<c019c8d4>] (memremap+0x118/0x194)
>> [<c019c8d4>] (memremap) from [<c03b88fc>]
>> (dma_init_coherent_memory+0x48/0x104)
>> [<c03b88fc>] (dma_init_coherent_memory) from [<c03b89e4>]
>> (dma_declare_coherent_memory+0x2c/0x68)
>> [<c03b89e4>] (dma_declare_coherent_memory) from [<c0115c0c>]
>> (rmem_omapfb_device_init+0x34/0x64)
>> [<c0115c0c>] (rmem_omapfb_device_init) from [<c045b0f8>]
>> (of_reserved_mem_device_init+0x94/0xd8)
>> [<c045b0f8>] (of_reserved_mem_device_init) from [<c080aaf8>]
>> (omapdss_init_of+0xe4/0x154)
>> [<c080aaf8>] (omapdss_init_of) from [<c08033f4>]
>> (customize_machine+0x20/0x44)
>> [<c08033f4>] (customize_machine) from [<c01016b8>]
>> (do_one_initcall+0xac/0x158)
>> [<c01016b8>] (do_one_initcall) from [<c0800d64>]
>> (kernel_init_freeable+0xf8/0x1c8)
>> [<c0800d64>] (kernel_init_freeable) from [<c05519dc>]
>> (kernel_init+0x8/0x110)
>> [<c05519dc>] (kernel_init) from [<c0107258>] (ret_from_fork+0x14/0x3c)
>> ---[ end trace 73a8c076df72166b ]---
>> omapfb: dma_declare_coherent_memory failed
>>
>>
>> The failing code looks like:
>> .
>> .
>> .
>> static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct
>> device *dev)
>> {
>>     int dma;
>>
>>     if (rmem->priv)
>>         return 0;
>>
>>     dma = dma_declare_coherent_memory(&omap_fb_device.dev, rmem->base,
>>                       rmem->base, rmem->size,
>>                       DMA_MEMORY_MAP |
>>                       DMA_MEMORY_EXCLUSIVE);
>>     if (!(dma & DMA_MEMORY_MAP)) {
>>             pr_err("omapfb: dma_declare_coherent_memory failed\n");
>>             return -ENOMEM;
>>     }
>>     else
>>         rmem->priv = omap_fb_device.dev.dma_mem;
>>
>>     return 0;
>> }
>>
>> static void rmem_omapfb_device_release(struct reserved_mem *rmem,
>>                        struct device *dev)
>> {
>>     dma_release_declared_memory(&omap_fb_device.dev);
>> }
>>
>> static const struct reserved_mem_ops rmem_omapfb_ops = {
>>     .device_init    = rmem_omapfb_device_init,
>>     .device_release = rmem_omapfb_device_release,
>> };
>>
>> static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
>> {
>>     rmem->ops = &rmem_omapfb_ops;
>>     pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size,
>> &rmem->base);
>>
>>     return 0;
>> }
>>
>> RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);
>>
>>
>> It turns out that dma_init_coherent_memory calls memremap with
>> MEMREMAP_WC flag, which is disallowed for RAM IIUC.
>>
>
> Right. If you want to use a memory region as coherent DMA memory, then
> that same region can't be System RAM, because System RAM is usually
> mapped in a non-coherent fashion.
>
>> I quickly hacked some code to fix the issue, but as memremap API is
>> relatively new(esp to me), I wonder if this is the correct way to go:
>>
>> diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
>> index bdf28f7..04b1687 100644
>> --- a/drivers/base/dma-coherent.c
>> +++ b/drivers/base/dma-coherent.c
>> @@ -5,6 +5,7 @@
>> #include <linux/io.h>
>> #include <linux/slab.h>
>> #include <linux/kernel.h>
>> +#include <linux/memblock.h>
>> #include <linux/module.h>
>> #include <linux/dma-mapping.h>
>>
>> @@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
>>        if (!size)
>>                goto out;
>>
>> -       if (flags & DMA_MEMORY_MAP)
>> -               mem_base = memremap(phys_addr, size, MEMREMAP_WC);
>> +       if (flags & DMA_MEMORY_MAP) {
>> +               unsigned long map_type =
>> memblock_is_map_memory(phys_addr) ?
>> +                                                MEMREMAP_WB :
>> MEMREMAP_WC;
>> +
>> +               mem_base = memremap(phys_addr, size, map_type);
>> +       }
>>        else
>>                mem_base = ioremap(phys_addr, size);
>>        if (!mem_base)
>>
>> Does the above code looks sane? How to fix the problem if not?
>
> AFAIK dma_declare_coherent_memory() should only be used on physical
> addresses which don't already have a CPU virtual mapping. Certainly
> returning (potentially) cached memory as DMA coherent is asking for
> trouble.
>
> I think what you want to do is add a "no-map;" property to your
> reserved-memory region in DT. That will stop the kernel from using it
> as System RAM, leaving it free for your framebuffer.
>

I tried to use "no-map", but for some reason it doesn't allow me to 
allocate more than 2MB, anything above results in:

qemu: fatal: Trying to execute code outside RAM or ROM at 0xffff0010

R00=22222220 R01=cf901000 R02=cfaffff4 R03=e0000000
R04=0008ff80 R05=00090000 R06=00000000 R07=c096333c
R08=c0904dc0 R09=c093a23c R10=00000001 R11=c0727b3b
R12=0000ff80 R13=c093af8c R14=c082aba0 R15=ffff0010
PSR=a00001d7 N-C- A abt32
Aborted (core dumped)

I was able to track the code to memblock_isolate_range() - if I return 
-ENOMEM on http://lxr.free-electrons.com/source/mm/memblock.c#L671, the 
above crash does not occur(however the allocation fails as well). 
Putting some printks doesn't reveal anything suspicious to me:

for 2MB:

[    0.000000] base 8fe00000, size 200000, rgn->base 80000000, rgn->size 
10000000, type->total_size 10000000
[    0.000000] base - rbase fe00000


for 4MB:

[    0.000000] base 8fc00000, size 400000, rgn->base 80000000, rgn->size 
10000000, type->total_size 10000000
[    0.000000] base - rbase fc00000


However, this is where my limited memblock knowledge ends, so any help 
is appreciated.

Maybe I should CC memblock authors as well?

Thanks,
Ivo

Re: dma_declare_coherent_memory fails for RAM allocated memory

[Ivaylo Dimitrov]

On  2.06.2016 20:14, Ivaylo Dimitrov wrote:
> Hi,
>
> On  1.06.2016 14:18, Brian Starkey wrote:
>> Hi Ivo,
>>
>> On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:
>>> Hi,
>>>
>>> When trying to declare and use DT reserved memory region on ARM
>>> (OMAP3), dma_declare_coherent_memory() fails in memremap(). This is
>>> from today's master:
>>>
>>> ------------[ cut here ]------------
>>> WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
>>> memremap attempted on ram 0x8f800000 size: 0x700000
>>> Modules linked in:
>>> CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
>>> Hardware name: Nokia RX-51 board
>>> [<c010bc30>] (unwind_backtrace) from [<c0109f70>] (show_stack+0x10/0x14)
>>> [<c0109f70>] (show_stack) from [<c0126d80>] (__warn+0xcc/0xf8)
>>> [<c0126d80>] (__warn) from [<c0126e40>] (warn_slowpath_fmt+0x34/0x44)
>>> [<c0126e40>] (warn_slowpath_fmt) from [<c019c8d4>]
>>> (memremap+0x118/0x194)
>>> [<c019c8d4>] (memremap) from [<c03b88fc>]
>>> (dma_init_coherent_memory+0x48/0x104)
>>> [<c03b88fc>] (dma_init_coherent_memory) from [<c03b89e4>]
>>> (dma_declare_coherent_memory+0x2c/0x68)
>>> [<c03b89e4>] (dma_declare_coherent_memory) from [<c0115c0c>]
>>> (rmem_omapfb_device_init+0x34/0x64)
>>> [<c0115c0c>] (rmem_omapfb_device_init) from [<c045b0f8>]
>>> (of_reserved_mem_device_init+0x94/0xd8)
>>> [<c045b0f8>] (of_reserved_mem_device_init) from [<c080aaf8>]
>>> (omapdss_init_of+0xe4/0x154)
>>> [<c080aaf8>] (omapdss_init_of) from [<c08033f4>]
>>> (customize_machine+0x20/0x44)
>>> [<c08033f4>] (customize_machine) from [<c01016b8>]
>>> (do_one_initcall+0xac/0x158)
>>> [<c01016b8>] (do_one_initcall) from [<c0800d64>]
>>> (kernel_init_freeable+0xf8/0x1c8)
>>> [<c0800d64>] (kernel_init_freeable) from [<c05519dc>]
>>> (kernel_init+0x8/0x110)
>>> [<c05519dc>] (kernel_init) from [<c0107258>] (ret_from_fork+0x14/0x3c)
>>> ---[ end trace 73a8c076df72166b ]---
>>> omapfb: dma_declare_coherent_memory failed
>>>
>>>
>>> The failing code looks like:
>>> .
>>> .
>>> .
>>> static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct
>>> device *dev)
>>> {
>>>     int dma;
>>>
>>>     if (rmem->priv)
>>>         return 0;
>>>
>>>     dma = dma_declare_coherent_memory(&omap_fb_device.dev, rmem->base,
>>>                       rmem->base, rmem->size,
>>>                       DMA_MEMORY_MAP |
>>>                       DMA_MEMORY_EXCLUSIVE);
>>>     if (!(dma & DMA_MEMORY_MAP)) {
>>>             pr_err("omapfb: dma_declare_coherent_memory failed\n");
>>>             return -ENOMEM;
>>>     }
>>>     else
>>>         rmem->priv = omap_fb_device.dev.dma_mem;
>>>
>>>     return 0;
>>> }
>>>
>>> static void rmem_omapfb_device_release(struct reserved_mem *rmem,
>>>                        struct device *dev)
>>> {
>>>     dma_release_declared_memory(&omap_fb_device.dev);
>>> }
>>>
>>> static const struct reserved_mem_ops rmem_omapfb_ops = {
>>>     .device_init    = rmem_omapfb_device_init,
>>>     .device_release = rmem_omapfb_device_release,
>>> };
>>>
>>> static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
>>> {
>>>     rmem->ops = &rmem_omapfb_ops;
>>>     pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size,
>>> &rmem->base);
>>>
>>>     return 0;
>>> }
>>>
>>> RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);
>>>
>>>
>>> It turns out that dma_init_coherent_memory calls memremap with
>>> MEMREMAP_WC flag, which is disallowed for RAM IIUC.
>>>
>>
>> Right. If you want to use a memory region as coherent DMA memory, then
>> that same region can't be System RAM, because System RAM is usually
>> mapped in a non-coherent fashion.
>>
>>> I quickly hacked some code to fix the issue, but as memremap API is
>>> relatively new(esp to me), I wonder if this is the correct way to go:
>>>
>>> diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
>>> index bdf28f7..04b1687 100644
>>> --- a/drivers/base/dma-coherent.c
>>> +++ b/drivers/base/dma-coherent.c
>>> @@ -5,6 +5,7 @@
>>> #include <linux/io.h>
>>> #include <linux/slab.h>
>>> #include <linux/kernel.h>
>>> +#include <linux/memblock.h>
>>> #include <linux/module.h>
>>> #include <linux/dma-mapping.h>
>>>
>>> @@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
>>>        if (!size)
>>>                goto out;
>>>
>>> -       if (flags & DMA_MEMORY_MAP)
>>> -               mem_base = memremap(phys_addr, size, MEMREMAP_WC);
>>> +       if (flags & DMA_MEMORY_MAP) {
>>> +               unsigned long map_type =
>>> memblock_is_map_memory(phys_addr) ?
>>> +                                                MEMREMAP_WB :
>>> MEMREMAP_WC;
>>> +
>>> +               mem_base = memremap(phys_addr, size, map_type);
>>> +       }
>>>        else
>>>                mem_base = ioremap(phys_addr, size);
>>>        if (!mem_base)
>>>
>>> Does the above code looks sane? How to fix the problem if not?
>>
>> AFAIK dma_declare_coherent_memory() should only be used on physical
>> addresses which don't already have a CPU virtual mapping. Certainly
>> returning (potentially) cached memory as DMA coherent is asking for
>> trouble.
>>
>> I think what you want to do is add a "no-map;" property to your
>> reserved-memory region in DT. That will stop the kernel from using it
>> as System RAM, leaving it free for your framebuffer.
>>
>
> I tried to use "no-map", but for some reason it doesn't allow me to
> allocate more than 2MB, anything above results in:
>
> qemu: fatal: Trying to execute code outside RAM or ROM at 0xffff0010
>
> R00=22222220 R01=cf901000 R02=cfaffff4 R03=e0000000
> R04=0008ff80 R05=00090000 R06=00000000 R07=c096333c
> R08=c0904dc0 R09=c093a23c R10=00000001 R11=c0727b3b
> R12=0000ff80 R13=c093af8c R14=c082aba0 R15=ffff0010
> PSR=a00001d7 N-C- A abt32
> Aborted (core dumped)
>
> I was able to track the code to memblock_isolate_range() - if I return
> -ENOMEM on http://lxr.free-electrons.com/source/mm/memblock.c#L671, the
> above crash does not occur(however the allocation fails as well).
> Putting some printks doesn't reveal anything suspicious to me:
>
> for 2MB:
>
> [    0.000000] base 8fe00000, size 200000, rgn->base 80000000, rgn->size
> 10000000, type->total_size 10000000
> [    0.000000] base - rbase fe00000
>
>
> for 4MB:
>
> [    0.000000] base 8fc00000, size 400000, rgn->base 80000000, rgn->size
> 10000000, type->total_size 10000000
> [    0.000000] base - rbase fc00000
>
>
> However, this is where my limited memblock knowledge ends, so any help
> is appreciated.
>

It turned out I have to increase CONFIG_FORCE_MAX_ZONEORDER in order to 
be able to use more than 2MB of allocated memory.

Ivo