[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Gregory CLEMENT]

For Armada 370/XP we have the same problem that for the commit
cb01b63, so we applied the same solution: "The default 256 KiB
coherent pool may be too small for some of the Kirkwood devices, so
increase it to make sure that devices will be able to allocate their
buffers with GFP_ATOMIC flag"

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/arm/mach-mvebu/armada-370-xp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
index 2af6ce5..cbad821 100644
--- a/arch/arm/mach-mvebu/armada-370-xp.c
+++ b/arch/arm/mach-mvebu/armada-370-xp.c
@@ -17,6 +17,7 @@
 #include <linux/of_platform.h>
 #include <linux/io.h>
 #include <linux/time-armada-370-xp.h>
+#include <linux/dma-mapping.h>
 #include <asm/mach/arch.h>
 #include <asm/mach/map.h>
 #include <asm/mach/time.h>
@@ -43,6 +44,16 @@ void __init armada_370_xp_timer_and_clk_init(void)
 	armada_370_xp_timer_init();
 }
 
+void __init armada_370_xp_init_early(void)
+{
+	/*
+	 * Some Armada 370/XP devices allocate their coherent buffers
+	 * from atomic context. Increase size of atomic coherent pool
+	 * to make sure such the allocations won't fail.
+	 */
+	init_dma_coherent_pool_size(SZ_1M);
+}
+
 struct sys_timer armada_370_xp_timer = {
 	.init		= armada_370_xp_timer_and_clk_init,
 };
@@ -61,6 +72,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
 DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
 	.init_machine	= armada_370_xp_dt_init,
 	.map_io		= armada_370_xp_map_io,
+	.init_early	= armada_370_xp_init_early,
 	.init_irq	= armada_370_xp_init_irq,
 	.handle_irq     = armada_370_xp_handle_irq,
 	.timer		= &armada_370_xp_timer,
-- 
1.7.9.5

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sören Moch]

resent as plain text, sorry.


 > For Armada 370/XP we have the same problem that for the commit
 > cb01b63, so we applied the same solution: "The default 256 KiB
 > coherent pool may be too small for some of the Kirkwood devices, so
 > increase it to make sure that devices will be able to allocate their
 > buffers with GFP_ATOMIC flag"

I see a regression from linux-3.5 to linux-3.6 and think there might be 
a fundamental problem
with this patch. On my Kirkwood system (guruplug server plus) with 
linux-3.6.2 I see following
errors and corresponding malfunction even with further increased (2M, 
4M) pool size:

Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent pool is 
too small!
Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool= 
kernel parameter!

So I had to downgrade to linux-3.5 which is running without problems.

I use SATA and several DVB sticks (em28xx / drxk and dib0700).

I already sent this bug report to m.szyprowski and 
gregory.clementearlier, but
Sebastian suggested to send it to the mailing list and the responsible 
maintainers, too.

Please write to my e-mail directly if you have further questions or 
patches for me to test.

Regards,
Soeren

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Andrew Lunn]

On Tue, Nov 06, 2012 at 10:28:45PM +0100, S?ren Moch wrote:
> resent as plain text, sorry.
> 
> 
> > For Armada 370/XP we have the same problem that for the commit
> > cb01b63, so we applied the same solution: "The default 256 KiB
> > coherent pool may be too small for some of the Kirkwood devices, so
> > increase it to make sure that devices will be able to allocate their
> > buffers with GFP_ATOMIC flag"
> 
> I see a regression from linux-3.5 to linux-3.6 and think there might
> be a fundamental problem
> with this patch. On my Kirkwood system (guruplug server plus) with
> linux-3.6.2 I see following
> errors and corresponding malfunction even with further increased
> (2M, 4M) pool size:
> 
> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
> pool is too small!
> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
> kernel parameter!
> 
> So I had to downgrade to linux-3.5 which is running without problems.
> 
> I use SATA and several DVB sticks (em28xx / drxk and dib0700).

I'm guess its the DVB sticks which are causing the problems. We have a
number of kirkwood devices with two SATA devices which had problems
until we extended the coherent_pool. The DVB sticks are probably take
more coherent RAM. There was also an issue found recently:

http://www.spinics.net/lists/arm-kernel/msg203962.html

That conversation has gone quiet, but that could be because the
participants are at ELCE.

	     Andrew

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sebastian Hesselbarth]

On 11/06/2012 11:32 PM, Andrew Lunn wrote:
> On Tue, Nov 06, 2012 at 10:28:45PM +0100, S?ren Moch wrote:
>> I see a regression from linux-3.5 to linux-3.6 and think there might
>> be a fundamental problem
>> with this patch. On my Kirkwood system (guruplug server plus) with
>> linux-3.6.2 I see following
>> errors and corresponding malfunction even with further increased
>> (2M, 4M) pool size:
>>
>> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
>> pool is too small!
>> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
>> kernel parameter!
>>
>> So I had to downgrade to linux-3.5 which is running without problems.
>>
>> I use SATA and several DVB sticks (em28xx / drxk and dib0700).
>
> I'm guess its the DVB sticks which are causing the problems. We have a
> number of kirkwood devices with two SATA devices which had problems
> until we extended the coherent_pool. The DVB sticks are probably take
> more coherent RAM. There was also an issue found recently:
>
> http://www.spinics.net/lists/arm-kernel/msg203962.html
>
> That conversation has gone quiet, but that could be because the
> participants are at ELCE.

So what is the call here? Should we just increase the coherent buffer
size back to what it was before? I am not into this too much but just
increasing the buffer will just postpone the actual issue to a later
point in running the kernel?

Sebastian

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sören Moch]

 >>> For Armada 370/XP we have the same problem that for the commit
 >>> cb01b63, so we applied the same solution: "The default 256 KiB
 >>> coherent pool may be too small for some of the Kirkwood devices, so
 >>> increase it to make sure that devices will be able to allocate their
 >>> buffers with GFP_ATOMIC flag"
 >>
 >> I see a regression from linux-3.5 to linux-3.6 and think there might
 >> be a fundamental problem
 >> with this patch. On my Kirkwood system (guruplug server plus) with
 >> linux-3.6.2 I see following
 >> errors and corresponding malfunction even with further increased
 >> (2M, 4M) pool size:
 >>
 >> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
 >> pool is too small!
 >> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
 >> kernel parameter!
 >>
 >> So I had to downgrade to linux-3.5 which is running without problems.
 >>
 >> I use SATA and several DVB sticks (em28xx / drxk and dib0700).
 >
 > I'm guess its the DVB sticks which are causing the problems. We have a
 > number of kirkwood devices with two SATA devices which had problems
 > until we extended the coherent_pool. The DVB sticks are probably take
 > more coherent RAM. There was also an issue found recently:
 >
 > http://www.spinics.net/lists/arm-kernel/msg203962.html
 >
 > That conversation has gone quiet, but that could be because the
 > participants are at ELCE.
 >
 >          Andrew

OK, I hope this GFP flag correction will help.

Could there be a fragmentation problem in the coherent_pool with the
different drivers running under heavy load?
With a pool size of 1M I see this error after several minutes, with a 4M
pool I see this error after several 10 minutes. Difficult to test, but not
acceptable on a production system.

Soeren