[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sören Moch]

resent as plain text, sorry.


 > For Armada 370/XP we have the same problem that for the commit
 > cb01b63, so we applied the same solution: "The default 256 KiB
 > coherent pool may be too small for some of the Kirkwood devices, so
 > increase it to make sure that devices will be able to allocate their
 > buffers with GFP_ATOMIC flag"

I see a regression from linux-3.5 to linux-3.6 and think there might be 
a fundamental problem
with this patch. On my Kirkwood system (guruplug server plus) with 
linux-3.6.2 I see following
errors and corresponding malfunction even with further increased (2M, 
4M) pool size:

Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent pool is 
too small!
Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool= 
kernel parameter!

So I had to downgrade to linux-3.5 which is running without problems.

I use SATA and several DVB sticks (em28xx / drxk and dib0700).

I already sent this bug report to m.szyprowski and 
gregory.clementearlier, but
Sebastian suggested to send it to the mailing list and the responsible 
maintainers, too.

Please write to my e-mail directly if you have further questions or 
patches for me to test.

Regards,
Soeren

Re: [PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sören Moch]

resent as plain text, sorry.


 > For Armada 370/XP we have the same problem that for the commit
 > cb01b63, so we applied the same solution: "The default 256 KiB
 > coherent pool may be too small for some of the Kirkwood devices, so
 > increase it to make sure that devices will be able to allocate their
 > buffers with GFP_ATOMIC flag"

I see a regression from linux-3.5 to linux-3.6 and think there might be 
a fundamental problem
with this patch. On my Kirkwood system (guruplug server plus) with 
linux-3.6.2 I see following
errors and corresponding malfunction even with further increased (2M, 
4M) pool size:

Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent pool is 
too small!
Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool= 
kernel parameter!

So I had to downgrade to linux-3.5 which is running without problems.

I use SATA and several DVB sticks (em28xx / drxk and dib0700).

I already sent this bug report to m.szyprowski and 
gregory.clementearlier, but
Sebastian suggested to send it to the mailing list and the responsible 
maintainers, too.

Please write to my e-mail directly if you have further questions or 
patches for me to test.

Regards,
Soeren

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Andrew Lunn]

On Tue, Nov 06, 2012 at 10:28:45PM +0100, S?ren Moch wrote:
> resent as plain text, sorry.
> 
> 
> > For Armada 370/XP we have the same problem that for the commit
> > cb01b63, so we applied the same solution: "The default 256 KiB
> > coherent pool may be too small for some of the Kirkwood devices, so
> > increase it to make sure that devices will be able to allocate their
> > buffers with GFP_ATOMIC flag"
> 
> I see a regression from linux-3.5 to linux-3.6 and think there might
> be a fundamental problem
> with this patch. On my Kirkwood system (guruplug server plus) with
> linux-3.6.2 I see following
> errors and corresponding malfunction even with further increased
> (2M, 4M) pool size:
> 
> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
> pool is too small!
> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
> kernel parameter!
> 
> So I had to downgrade to linux-3.5 which is running without problems.
> 
> I use SATA and several DVB sticks (em28xx / drxk and dib0700).

I'm guess its the DVB sticks which are causing the problems. We have a
number of kirkwood devices with two SATA devices which had problems
until we extended the coherent_pool. The DVB sticks are probably take
more coherent RAM. There was also an issue found recently:

http://www.spinics.net/lists/arm-kernel/msg203962.html

That conversation has gone quiet, but that could be because the
participants are at ELCE.

	     Andrew

Re: [PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Andrew Lunn]

On Tue, Nov 06, 2012 at 10:28:45PM +0100, S?ren Moch wrote:
> resent as plain text, sorry.
> 
> 
> > For Armada 370/XP we have the same problem that for the commit
> > cb01b63, so we applied the same solution: "The default 256 KiB
> > coherent pool may be too small for some of the Kirkwood devices, so
> > increase it to make sure that devices will be able to allocate their
> > buffers with GFP_ATOMIC flag"
> 
> I see a regression from linux-3.5 to linux-3.6 and think there might
> be a fundamental problem
> with this patch. On my Kirkwood system (guruplug server plus) with
> linux-3.6.2 I see following
> errors and corresponding malfunction even with further increased
> (2M, 4M) pool size:
> 
> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
> pool is too small!
> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
> kernel parameter!
> 
> So I had to downgrade to linux-3.5 which is running without problems.
> 
> I use SATA and several DVB sticks (em28xx / drxk and dib0700).

I'm guess its the DVB sticks which are causing the problems. We have a
number of kirkwood devices with two SATA devices which had problems
until we extended the coherent_pool. The DVB sticks are probably take
more coherent RAM. There was also an issue found recently:

http://www.spinics.net/lists/arm-kernel/msg203962.html

That conversation has gone quiet, but that could be because the
participants are at ELCE.

	     Andrew

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sebastian Hesselbarth]

On 11/06/2012 11:32 PM, Andrew Lunn wrote:
> On Tue, Nov 06, 2012 at 10:28:45PM +0100, S?ren Moch wrote:
>> I see a regression from linux-3.5 to linux-3.6 and think there might
>> be a fundamental problem
>> with this patch. On my Kirkwood system (guruplug server plus) with
>> linux-3.6.2 I see following
>> errors and corresponding malfunction even with further increased
>> (2M, 4M) pool size:
>>
>> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
>> pool is too small!
>> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
>> kernel parameter!
>>
>> So I had to downgrade to linux-3.5 which is running without problems.
>>
>> I use SATA and several DVB sticks (em28xx / drxk and dib0700).
>
> I'm guess its the DVB sticks which are causing the problems. We have a
> number of kirkwood devices with two SATA devices which had problems
> until we extended the coherent_pool. The DVB sticks are probably take
> more coherent RAM. There was also an issue found recently:
>
> http://www.spinics.net/lists/arm-kernel/msg203962.html
>
> That conversation has gone quiet, but that could be because the
> participants are at ELCE.

So what is the call here? Should we just increase the coherent buffer
size back to what it was before? I am not into this too much but just
increasing the buffer will just postpone the actual issue to a later
point in running the kernel?

Sebastian

Re: [PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sebastian Hesselbarth]

On 11/06/2012 11:32 PM, Andrew Lunn wrote:
> On Tue, Nov 06, 2012 at 10:28:45PM +0100, S?ren Moch wrote:
>> I see a regression from linux-3.5 to linux-3.6 and think there might
>> be a fundamental problem
>> with this patch. On my Kirkwood system (guruplug server plus) with
>> linux-3.6.2 I see following
>> errors and corresponding malfunction even with further increased
>> (2M, 4M) pool size:
>>
>> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
>> pool is too small!
>> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
>> kernel parameter!
>>
>> So I had to downgrade to linux-3.5 which is running without problems.
>>
>> I use SATA and several DVB sticks (em28xx / drxk and dib0700).
>
> I'm guess its the DVB sticks which are causing the problems. We have a
> number of kirkwood devices with two SATA devices which had problems
> until we extended the coherent_pool. The DVB sticks are probably take
> more coherent RAM. There was also an issue found recently:
>
> http://www.spinics.net/lists/arm-kernel/msg203962.html
>
> That conversation has gone quiet, but that could be because the
> participants are at ELCE.

So what is the call here? Should we just increase the coherent buffer
size back to what it was before? I am not into this too much but just
increasing the buffer will just postpone the actual issue to a later
point in running the kernel?

Sebastian

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sören Moch]

 >>> For Armada 370/XP we have the same problem that for the commit
 >>> cb01b63, so we applied the same solution: "The default 256 KiB
 >>> coherent pool may be too small for some of the Kirkwood devices, so
 >>> increase it to make sure that devices will be able to allocate their
 >>> buffers with GFP_ATOMIC flag"
 >>
 >> I see a regression from linux-3.5 to linux-3.6 and think there might
 >> be a fundamental problem
 >> with this patch. On my Kirkwood system (guruplug server plus) with
 >> linux-3.6.2 I see following
 >> errors and corresponding malfunction even with further increased
 >> (2M, 4M) pool size:
 >>
 >> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
 >> pool is too small!
 >> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
 >> kernel parameter!
 >>
 >> So I had to downgrade to linux-3.5 which is running without problems.
 >>
 >> I use SATA and several DVB sticks (em28xx / drxk and dib0700).
 >
 > I'm guess its the DVB sticks which are causing the problems. We have a
 > number of kirkwood devices with two SATA devices which had problems
 > until we extended the coherent_pool. The DVB sticks are probably take
 > more coherent RAM. There was also an issue found recently:
 >
 > http://www.spinics.net/lists/arm-kernel/msg203962.html
 >
 > That conversation has gone quiet, but that could be because the
 > participants are at ELCE.
 >
 >          Andrew

OK, I hope this GFP flag correction will help.

Could there be a fragmentation problem in the coherent_pool with the
different drivers running under heavy load?
With a pool size of 1M I see this error after several minutes, with a 4M
pool I see this error after several 10 minutes. Difficult to test, but not
acceptable on a production system.

Soeren

Re: [PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Sören Moch]

 >>> For Armada 370/XP we have the same problem that for the commit
 >>> cb01b63, so we applied the same solution: "The default 256 KiB
 >>> coherent pool may be too small for some of the Kirkwood devices, so
 >>> increase it to make sure that devices will be able to allocate their
 >>> buffers with GFP_ATOMIC flag"
 >>
 >> I see a regression from linux-3.5 to linux-3.6 and think there might
 >> be a fundamental problem
 >> with this patch. On my Kirkwood system (guruplug server plus) with
 >> linux-3.6.2 I see following
 >> errors and corresponding malfunction even with further increased
 >> (2M, 4M) pool size:
 >>
 >> Oct 19 00:41:22 guru kernel: ERROR: 4096 KiB atomic DMA coherent
 >> pool is too small!
 >> Oct 19 00:41:22 guru kernel: Please increase it with coherent_pool=
 >> kernel parameter!
 >>
 >> So I had to downgrade to linux-3.5 which is running without problems.
 >>
 >> I use SATA and several DVB sticks (em28xx / drxk and dib0700).
 >
 > I'm guess its the DVB sticks which are causing the problems. We have a
 > number of kirkwood devices with two SATA devices which had problems
 > until we extended the coherent_pool. The DVB sticks are probably take
 > more coherent RAM. There was also an issue found recently:
 >
 > http://www.spinics.net/lists/arm-kernel/msg203962.html
 >
 > That conversation has gone quiet, but that could be because the
 > participants are at ELCE.
 >
 >          Andrew

OK, I hope this GFP flag correction will help.

Could there be a fragmentation problem in the coherent_pool with the
different drivers running under heavy load?
With a pool size of 1M I see this error after several minutes, with a 4M
pool I see this error after several 10 minutes. Difficult to test, but not
acceptable on a production system.

Soeren

[PATCH V2 0/4] Adding SATA support for Armada 370/XP

[Gregory CLEMENT]

Hello,

this patch set adds the SATA support for Armada 370 and Armada XP. Few
changes have been done since the first version by taking in account
the comments received for the first version.

The evaluation boards for Armada 370 and Armada XP come with 2 SATA
ports, and when both are enable the coherent pool for DMA mapping was
too short. It was exactly the same issue that was fixed for Kirkwood
two months ago. So I used the same fix in the first patch. Later when
Kirkwood will be part of mach-mvebu, then this fix will be shared
between the 2 SoCs families.

This patch set is based on 3.7-rc2 and depends one the framework clock
support (the last version was posted last week:
http://thread.gmane.org/gmane.linux.kernel/1375701). The git branch
called mvebu-SATA-for-3.8 is also available at
https://github.com/MISL-EBU-System-SW/mainline-public.git.

Changelog:

V1 -> V2:
- Added the acked-by from Marek Szyprowski for the first patch.
- Moved the port number from dtsi to board file t o be coherent with
  Kirwkood boards.
- Cleaned armada-370-xp.dtsi from test strings.
- Split the second patch in 3 part one for dts, one for config update,
  and the last for dtsi.
- Updated the mvebu_defconfig file with only the necessary symbols.
- Updated also the multi_v7_defconfig file.

Regards,

Gregory

Gregory CLEMENT (4):
  arm: mvebu: increase atomic coherent pool size for armada 370/XP
  arm: mvebu: adding SATA support: dt binding for Armada 370/XP
  arm: mvebu: adding SATA support: configs update
  arm: mvebu: adding SATA support: dt binding for Armada 370/XP boards

 arch/arm/boot/dts/armada-370-db.dts  |    4 ++++
 arch/arm/boot/dts/armada-370-xp.dtsi |    9 +++++++++
 arch/arm/boot/dts/armada-xp-db.dts   |    4 ++++
 arch/arm/configs/multi_v7_defconfig  |    2 ++
 arch/arm/configs/mvebu_defconfig     |    3 +++
 arch/arm/mach-mvebu/armada-370-xp.c  |   12 ++++++++++++
 6 files changed, 34 insertions(+)

-- 
1.7.9.5

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Gregory CLEMENT]

For Armada 370/XP we have the same problem that for the commit
cb01b63, so we applied the same solution: "The default 256 KiB
coherent pool may be too small for some of the Kirkwood devices, so
increase it to make sure that devices will be able to allocate their
buffers with GFP_ATOMIC flag"

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/arm/mach-mvebu/armada-370-xp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
index 2af6ce5..cbad821 100644
--- a/arch/arm/mach-mvebu/armada-370-xp.c
+++ b/arch/arm/mach-mvebu/armada-370-xp.c
@@ -17,6 +17,7 @@
 #include <linux/of_platform.h>
 #include <linux/io.h>
 #include <linux/time-armada-370-xp.h>
+#include <linux/dma-mapping.h>
 #include <asm/mach/arch.h>
 #include <asm/mach/map.h>
 #include <asm/mach/time.h>
@@ -43,6 +44,16 @@ void __init armada_370_xp_timer_and_clk_init(void)
 	armada_370_xp_timer_init();
 }
 
+void __init armada_370_xp_init_early(void)
+{
+	/*
+	 * Some Armada 370/XP devices allocate their coherent buffers
+	 * from atomic context. Increase size of atomic coherent pool
+	 * to make sure such the allocations won't fail.
+	 */
+	init_dma_coherent_pool_size(SZ_1M);
+}
+
 struct sys_timer armada_370_xp_timer = {
 	.init		= armada_370_xp_timer_and_clk_init,
 };
@@ -61,6 +72,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
 DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
 	.init_machine	= armada_370_xp_dt_init,
 	.map_io		= armada_370_xp_map_io,
+	.init_early	= armada_370_xp_init_early,
 	.init_irq	= armada_370_xp_init_irq,
 	.handle_irq     = armada_370_xp_handle_irq,
 	.timer		= &armada_370_xp_timer,
-- 
1.7.9.5

[PATCH V2 1/4] arm: mvebu: increase atomic coherent pool size for armada 370/XP

[Gregory CLEMENT]

For Armada 370/XP we have the same problem that for the commit
cb01b63, so we applied the same solution: "The default 256 KiB
coherent pool may be too small for some of the Kirkwood devices, so
increase it to make sure that devices will be able to allocate their
buffers with GFP_ATOMIC flag"

Signed-off-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
 arch/arm/mach-mvebu/armada-370-xp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
index 2af6ce5..cbad821 100644
--- a/arch/arm/mach-mvebu/armada-370-xp.c
+++ b/arch/arm/mach-mvebu/armada-370-xp.c
@@ -17,6 +17,7 @@
 #include <linux/of_platform.h>
 #include <linux/io.h>
 #include <linux/time-armada-370-xp.h>
+#include <linux/dma-mapping.h>
 #include <asm/mach/arch.h>
 #include <asm/mach/map.h>
 #include <asm/mach/time.h>
@@ -43,6 +44,16 @@ void __init armada_370_xp_timer_and_clk_init(void)
 	armada_370_xp_timer_init();
 }
 
+void __init armada_370_xp_init_early(void)
+{
+	/*
+	 * Some Armada 370/XP devices allocate their coherent buffers
+	 * from atomic context. Increase size of atomic coherent pool
+	 * to make sure such the allocations won't fail.
+	 */
+	init_dma_coherent_pool_size(SZ_1M);
+}
+
 struct sys_timer armada_370_xp_timer = {
 	.init		= armada_370_xp_timer_and_clk_init,
 };
@@ -61,6 +72,7 @@ static const char * const armada_370_xp_dt_board_dt_compat[] = {
 DT_MACHINE_START(ARMADA_XP_DT, "Marvell Aramada 370/XP (Device Tree)")
 	.init_machine	= armada_370_xp_dt_init,
 	.map_io		= armada_370_xp_map_io,
+	.init_early	= armada_370_xp_init_early,
 	.init_irq	= armada_370_xp_init_irq,
 	.handle_irq     = armada_370_xp_handle_irq,
 	.timer		= &armada_370_xp_timer,
-- 
1.7.9.5