Re: [PATCH] powerpc: align DTL buffer to AMS boundary

Re: [PATCH] powerpc: align DTL buffer to AMS boundary

[David Laight]

> From:=20
> linuxppc-dev-bounces+david.laight=3Daculab.com@lists.ozlabs.org=20
> [mailto:linuxppc-dev-bounces+david.laight=3Daculab.com@lists.ozl
> abs.org] On Behalf Of Nishanth Aravamudan
> Sent: 13 April 2011 15:53
> To: Ben Herrenschmidt
> Cc: linuxppc-dev@ozlabs.org; Paul Mackerras; Anton Blanchard
> Subject: [PATCH] powerpc: align DTL buffer to AMS boundary
>=20
> PAPR specifies that DTL buffers can not cross AMS environments (aka
CMO
> in the PAPR) and can not cross a memory entitlement granule boundary
> (4k)....

How big is the buffer being allocated?
If it is much less than 4k then it might be worth allocating
a buffer of the correct size, and only if that crosses a 4k boundary
allocate the larger buffer.

Also, if the buffer is ever freed, the actual base address is needed
for the free.

	David

Re: [PATCH] powerpc: align DTL buffer to AMS boundary

[Nishanth Aravamudan]

On 13.04.2011 [15:59:44 +0100], David Laight wrote:
> > From: 
> > linuxppc-dev-bounces+david.laight=aculab.com@lists.ozlabs.org 
> > [mailto:linuxppc-dev-bounces+david.laight=aculab.com@lists.ozl
> > abs.org] On Behalf Of Nishanth Aravamudan
> > Sent: 13 April 2011 15:53
> > To: Ben Herrenschmidt
> > Cc: linuxppc-dev@ozlabs.org; Paul Mackerras; Anton Blanchard
> > Subject: [PATCH] powerpc: align DTL buffer to AMS boundary
> > 
> > PAPR specifies that DTL buffers can not cross AMS environments (aka
> CMO
> > in the PAPR) and can not cross a memory entitlement granule boundary
> > (4k)....
> 
> How big is the buffer being allocated?
> If it is much less than 4k then it might be worth allocating
> a buffer of the correct size, and only if that crosses a 4k boundary
> allocate the larger buffer.

Currently, DISPATCH_LOG_BYTES is 4k (which I assume was actually in
deference to this requirement).

> Also, if the buffer is ever freed, the actual base address is needed
> for the free.

I didn't see any free patch for this sequence of code -- which make
sense as this allocation occurs under a for_each_possible_cpu loop.

Thanks for the feedback,
Nish

[PATCH] powerpc: align DTL buffer to AMS boundary

[Nishanth Aravamudan]

PAPR specifies that DTL buffers can not cross AMS environments (aka CMO
in the PAPR) and can not cross a memory entitlement granule boundary
(4k). This is found in section 14.11.3.2 H_REGISTER_VPA of the PAPR.
kmalloc does not guarantee an alignment of the allocation, though,
beyond 8 bytes (at least in my understanding). Over-allocate and align
the resulting address. Tested both with and without AMS on a p7
partition.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>

---
Note, I initially put this in a firmware check if-block, but we have
also seen some issues with alignment with non-AMS partitions. The wasted
memory is unfortunate, though.

Cc: Paul Mackerras <paulus@samba.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: linuxppc-dev@ozlabs.org

 arch/powerpc/platforms/pseries/setup.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 0007241..7df5ddb 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -39,6 +39,7 @@
 #include <linux/irq.h>
 #include <linux/seq_file.h>
 #include <linux/root_dev.h>
+#include <linux/memblock.h>
 
 #include <asm/mmu.h>
 #include <asm/processor.h>
@@ -293,14 +294,15 @@ static int alloc_dispatch_logs(void)
 
 	for_each_possible_cpu(cpu) {
 		pp = &paca[cpu];
-		dtl = kmalloc_node(DISPATCH_LOG_BYTES, GFP_KERNEL,
-				   cpu_to_node(cpu));
+		dtl = kmalloc_node(DISPATCH_LOG_BYTES + DISPATCH_LOG_BYTES - 1,
+				   GFP_KERNEL, cpu_to_node(cpu));
 		if (!dtl) {
 			pr_warn("Failed to allocate dispatch trace log for cpu %d\n",
 				cpu);
 			pr_warn("Stolen time statistics will be unreliable\n");
 			break;
 		}
+		dtl = PTR_ALIGN(dtl, DISPATCH_LOG_BYTES);
 
 		pp->dtl_ridx = 0;
 		pp->dispatch_log = dtl;
-- 
1.7.4.1