[PATCH v3] powerpc32: memcpy: only use dcbz once cache is enabled

[PATCH v3] powerpc32: memcpy: only use dcbz once cache is enabled

[Christophe Leroy]

memcpy() uses instruction dcbz to speed up copy by not wasting time
loading cache line with data that will be overwritten.
Some platform like mpc52xx do no have cache active at startup and
can therefore not use memcpy(). Allthough no part of the code
explicitly uses memcpy(), GCC makes calls to it.

This patch modifies memcpy() such that at startup, memcpy()
unconditionally jumps to generic_memcpy() which doesn't use
the dcbz instruction.

Once the initial MMU is set up, in machine_init() we patch memcpy()
by replacing this inconditional jump by a NOP

Reported-by: Michal Sojka <sojkam1@fel.cvut.cz>
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
---
changes in v2:
  Using feature-fixups instead of hardcoded call to patch_instruction()
  Handling of memset() added
changes in v3:
  Not using anymore feature-fixups
  Handling of memset() removed

 arch/powerpc/kernel/setup_32.c | 3 +++
 arch/powerpc/lib/copy_32.S     | 5 +++++
 2 files changed, 8 insertions(+)

diff --git a/arch/powerpc/kernel/setup_32.c b/arch/powerpc/kernel/setup_32.c
index 07831ed..362495f 100644
--- a/arch/powerpc/kernel/setup_32.c
+++ b/arch/powerpc/kernel/setup_32.c
@@ -39,6 +39,7 @@
 #include <asm/udbg.h>
 #include <asm/mmu_context.h>
 #include <asm/epapr_hcalls.h>
+#include <asm/code-patching.h>
 
 #define DBG(fmt...)
 
@@ -122,6 +123,8 @@ notrace void __init machine_init(u64 dt_ptr)
 	/* Enable early debugging if any specified (see udbg.h) */
 	udbg_early_init();
 
+	patch_instruction((unsigned int *)&memcpy, PPC_INST_NOP);
+
 	/* Do some early initialization based on the flat device tree */
 	early_init_devtree(__va(dt_ptr));
 
diff --git a/arch/powerpc/lib/copy_32.S b/arch/powerpc/lib/copy_32.S
index 2ef50c6..da5847d 100644
--- a/arch/powerpc/lib/copy_32.S
+++ b/arch/powerpc/lib/copy_32.S
@@ -128,6 +128,10 @@ _GLOBAL(memset)
  * the destination area is cacheable.
  * We only use this version if the source and dest don't overlap.
  * -- paulus.
+ *
+ * During early init, cache might not be active yet, so dcbz cannot be used.
+ * We therefore jump to generic_memcpy which doesn't use dcbz. This jump is
+ * replaced by a nop once cache is active. This is done in machine_init()
  */
 _GLOBAL(memmove)
 	cmplw	0,r3,r4
@@ -135,6 +139,7 @@ _GLOBAL(memmove)
 	/* fall through */
 
 _GLOBAL(memcpy)
+	b	generic_memcpy
 	add	r7,r3,r5		/* test if the src & dst overlap */
 	add	r8,r4,r5
 	cmplw	0,r4,r7
-- 
2.1.0

Re: [PATCH v3] powerpc32: memcpy: only use dcbz once cache is enabled

[Scott Wood]

On Fri, 2015-09-11 at 16:33 +0200, Christophe Leroy wrote:
> memcpy() uses instruction dcbz to speed up copy by not wasting time
> loading cache line with data that will be overwritten.
> Some platform like mpc52xx do no have cache active at startup and
> can therefore not use memcpy(). Allthough no part of the code
> explicitly uses memcpy(), GCC makes calls to it.
> 
> This patch modifies memcpy() such that at startup, memcpy()
> unconditionally jumps to generic_memcpy() which doesn't use
> the dcbz instruction.
> 
> Once the initial MMU is set up, in machine_init() we patch memcpy()
> by replacing this inconditional jump by a NOP
> 
> Reported-by: Michal Sojka <sojkam1@fel.cvut.cz>
> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> ---
> changes in v2:
>   Using feature-fixups instead of hardcoded call to patch_instruction()
>   Handling of memset() added
> changes in v3:
>   Not using anymore feature-fixups
>   Handling of memset() removed

Why was handling of memset() removed?

-Scott

Re: [PATCH v3] powerpc32: memcpy: only use dcbz once cache is enabled

[christophe leroy]

Le 11/09/2015 23:35, Scott Wood a écrit :
> On Fri, 2015-09-11 at 16:33 +0200, Christophe Leroy wrote:
>> memcpy() uses instruction dcbz to speed up copy by not wasting time
>> loading cache line with data that will be overwritten.
>> Some platform like mpc52xx do no have cache active at startup and
>> can therefore not use memcpy(). Allthough no part of the code
>> explicitly uses memcpy(), GCC makes calls to it.
>>
>> This patch modifies memcpy() such that at startup, memcpy()
>> unconditionally jumps to generic_memcpy() which doesn't use
>> the dcbz instruction.
>>
>> Once the initial MMU is set up, in machine_init() we patch memcpy()
>> by replacing this inconditional jump by a NOP
>>
>> Reported-by: Michal Sojka <sojkam1@fel.cvut.cz>
>> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
>> ---
>> changes in v2:
>>    Using feature-fixups instead of hardcoded call to patch_instruction()
>>    Handling of memset() added
>> changes in v3:
>>    Not using anymore feature-fixups
>>    Handling of memset() removed
> Why was handling of memset() removed?
>
>

Initially, the issue was reported for memcpy() only. v1 was only 
handling memcpy()
When it came to using fixups with v2, it took the opportunity to also 
handle memset() just in case, as it was quite simple.
Now with v3, we are back to using simple solution with 
patch_instruction() as recommended by Michael, having to point to the 
instruction to be replaced by a NOP.
For memcpy() it is quite easy, we just put a jump to generic_memcpy() as 
first instruction of memcpy().
For memset() we don't have a generic_memset() to jump to in the begining 
of memset(). We have to skip the handling of complete cache lines (block 
starting with "clrlwi    r7,r6,32-LG_CACHELINE_BYTES"). This means we 
have to add a global symbol for that to use with patch_instruction()

memset() doesn't seem to be an issue for the time being. Should we do it 
anyway ?

Christophe

---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus