* [PATCH 00/14] Backport 8xx TLB to 2.4
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
This is a backport from 2.6 which I did to overcome 8xx CPU
bugs. 8xx does not update the DAR register when taking a TLB
error caused by dcbX and icbi insns which makes it very
tricky to use these insns. Also the dcbst wrongly sets the
the store bit when faulting into DTLB error.
A few more bugs very found during development.
I know 2.4 is in strict maintenance mode and 8xx is obsolete
but as it is still in use I wanted 8xx to age with grace.
Addendum:
I have now ported our 8xx custom board to 2.4.37.11 and
tested these patches there.
V2:
- Remove mandatory pinning of kernel ITLB. It is not
needed in 2.4
8 MB Large page support will follow.
Joakim Tjernlund (14):
8xx: Use a macro to simpliy CPU6 errata code.
8xx: Tag DAR with 0x00f0 to catch buggy instructions.
8xx: invalidate non present TLBs
8xx: Fix CONFIG_PIN_TLB
8xx: Update TLB asm so it behaves as linux mm expects.
8xx: Fixup DAR from buggy dcbX instructions.
8xx: CPU6 errata make DTLB error too big to fit.
8xx: Add missing Guarded setting in DTLB Error.
8xx: Restore _PAGE_WRITETHRU
8xx: Set correct HW pte flags in DTLB Error too
8xx: start using dcbX instructions in various copy routines
8xx: Use symbolic constants in TLB asm
8xx: Optimize TLB Miss handlers
8xx: The TLB miss handler manages ACCESSED correctly.
arch/ppc/kernel/head_8xx.S | 367 ++++++++++++++++++++++++++++++-------------
arch/ppc/kernel/misc.S | 18 --
arch/ppc/lib/string.S | 17 --
include/asm-ppc/pgtable.h | 26 +--
4 files changed, 264 insertions(+), 164 deletions(-)
--
1.7.3.4
^ permalink raw reply
* [PATCH 02/14] 8xx: Tag DAR with 0x00f0 to catch buggy instructions.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
dcbz, dcbf, dcbi, dcbst and icbi do not set DAR when they
cause a DTLB Error. Dectect this by tagging DAR with 0x00f0
at every exception exit that modifies DAR.
This also fixes MachineCheck to pass DAR and DSISR as well.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 18 +++++++++++++++++-
1 files changed, 17 insertions(+), 1 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index ba05a57..57858ce 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -197,7 +197,17 @@ label: \
STD_EXCEPTION(0x100, Reset, UnknownException)
/* Machine check */
- STD_EXCEPTION(0x200, MachineCheck, MachineCheckException)
+ . = 0x200
+MachineCheck:
+ EXCEPTION_PROLOG
+ mfspr r20,DSISR
+ stw r20,_DSISR(r21)
+ mfspr r20,DAR
+ stw r20,_DAR(r21)
+ li r20,0x00f0
+ mtspr DAR,r20 /* Tag DAR */
+ addi r3,r1,STACK_FRAME_OVERHEAD
+ FINISH_EXCEPTION(MachineCheckException)
/* Data access exception.
* This is "never generated" by the MPC8xx. We jump to it for other
@@ -211,6 +221,8 @@ DataAccess:
mr r5,r20
mfspr r4,DAR
stw r4,_DAR(r21)
+ li r20,0x00f0
+ mtspr DAR,r20 /* Tag DAR */
addi r3,r1,STACK_FRAME_OVERHEAD
li r20,MSR_KERNEL
rlwimi r20,r23,0,16,16 /* copy EE bit from saved MSR */
@@ -249,6 +261,8 @@ Alignment:
EXCEPTION_PROLOG
mfspr r4,DAR
stw r4,_DAR(r21)
+ li r20,0x00f0
+ mtspr DAR,r20 /* Tag DAR */
mfspr r5,DSISR
stw r5,_DSISR(r21)
addi r3,r1,STACK_FRAME_OVERHEAD
@@ -433,6 +447,7 @@ DataStoreTLBMiss:
* of the MMU.
*/
2: li r21, 0x00f0
+ mtspr DAR, r21 /* Tag DAR */
rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
DO_8xx_CPU6(0x3d80, r3)
mtspr MD_RPN, r20 /* Update TLB entry */
@@ -543,6 +558,7 @@ DataTLBError:
* of the MMU.
*/
li r21, 0x00f0
+ mtspr DAR, r21 /* Tag DAR */
rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
DO_8xx_CPU6(0x3d80, r3)
mtspr MD_RPN, r20 /* Update TLB entry */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 01/14] 8xx: Use a macro to simpliy CPU6 errata code.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 84 +++++++++++--------------------------------
1 files changed, 22 insertions(+), 62 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index f9a30f3..ba05a57 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -31,6 +31,15 @@
#include <asm/ppc_asm.h>
#include "ppc_defs.h"
+/* Macro to make the code more readable. */
+#ifdef CONFIG_8xx_CPU6
+ #define DO_8xx_CPU6(val, reg) \
+ li reg, val; \
+ stw reg, 12(r0); \
+ lwz reg, 12(r0);
+#else
+ #define DO_8xx_CPU6(val, reg)
+#endif
.text
.globl _stext
_stext:
@@ -310,20 +319,14 @@ SystemCall:
InstructionTLBMiss:
#ifdef CONFIG_8xx_CPU6
stw r3, 8(r0)
- li r3, 0x3f80
- stw r3, 12(r0)
- lwz r3, 12(r0)
#endif
+ DO_8xx_CPU6(0x3f80, r3)
mtspr M_TW, r20 /* Save a couple of working registers */
mfcr r20
stw r20, 0(r0)
stw r21, 4(r0)
mfspr r20, SRR0 /* Get effective address of fault */
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3780
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3780, r3)
mtspr MD_EPN, r20 /* Have to use MD_EPN for walk, MI_EPN can't */
mfspr r20, M_TWB /* Get level 1 table entry address */
@@ -345,17 +348,9 @@ InstructionTLBMiss:
tophys(r21,r21)
ori r21,r21,1 /* Set valid bit */
beq- 2f /* If zero, don't try to find a pte */
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x2b80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x2b80, r3)
mtspr MI_TWC, r21 /* Set segment attributes */
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3b80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21 /* Load pte table base address */
mfspr r21, MD_TWC /* ....and get the pte address */
lwz r20, 0(r21) /* Get the pte */
@@ -371,12 +366,7 @@ InstructionTLBMiss:
*/
2: li r21, 0x00f0
rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
-
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x2d80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x2d80, r3)
mtspr MI_RPN, r20 /* Update TLB entry */
mfspr r20, M_TW /* Restore registers */
@@ -392,10 +382,8 @@ InstructionTLBMiss:
DataStoreTLBMiss:
#ifdef CONFIG_8xx_CPU6
stw r3, 8(r0)
- li r3, 0x3f80
- stw r3, 12(r0)
- lwz r3, 12(r0)
#endif
+ DO_8xx_CPU6(0x3f80, r3)
mtspr M_TW, r20 /* Save a couple of working registers */
mfcr r20
stw r20, 0(r0)
@@ -419,11 +407,7 @@ DataStoreTLBMiss:
tophys(r21, r21)
ori r21, r21, 1 /* Set valid bit in physical L2 page */
beq- 2f /* If zero, don't try to find a pte */
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3b80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21 /* Load pte table base address */
mfspr r20, MD_TWC /* ....and get the pte address */
lwz r20, 0(r20) /* Get the pte */
@@ -435,11 +419,7 @@ DataStoreTLBMiss:
* above.
*/
rlwimi r21, r20, 0, 27, 27
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3b80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
mfspr r21, MD_TWC /* get the pte address again */
@@ -454,12 +434,7 @@ DataStoreTLBMiss:
*/
2: li r21, 0x00f0
rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
-
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3d80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3d80, r3)
mtspr MD_RPN, r20 /* Update TLB entry */
mfspr r20, M_TW /* Restore registers */
@@ -491,10 +466,8 @@ InstructionTLBError:
DataTLBError:
#ifdef CONFIG_8xx_CPU6
stw r3, 8(r0)
- li r3, 0x3f80
- stw r3, 12(r0)
- lwz r3, 12(r0)
#endif
+ DO_8xx_CPU6(0x3f80, r3)
mtspr M_TW, r20 /* Save a couple of working registers */
mfcr r20
stw r20, 0(r0)
@@ -527,11 +500,7 @@ DataTLBError:
ori r21, r21, MD_EVALID
mfspr r20, M_CASID
rlwimi r21, r20, 0, 28, 31
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3780
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3780, r3)
mtspr MD_EPN, r21
mfspr r20, M_TWB /* Get level 1 table entry address */
@@ -553,11 +522,7 @@ DataTLBError:
*/
tophys(r21, r21)
ori r21, r21, 1 /* Set valid bit in physical L2 page */
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3b80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21 /* Load pte table base address */
mfspr r21, MD_TWC /* ....and get the pte address */
lwz r20, 0(r21) /* Get the pte */
@@ -579,12 +544,7 @@ DataTLBError:
*/
li r21, 0x00f0
rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
-
-#ifdef CONFIG_8xx_CPU6
- li r3, 0x3d80
- stw r3, 12(r0)
- lwz r3, 12(r0)
-#endif
+ DO_8xx_CPU6(0x3d80, r3)
mtspr MD_RPN, r20 /* Update TLB entry */
mfspr r20, M_TW /* Restore registers */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 03/14] 8xx: invalidate non present TLBs
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
8xx sometimes need to load a invalid/non-present TLBs in
it DTLB asm handler.
These must be invalidated separately as 8xx MMU don't.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 12 ++++++++++--
1 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 57858ce..b3aff21 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -221,7 +221,11 @@ DataAccess:
mr r5,r20
mfspr r4,DAR
stw r4,_DAR(r21)
- li r20,0x00f0
+ /* invalidate ~PRESENT TLBs, 8xx MMU don't do this */
+ andis. r20,r5,0x4000
+ beq+ 1f
+ tlbie r4
+1: li r20,0x00f0
mtspr DAR,r20 /* Tag DAR */
addi r3,r1,STACK_FRAME_OVERHEAD
li r20,MSR_KERNEL
@@ -238,7 +242,11 @@ InstructionAccess:
addi r3,r1,STACK_FRAME_OVERHEAD
mr r4,r22
mr r5,r23
- li r20,MSR_KERNEL
+ /* invalidate ~PRESENT TLBs, 8xx MMU don't do this */
+ andis. r20,r5,0x4000
+ beq+ 1f
+ tlbie r4
+1: li r20,MSR_KERNEL
rlwimi r20,r23,0,16,16 /* copy EE bit from saved MSR */
FINISH_EXCEPTION(do_page_fault)
--
1.7.3.4
^ permalink raw reply related
* [PATCH 04/14] 8xx: Fix CONFIG_PIN_TLB
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
The wrong register was loaded into MD_RPN.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index b3aff21..9d8a1b5 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -848,13 +848,13 @@ initial_mmu:
mtspr MD_TWC, r9
li r11, MI_BOOTINIT /* Create RPN for address 0 */
addis r11, r11, 0x0080 /* Add 8M */
- mtspr MD_RPN, r8
+ mtspr MD_RPN, r11
addis r8, r8, 0x0080 /* Add 8M */
mtspr MD_EPN, r8
mtspr MD_TWC, r9
addis r11, r11, 0x0080 /* Add 8M */
- mtspr MD_RPN, r8
+ mtspr MD_RPN, r11
#endif
/* Since the cache is enabled according to the information we
--
1.7.3.4
^ permalink raw reply related
* [PATCH 06/14] 8xx: Fixup DAR from buggy dcbX instructions.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
This is an assembler version to fixup DAR not being set
by dcbX, icbi instructions. There are two versions, one
uses selfmodifing code, the other uses a
jump table but is much bigger(default).
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 149 +++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 146 insertions(+), 3 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index c9770b6..0891b96 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -511,8 +511,17 @@ DataTLBError:
stw r20, 0(r0)
stw r21, 4(r0)
- mfspr r20, DSISR
- andis. r21, r20, 0x4800 /* !translation or protection */
+ mfspr r20, DAR
+ cmpwi cr0, r20, 0x00f0
+ beq- FixupDAR /* must be a buggy dcbX, icbi insn. */
+DARFixed:
+ /* As the DAR fixup may clear store we may have all 3 states zero.
+ * Make sure only 0x0200(store) falls down into DIRTY handling
+ */
+ mfspr r21, DSISR
+ andis. r21, r21, 0x4a00 /* !translation, protection or store */
+ srwi r21, r21, 16
+ cmpwi cr0, r21, 0x0200 /* just store ? */
bne- 2f
/* Only Change bit left now, do it here as it is faster
* than trapping to the C fault handler.
@@ -534,7 +543,7 @@ DataTLBError:
* are initialized in mapin_ram(). This will avoid the problem,
* assuming we only use the dcbi instruction on kernel addresses.
*/
- mfspr r20, DAR
+ /* DAR is in r20 already */
rlwinm r21, r20, 0, 0, 19
ori r21, r21, MD_EVALID
mfspr r20, M_CASID
@@ -618,6 +627,140 @@ DataTLBError:
STD_EXCEPTION(0x1f00, Trap_1f, UnknownException)
. = 0x2000
+/* This is the procedure to calculate the data EA for buggy dcbx,dcbi instructions
+ * by decoding the registers used by the dcbx instruction and adding them.
+ * DAR is set to the calculated address and r10 also holds the EA on exit.
+ */
+ /* define if you don't want to use self modifying code */
+#define NO_SELF_MODIFYING_CODE
+FixupDAR:/* Entry point for dcbx workaround. */
+ /* fetch instruction from memory. */
+ mfspr r20, SRR0
+ andis. r21, r20, 0x8000 /* Address >= 0x80000000 */
+ DO_8xx_CPU6(0x3780, r3)
+ mtspr MD_EPN, r20
+ mfspr r21, M_TWB /* Get level 1 table entry address */
+ beq- 3f /* Branch if user space */
+ lis r21, (swapper_pg_dir-PAGE_OFFSET)@h
+ ori r21, r21, (swapper_pg_dir-PAGE_OFFSET)@l
+ rlwimi r21, r20, 32-20, 0xffc /* r21 = r21&~0xffc|(r20>>20)&0xffc */
+3: lwz r21, 0(r21) /* Get the level 1 entry */
+ tophys (r21, r21)
+ DO_8xx_CPU6(0x3b80, r3)
+ mtspr MD_TWC, r21 /* Load pte table base address */
+ mfspr r21, MD_TWC /* ....and get the pte address */
+ lwz r21, 0(r21) /* Get the pte */
+ /* concat physical page address(r21) and page offset(r20) */
+ rlwimi r21, r20, 0, 20, 31
+ lwz r21,0(r21)
+/* Check if it really is a dcbx instruction. */
+/* dcbt and dcbtst does not generate DTLB Misses/Errors,
+ * no need to include them here */
+ srwi r20, r21, 26 /* check if major OP code is 31 */
+ cmpwi cr0, r20, 31
+ bne- 141f
+ rlwinm r20, r21, 0, 21, 30
+ cmpwi cr0, r20, 2028 /* Is dcbz? */
+ beq+ 142f
+ cmpwi cr0, r20, 940 /* Is dcbi? */
+ beq+ 142f
+ cmpwi cr0, r20, 108 /* Is dcbst? */
+ beq+ 144f /* Fix up store bit! */
+ cmpwi cr0, r20, 172 /* Is dcbf? */
+ beq+ 142f
+ cmpwi cr0, r20, 1964 /* Is icbi? */
+ beq+ 142f
+141: mfspr r20, DAR /* r20 must hold DAR at exit */
+ b DARFixed /* Nope, go back to normal TLB processing */
+
+144: mfspr r20, DSISR
+ rlwinm r20, r20,0,7,5 /* Clear store bit for buggy dcbst insn */
+ mtspr DSISR, r20
+142: /* continue, it was a dcbx, dcbi instruction. */
+#ifdef CONFIG_8xx_CPU6
+ lwz r3, 8(r0) /* restore r3 from memory */
+#endif
+#ifndef NO_SELF_MODIFYING_CODE
+ andis. r20,r21,0x1f /* test if reg RA is r0 */
+ li r20,modified_instr@l
+ dcbtst r0,r20 /* touch for store */
+ rlwinm r21,r21,0,0,20 /* Zero lower 10 bits */
+ oris r21,r21,640 /* Transform instr. to a "add r20,RA,RB" */
+ ori r21,r21,532
+ stw r21,0(r20) /* store add/and instruction */
+ dcbf 0,r20 /* flush new instr. to memory. */
+ icbi 0,r20 /* invalidate instr. cache line */
+ lwz r21, 4(r0) /* restore r21 from memory */
+ mfspr r20, M_TW /* restore r20 from M_TW */
+ isync /* Wait until new instr is loaded from memory */
+modified_instr:
+ .space 4 /* this is where the add instr. is stored */
+ bne+ 143f
+ subf r20,r0,r20 /* r20=r20-r0, only if reg RA is r0 */
+143: mtdar r20 /* store faulting EA in DAR */
+ b DARFixed /* Go back to normal TLB handling */
+#else
+ mfctr r20
+ mtdar r20 /* save ctr reg in DAR */
+ rlwinm r20, r21, 24, 24, 28 /* offset into jump table for reg RB */
+ addi r20, r20, 150f@l /* add start of table */
+ mtctr r20 /* load ctr with jump address */
+ xor r20, r20, r20 /* sum starts at zero */
+ bctr /* jump into table */
+150:
+ add r20, r20, r0 ;b 151f
+ add r20, r20, r1 ;b 151f
+ add r20, r20, r2 ;b 151f
+ add r20, r20, r3 ;b 151f
+ add r20, r20, r4 ;b 151f
+ add r20, r20, r5 ;b 151f
+ add r20, r20, r6 ;b 151f
+ add r20, r20, r7 ;b 151f
+ add r20, r20, r8 ;b 151f
+ add r20, r20, r9 ;b 151f
+ add r20, r20, r10 ;b 151f
+ add r20, r20, r11 ;b 151f
+ add r20, r20, r12 ;b 151f
+ add r20, r20, r13 ;b 151f
+ add r20, r20, r14 ;b 151f
+ add r20, r20, r15 ;b 151f
+ add r20, r20, r16 ;b 151f
+ add r20, r20, r17 ;b 151f
+ add r20, r20, r18 ;b 151f
+ add r20, r20, r19 ;b 151f
+ mtctr r21 ;b 154f /* r20 needs special handling */
+ mtctr r21 ;b 153f /* r21 needs special handling */
+ add r20, r20, r22 ;b 151f
+ add r20, r20, r23 ;b 151f
+ add r20, r20, r24 ;b 151f
+ add r20, r20, r25 ;b 151f
+ add r20, r20, r26 ;b 151f
+ add r20, r20, r27 ;b 151f
+ add r20, r20, r28 ;b 151f
+ add r20, r20, r29 ;b 151f
+ add r20, r20, r30 ;b 151f
+ add r20, r20, r31
+151:
+ rlwinm. r21,r21,19,24,28 /* offset into jump table for reg RA */
+ beq 152f /* if reg RA is zero, don't add it */
+ addi r21, r21, 150b@l /* add start of table */
+ mtctr r21 /* load ctr with jump address */
+ rlwinm r21,r21,0,16,10 /* make sure we don't execute this more than once */
+ bctr /* jump into table */
+152:
+ mfdar r21
+ mtctr r21 /* restore ctr reg from DAR */
+ mtdar r20 /* save fault EA to DAR */
+ b DARFixed /* Go back to normal TLB handling */
+
+ /* special handling for r20,r21 since these are modified already */
+153: lwz r21, 4(r0) /* load r21 from memory */
+ b 155f
+154: mfspr r21, M_TW /* load r20 from M_TW */
+155: add r20, r20, r21 /* add it */
+ mfctr r21 /* restore r21 */
+ b 151b
+#endif
/*
* This code finishes saving the registers to the exception frame
--
1.7.3.4
^ permalink raw reply related
* [PATCH 05/14] 8xx: Update TLB asm so it behaves as linux mm expects.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
Update the TLB asm to make proper use of _PAGE_DIRTY and _PAGE_ACCESSED.
Get rid of _PAGE_HWWRITE too.
Pros:
- PRESENT is copied to ACCESSED, fixing accounting
- DIRTY is mapped to 0x100, the changed bit, and is set directly
when a page has been made dirty.
- Proper RO/RW mapping of user space.
- Free up 2 SW TLB bits in the linux pte(add back _PAGE_WRITETHRU ?)
- kernel RO/user NA support. Not sure this is really needed, would save
a few insn if not required.
Cons:
- A few more instructions in the DTLB Miss routine.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 53 ++++++++++++++++++++++++++-----------------
include/asm-ppc/pgtable.h | 15 +++++------
2 files changed, 39 insertions(+), 29 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 9d8a1b5..c9770b6 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -369,25 +369,27 @@ InstructionTLBMiss:
*/
tophys(r21,r21)
ori r21,r21,1 /* Set valid bit */
- beq- 2f /* If zero, don't try to find a pte */
DO_8xx_CPU6(0x2b80, r3)
mtspr MI_TWC, r21 /* Set segment attributes */
+ beq- 2f /* If zero, don't try to find a pte */
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21 /* Load pte table base address */
mfspr r21, MD_TWC /* ....and get the pte address */
lwz r20, 0(r21) /* Get the pte */
- ori r20, r20, _PAGE_ACCESSED
- stw r20, 0(r21)
-
+#if 1
+ /* if !swap, you can delete this */
+ rlwimi r20, r20, 5, _PAGE_PRESENT<<5 /* Copy PRESENT to ACCESSED */
+ stw r20, 0(r21) /* Update pte */
+#endif
/* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 21, 22 and 28 must be clear.
+ * Software indicator bits 21 and 28 must be clear.
* Software indicator bits 24, 25, 26, and 27 must be
* set. All other Linux PTE bits control the behavior
* of the MMU.
*/
2: li r21, 0x00f0
- rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
+ rlwimi r20, r21, 0, 0x07f8 /* Set 24-27, clear 21-23,28 */
DO_8xx_CPU6(0x2d80, r3)
mtspr MI_RPN, r20 /* Update TLB entry */
@@ -444,12 +446,25 @@ DataStoreTLBMiss:
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
- mfspr r21, MD_TWC /* get the pte address again */
- ori r20, r20, _PAGE_ACCESSED
- stw r20, 0(r21)
+#if 1
+ /* if !swap, you can delete this */
+ mfspr r21, MD_TWC /* get the pte address */
+ rlwimi r20, r20, 5, _PAGE_PRESENT<<5 /* Copy PRESENT to ACCESSED */
+ stw r20, 0(r21) /* Update pte */
+#endif
+
+ /* Honour kernel RO, User NA */
+ /* 0x200 == Extended encoding, bit 22 */
+ /* r20 |= (r20 & _PAGE_USER) >> 2 */
+ rlwimi r20, r20, 32-2, 0x200
+ /* r21 = (r20 & _PAGE_RW) >> 1 */
+ rlwinm r21, r20, 32-1, 0x200
+ or r20, r21, r20
+ /* invert RW and 0x200 bits */
+ xori r20, r20, _PAGE_RW | 0x200
/* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 21, 22 and 28 must be clear.
+ * Software indicator bits 22 and 28 must be clear.
* Software indicator bits 24, 25, 26, and 27 must be
* set. All other Linux PTE bits control the behavior
* of the MMU.
@@ -496,11 +511,12 @@ DataTLBError:
stw r20, 0(r0)
stw r21, 4(r0)
- /* First, make sure this was a store operation.
- */
mfspr r20, DSISR
- andis. r21, r20, 0x0200 /* If set, indicates store op */
- beq 2f
+ andis. r21, r20, 0x4800 /* !translation or protection */
+ bne- 2f
+ /* Only Change bit left now, do it here as it is faster
+ * than trapping to the C fault handler.
+ */
/* The EA of a data TLB miss is automatically stored in the MD_EPN
* register. The EA of a data TLB error is automatically stored in
@@ -550,17 +566,12 @@ DataTLBError:
mfspr r21, MD_TWC /* ....and get the pte address */
lwz r20, 0(r21) /* Get the pte */
- andi. r21, r20, _PAGE_RW /* Is it writeable? */
- beq 2f /* Bail out if not */
-
- /* Update 'changed', among others.
- */
ori r20, r20, _PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITE
- mfspr r21, MD_TWC /* Get pte address again */
stw r20, 0(r21) /* and update pte in table */
+ xori r20, r20, _PAGE_RW /* RW bit is inverted */
/* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 21, 22 and 28 must be clear.
+ * Software indicator bits 22 and 28 must be clear.
* Software indicator bits 24, 25, 26, and 27 must be
* set. All other Linux PTE bits control the behavior
* of the MMU.
diff --git a/include/asm-ppc/pgtable.h b/include/asm-ppc/pgtable.h
index 71b2165..2ba37d3 100644
--- a/include/asm-ppc/pgtable.h
+++ b/include/asm-ppc/pgtable.h
@@ -298,21 +298,20 @@ extern unsigned long vmalloc_start;
#define _PAGE_NO_CACHE 0x0002 /* I: cache inhibit */
#define _PAGE_SHARED 0x0004 /* No ASID (context) compare */
-/* These five software bits must be masked out when the entry is loaded
- * into the TLB.
+/* These three software bits must be masked out when the entry is loaded
+ * into the TLB, 2 SW bits free.
*/
#define _PAGE_EXEC 0x0008 /* software: i-cache coherency required */
#define _PAGE_GUARDED 0x0010 /* software: guarded access */
-#define _PAGE_DIRTY 0x0020 /* software: page changed */
-#define _PAGE_RW 0x0040 /* software: user write access allowed */
-#define _PAGE_ACCESSED 0x0080 /* software: page referenced */
+#define _PAGE_ACCESSED 0x0020 /* software: page referenced */
/* Setting any bits in the nibble with the follow two controls will
* require a TLB exception handler change. It is assumed unused bits
- * are always zero.
+ * are always zero, encoding(bit 22).
*/
-#define _PAGE_HWWRITE 0x0100 /* h/w write enable: never set in Linux PTE */
-#define _PAGE_USER 0x0800 /* One of the PP bits, the other is USER&~RW */
+#define _PAGE_DIRTY 0x0100 /* Changed: page changed */
+#define _PAGE_RW 0x0400 /* PP lsb(bit 21), user write access allowed */
+#define _PAGE_USER 0x0800 /* PP msb(bit 20), user access allowed */
#define _PMD_PRESENT PAGE_MASK
#define _PMD_PAGE_MASK 0x000c
--
1.7.3.4
^ permalink raw reply related
* [PATCH 07/14] 8xx: CPU6 errata make DTLB error too big to fit.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
branch to common code in DTLB Miss instead.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 23 ++---------------------
1 files changed, 2 insertions(+), 21 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 0891b96..367fec0 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -469,6 +469,7 @@ DataStoreTLBMiss:
* set. All other Linux PTE bits control the behavior
* of the MMU.
*/
+finish_DTLB:
2: li r21, 0x00f0
mtspr DAR, r21 /* Tag DAR */
rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
@@ -578,27 +579,7 @@ DARFixed:
ori r20, r20, _PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITE
stw r20, 0(r21) /* and update pte in table */
xori r20, r20, _PAGE_RW /* RW bit is inverted */
-
- /* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 22 and 28 must be clear.
- * Software indicator bits 24, 25, 26, and 27 must be
- * set. All other Linux PTE bits control the behavior
- * of the MMU.
- */
- li r21, 0x00f0
- mtspr DAR, r21 /* Tag DAR */
- rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
- DO_8xx_CPU6(0x3d80, r3)
- mtspr MD_RPN, r20 /* Update TLB entry */
-
- mfspr r20, M_TW /* Restore registers */
- lwz r21, 0(r0)
- mtcr r21
- lwz r21, 4(r0)
-#ifdef CONFIG_8xx_CPU6
- lwz r3, 8(r0)
-#endif
- rfi
+ b finish_DTLB
2:
mfspr r20, M_TW /* Restore registers */
lwz r21, 0(r0)
--
1.7.3.4
^ permalink raw reply related
* [PATCH 08/14] 8xx: Add missing Guarded setting in DTLB Error.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
only DTLB Miss did set this bit, DTLB Error needs too otherwise
the setting is lost when the page becomes dirty.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 12 +++++++++---
1 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 367fec0..86bc727 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -573,9 +573,15 @@ DARFixed:
ori r21, r21, 1 /* Set valid bit in physical L2 page */
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21 /* Load pte table base address */
- mfspr r21, MD_TWC /* ....and get the pte address */
- lwz r20, 0(r21) /* Get the pte */
-
+ mfspr r20, MD_TWC /* ....and get the pte address */
+ lwz r20, 0(r20) /* Get the pte */
+ /* Insert the Guarded flag into the TWC from the Linux PTE.
+ * It is bit 27 of both the Linux PTE and the TWC
+ */
+ rlwimi r21, r20, 0, 27, 27
+ DO_8xx_CPU6(0x3b80, r3)
+ mtspr MD_TWC, r21
+ mfspr r21, MD_TWC /* get the pte address again */
ori r20, r20, _PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITE
stw r20, 0(r21) /* and update pte in table */
xori r20, r20, _PAGE_RW /* RW bit is inverted */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 09/14] 8xx: Restore _PAGE_WRITETHRU
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
8xx has not had WRITETHRU due to lack of bits in the pte.
After the recent rewrite of the 8xx TLB code, there are
two bits left. Use one of them to WRITETHRU.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 8 ++++++++
include/asm-ppc/pgtable.h | 5 +++--
2 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 86bc727..402158d 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -443,6 +443,10 @@ DataStoreTLBMiss:
* above.
*/
rlwimi r21, r20, 0, 27, 27
+ /* Insert the WriteThru flag into the TWC from the Linux PTE.
+ * It is bit 25 in the Linux PTE and bit 30 in the TWC
+ */
+ rlwimi r21, r20, 32-5, 30, 30
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
@@ -579,6 +583,10 @@ DARFixed:
* It is bit 27 of both the Linux PTE and the TWC
*/
rlwimi r21, r20, 0, 27, 27
+ /* Insert the WriteThru flag into the TWC from the Linux PTE.
+ * It is bit 25 in the Linux PTE and bit 30 in the TWC
+ */
+ rlwimi r21, r20, 32-5, 30, 30
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
mfspr r21, MD_TWC /* get the pte address again */
diff --git a/include/asm-ppc/pgtable.h b/include/asm-ppc/pgtable.h
index 2ba37d3..6cfc5fc 100644
--- a/include/asm-ppc/pgtable.h
+++ b/include/asm-ppc/pgtable.h
@@ -298,12 +298,13 @@ extern unsigned long vmalloc_start;
#define _PAGE_NO_CACHE 0x0002 /* I: cache inhibit */
#define _PAGE_SHARED 0x0004 /* No ASID (context) compare */
-/* These three software bits must be masked out when the entry is loaded
- * into the TLB, 2 SW bits free.
+/* These four software bits must be masked out when the entry is loaded
+ * into the TLB, 1 SW bits left(0x0080).
*/
#define _PAGE_EXEC 0x0008 /* software: i-cache coherency required */
#define _PAGE_GUARDED 0x0010 /* software: guarded access */
#define _PAGE_ACCESSED 0x0020 /* software: page referenced */
+#define _PAGE_WRITETHRU 0x0040 /* software: caching is write through */
/* Setting any bits in the nibble with the follow two controls will
* require a TLB exception handler change. It is assumed unused bits
--
1.7.3.4
^ permalink raw reply related
* [PATCH 10/14] 8xx: Set correct HW pte flags in DTLB Error too
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
DTLB Error needs to adjust the HW PTE bits as DTLB Miss
does.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 402158d..4bcd9b3 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -592,7 +592,12 @@ DARFixed:
mfspr r21, MD_TWC /* get the pte address again */
ori r20, r20, _PAGE_DIRTY|_PAGE_ACCESSED|_PAGE_HWWRITE
stw r20, 0(r21) /* and update pte in table */
- xori r20, r20, _PAGE_RW /* RW bit is inverted */
+ rlwimi r20, r20, 32-2, _PAGE_USER>>2 /* Copy USER to Encoding */
+ /* r21 = (r20 & _PAGE_RW) >> 1 */
+ rlwinm r21, r20, 32-1, _PAGE_RW>>1
+ or r20, r21, r20
+ /* invert RW and 0x200 bits */
+ xori r20, r20, _PAGE_RW | 0x200
b finish_DTLB
2:
mfspr r20, M_TW /* Restore registers */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 12/14] 8xx: Use symbolic constants in TLB asm
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
Use the PTE #defines where possible instead of
hardcoded constants.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 12 ++++++------
1 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 4bcd9b3..0f2101d 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -442,11 +442,11 @@ DataStoreTLBMiss:
* this into the Linux pgd/pmd and load it in the operation
* above.
*/
- rlwimi r21, r20, 0, 27, 27
+ rlwimi r21, r20, 0, _PAGE_GUARDED
/* Insert the WriteThru flag into the TWC from the Linux PTE.
* It is bit 25 in the Linux PTE and bit 30 in the TWC
*/
- rlwimi r21, r20, 32-5, 30, 30
+ rlwimi r21, r20, 32-5, _PAGE_WRITETHRU>>5
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
@@ -460,9 +460,9 @@ DataStoreTLBMiss:
/* Honour kernel RO, User NA */
/* 0x200 == Extended encoding, bit 22 */
/* r20 |= (r20 & _PAGE_USER) >> 2 */
- rlwimi r20, r20, 32-2, 0x200
+ rlwimi r20, r20, 32-2, _PAGE_USER>>2 /* Copy USER to Encoding */
/* r21 = (r20 & _PAGE_RW) >> 1 */
- rlwinm r21, r20, 32-1, 0x200
+ rlwinm r21, r20, 32-1, _PAGE_RW>>1
or r20, r21, r20
/* invert RW and 0x200 bits */
xori r20, r20, _PAGE_RW | 0x200
@@ -582,11 +582,11 @@ DARFixed:
/* Insert the Guarded flag into the TWC from the Linux PTE.
* It is bit 27 of both the Linux PTE and the TWC
*/
- rlwimi r21, r20, 0, 27, 27
+ rlwimi r21, r20, 0, _PAGE_GUARDED
/* Insert the WriteThru flag into the TWC from the Linux PTE.
* It is bit 25 in the Linux PTE and bit 30 in the TWC
*/
- rlwimi r21, r20, 32-5, 30, 30
+ rlwimi r21, r20, 32-5, _PAGE_WRITETHRU>>5
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
mfspr r21, MD_TWC /* get the pte address again */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 11/14] 8xx: start using dcbX instructions in various copy routines
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
Now that 8xx can fixup dcbX instructions, start using them
where possible like every other PowerPc arch do.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/misc.S | 18 ------------------
arch/ppc/lib/string.S | 17 -----------------
2 files changed, 0 insertions(+), 35 deletions(-)
diff --git a/arch/ppc/kernel/misc.S b/arch/ppc/kernel/misc.S
index c616098..c291005 100644
--- a/arch/ppc/kernel/misc.S
+++ b/arch/ppc/kernel/misc.S
@@ -662,15 +662,7 @@ _GLOBAL(__flush_dcache_icache)
_GLOBAL(clear_page)
li r0,4096/L1_CACHE_LINE_SIZE
mtctr r0
-#ifdef CONFIG_8xx
- li r4, 0
-1: stw r4, 0(r3)
- stw r4, 4(r3)
- stw r4, 8(r3)
- stw r4, 12(r3)
-#else
1: dcbz 0,r3
-#endif
addi r3,r3,L1_CACHE_LINE_SIZE
bdnz 1b
blr
@@ -695,15 +687,6 @@ _GLOBAL(copy_page)
addi r3,r3,-4
addi r4,r4,-4
-#ifdef CONFIG_8xx
- /* don't use prefetch on 8xx */
- li r0,4096/L1_CACHE_LINE_SIZE
- mtctr r0
-1: COPY_16_BYTES
- bdnz 1b
- blr
-
-#else /* not 8xx, we can prefetch */
li r5,4
#if MAX_COPY_PREFETCH > 1
@@ -744,7 +727,6 @@ _GLOBAL(copy_page)
li r0,MAX_COPY_PREFETCH
li r11,4
b 2b
-#endif /* CONFIG_8xx */
/*
* Atomic [test&set] exchange
diff --git a/arch/ppc/lib/string.S b/arch/ppc/lib/string.S
index 6ca54b4..b6ea44b 100644
--- a/arch/ppc/lib/string.S
+++ b/arch/ppc/lib/string.S
@@ -159,14 +159,7 @@ _GLOBAL(cacheable_memzero)
bdnz 4b
3: mtctr r9
li r7,4
-#if !defined(CONFIG_8xx)
10: dcbz r7,r6
-#else
-10: stw r4, 4(r6)
- stw r4, 8(r6)
- stw r4, 12(r6)
- stw r4, 16(r6)
-#endif
addi r6,r6,CACHELINE_BYTES
bdnz 10b
clrlwi r5,r8,32-LG_CACHELINE_BYTES
@@ -261,9 +254,7 @@ _GLOBAL(cacheable_memcpy)
mtctr r0
beq 63f
53:
-#if !defined(CONFIG_8xx)
dcbz r11,r6
-#endif
COPY_16_BYTES
#if L1_CACHE_LINE_SIZE >= 32
COPY_16_BYTES
@@ -443,13 +434,6 @@ _GLOBAL(__copy_tofrom_user)
li r11,4
beq 63f
-#ifdef CONFIG_8xx
- /* Don't use prefetch on 8xx */
- mtctr r0
-53: COPY_16_BYTES_WITHEX(0)
- bdnz 53b
-
-#else /* not CONFIG_8xx */
/* Here we decide how far ahead to prefetch the source */
li r3,4
cmpwi r0,1
@@ -502,7 +486,6 @@ _GLOBAL(__copy_tofrom_user)
li r3,4
li r7,0
bne 114b
-#endif /* CONFIG_8xx */
63: srwi. r0,r5,2
mtctr r0
--
1.7.3.4
^ permalink raw reply related
* [PATCH 13/14] 8xx: Optimize TLB Miss handlers
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
Only update pte w.r.t ACCESSED if it isn't already set
Wrap ACCESSED with #ifndef NO_SWAP for too ease optimization.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 11 +++++++++--
1 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 0f2101d..36089cc 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -377,10 +377,14 @@ InstructionTLBMiss:
mfspr r21, MD_TWC /* ....and get the pte address */
lwz r20, 0(r21) /* Get the pte */
-#if 1
+#ifndef NO_SWAP
/* if !swap, you can delete this */
+ andi. r21, r20, _PAGE_ACCESSED /* test ACCESSED bit */
+ bne+ 4f /* Branch if set */
+ mfspr r21, MD_TWC /* get the pte address */
rlwimi r20, r20, 5, _PAGE_PRESENT<<5 /* Copy PRESENT to ACCESSED */
stw r20, 0(r21) /* Update pte */
+4:
#endif
/* The Linux PTE won't go exactly into the MMU TLB.
* Software indicator bits 21 and 28 must be clear.
@@ -450,11 +454,14 @@ DataStoreTLBMiss:
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21
-#if 1
+#ifndef NO_SWAP
/* if !swap, you can delete this */
+ andi. r21, r20, _PAGE_ACCESSED /* test ACCESSED bit */
+ bne+ 4f /* Branch if set */
mfspr r21, MD_TWC /* get the pte address */
rlwimi r20, r20, 5, _PAGE_PRESENT<<5 /* Copy PRESENT to ACCESSED */
stw r20, 0(r21) /* Update pte */
+4:
#endif
/* Honour kernel RO, User NA */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 14/14] 8xx: The TLB miss handler manages ACCESSED correctly.
From: Joakim Tjernlund @ 2011-10-10 11:30 UTC (permalink / raw)
To: linuxppc-dev, Scott Wood, Willy Tarreau, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
The new MMU/TLB code no longer sets ACCESSED unconditionally
so remove the exception.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
include/asm-ppc/pgtable.h | 10 ----------
1 files changed, 0 insertions(+), 10 deletions(-)
diff --git a/include/asm-ppc/pgtable.h b/include/asm-ppc/pgtable.h
index 6cfc5fc..b94e8a8 100644
--- a/include/asm-ppc/pgtable.h
+++ b/include/asm-ppc/pgtable.h
@@ -318,16 +318,6 @@ extern unsigned long vmalloc_start;
#define _PMD_PAGE_MASK 0x000c
#define _PMD_PAGE_8M 0x000c
-/*
- * The 8xx TLB miss handler allegedly sets _PAGE_ACCESSED in the PTE
- * for an address even if _PAGE_PRESENT is not set, as a performance
- * optimization. This is a bug if you ever want to use swap unless
- * _PAGE_ACCESSED is 2, which it isn't, or unless you have 8xx-specific
- * definitions for __swp_entry etc. below, which would be gross.
- * -- paulus
- */
-#define _PTE_NONE_MASK _PAGE_ACCESSED
-
#else /* CONFIG_6xx */
/* Definitions for 60x, 740/750, etc. */
#define _PAGE_PRESENT 0x001 /* software: pte contains a translation */
--
1.7.3.4
^ permalink raw reply related
* [PATCH 0/3] 8xx: Large page(8MB) support for 2.4
From: Joakim Tjernlund @ 2011-10-10 11:38 UTC (permalink / raw)
To: Dan Malek, linuxppc-dev, Scott Wood, Willy Tarreau
This adds Large page support for 8xx and uses it
for all kernel RAM.
Further usage is possible, IMAP_ADDR and on board
flash comes to mind.
There is one bit free the pte which could be used for
selecting different large page sizes but that is for another
day.
- Dan, what do you think :)
Joakim Tjernlund (3):
8xx: replace _PAGE_EXEC with _PAGE_PSE
8xx: Support LARGE pages in TLB code.
8xx: Use LARGE pages for kernel RAM.
arch/ppc/kernel/head_8xx.S | 30 +++++++++++++++++++-----------
arch/ppc/mm/pgtable.c | 4 +++-
include/asm-ppc/pgtable.h | 6 +++++-
3 files changed, 27 insertions(+), 13 deletions(-)
--
1.7.3.4
^ permalink raw reply
* [PATCH 1/3] 8xx: replace _PAGE_EXEC with _PAGE_PSE
From: Joakim Tjernlund @ 2011-10-10 11:38 UTC (permalink / raw)
To: Dan Malek, linuxppc-dev, Scott Wood, Willy Tarreau
In-Reply-To: <1318246714-5015-1-git-send-email-Joakim.Tjernlund@transmode.se>
We need this bit for large pages(8MB). Adjust TLB code
to not clear bit 28 Mx_RPN
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 8 ++++----
include/asm-ppc/pgtable.h | 6 +++++-
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 36089cc..8e3fe40 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -387,13 +387,13 @@ InstructionTLBMiss:
4:
#endif
/* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 21 and 28 must be clear.
+ * Software indicator bit 21 must be clear.
* Software indicator bits 24, 25, 26, and 27 must be
* set. All other Linux PTE bits control the behavior
* of the MMU.
*/
2: li r21, 0x00f0
- rlwimi r20, r21, 0, 0x07f8 /* Set 24-27, clear 21-23,28 */
+ rlwimi r20, r21, 0, 0x07f0 /* Set 24-27, clear 21-23 */
DO_8xx_CPU6(0x2d80, r3)
mtspr MI_RPN, r20 /* Update TLB entry */
@@ -475,7 +475,7 @@ DataStoreTLBMiss:
xori r20, r20, _PAGE_RW | 0x200
/* The Linux PTE won't go exactly into the MMU TLB.
- * Software indicator bits 22 and 28 must be clear.
+ * Software indicator bit 22 must be clear.
* Software indicator bits 24, 25, 26, and 27 must be
* set. All other Linux PTE bits control the behavior
* of the MMU.
@@ -483,7 +483,7 @@ DataStoreTLBMiss:
finish_DTLB:
2: li r21, 0x00f0
mtspr DAR, r21 /* Tag DAR */
- rlwimi r20, r21, 0, 24, 28 /* Set 24-27, clear 28 */
+ rlwimi r20, r21, 0, 0x00f0 /* Set 24-27 */
DO_8xx_CPU6(0x3d80, r3)
mtspr MD_RPN, r20 /* Update TLB entry */
diff --git a/include/asm-ppc/pgtable.h b/include/asm-ppc/pgtable.h
index b94e8a8..1a0ca7b 100644
--- a/include/asm-ppc/pgtable.h
+++ b/include/asm-ppc/pgtable.h
@@ -297,11 +297,11 @@ extern unsigned long vmalloc_start;
#define _PAGE_PRESENT 0x0001 /* Page is valid */
#define _PAGE_NO_CACHE 0x0002 /* I: cache inhibit */
#define _PAGE_SHARED 0x0004 /* No ASID (context) compare */
+#define _PAGE_PSE 0x0008 /* Large Page, 8MB */
/* These four software bits must be masked out when the entry is loaded
* into the TLB, 1 SW bits left(0x0080).
*/
-#define _PAGE_EXEC 0x0008 /* software: i-cache coherency required */
#define _PAGE_GUARDED 0x0010 /* software: guarded access */
#define _PAGE_ACCESSED 0x0020 /* software: page referenced */
#define _PAGE_WRITETHRU 0x0040 /* software: caching is write through */
@@ -359,6 +359,10 @@ extern unsigned long vmalloc_start;
#define _PAGE_EXEC 0
#endif
+#ifndef _PAGE_PSE
+#define _PAGE_PSE 0
+#endif
+
#define _PAGE_CHG_MASK (PAGE_MASK | _PAGE_ACCESSED | _PAGE_DIRTY)
/*
--
1.7.3.4
^ permalink raw reply related
* [PATCH 2/3] 8xx: Support LARGE pages in TLB code.
From: Joakim Tjernlund @ 2011-10-10 11:38 UTC (permalink / raw)
To: Dan Malek, linuxppc-dev, Scott Wood, Willy Tarreau
In-Reply-To: <1318246714-5015-1-git-send-email-Joakim.Tjernlund@transmode.se>
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/kernel/head_8xx.S | 22 +++++++++++++++-------
1 files changed, 15 insertions(+), 7 deletions(-)
diff --git a/arch/ppc/kernel/head_8xx.S b/arch/ppc/kernel/head_8xx.S
index 8e3fe40..439e7f2 100644
--- a/arch/ppc/kernel/head_8xx.S
+++ b/arch/ppc/kernel/head_8xx.S
@@ -368,15 +368,19 @@ InstructionTLBMiss:
* for this "segment."
*/
tophys(r21,r21)
- ori r21,r21,1 /* Set valid bit */
- DO_8xx_CPU6(0x2b80, r3)
- mtspr MI_TWC, r21 /* Set segment attributes */
beq- 2f /* If zero, don't try to find a pte */
DO_8xx_CPU6(0x3b80, r3)
mtspr MD_TWC, r21 /* Load pte table base address */
- mfspr r21, MD_TWC /* ....and get the pte address */
- lwz r20, 0(r21) /* Get the pte */
+ mfspr r20, MD_TWC /* ....and get the pte address */
+ lwz r20, 0(r20) /* Get the pte */
+
+ ori r21, r21, MI_SVALID /* Set valid bit */
+ /* Copy PSE to PS bits(8MB) */
+ rlwimi r21, r20, 0, _PAGE_PSE
+ rlwimi r21, r20, 32-1, _PAGE_PSE>>1
+ DO_8xx_CPU6(0x2b80, r3)
+ mtspr MI_TWC, r21 /* Set segment attributes */
#ifndef NO_SWAP
/* if !swap, you can delete this */
andi. r21, r20, _PAGE_ACCESSED /* test ACCESSED bit */
@@ -446,7 +450,9 @@ DataStoreTLBMiss:
* this into the Linux pgd/pmd and load it in the operation
* above.
*/
- rlwimi r21, r20, 0, _PAGE_GUARDED
+ rlwimi r21, r20, 0, _PAGE_GUARDED | _PAGE_PSE
+ /* Copy PSE to PS bits(8MB), combine with GUARDED above */
+ rlwimi r21, r20, 32-1, _PAGE_PSE>>1
/* Insert the WriteThru flag into the TWC from the Linux PTE.
* It is bit 25 in the Linux PTE and bit 30 in the TWC
*/
@@ -589,7 +595,9 @@ DARFixed:
/* Insert the Guarded flag into the TWC from the Linux PTE.
* It is bit 27 of both the Linux PTE and the TWC
*/
- rlwimi r21, r20, 0, _PAGE_GUARDED
+ rlwimi r21, r20, 0, _PAGE_GUARDED | _PAGE_PSE
+ /* Copy PSE to PS bits(8MB), combine with GUARDED above */
+ rlwimi r21, r20, 32-1, _PAGE_PSE>>1
/* Insert the WriteThru flag into the TWC from the Linux PTE.
* It is bit 25 in the Linux PTE and bit 30 in the TWC
*/
--
1.7.3.4
^ permalink raw reply related
* [PATCH 3/3] 8xx: Use LARGE pages for kernel RAM.
From: Joakim Tjernlund @ 2011-10-10 11:38 UTC (permalink / raw)
To: Dan Malek, linuxppc-dev, Scott Wood, Willy Tarreau
In-Reply-To: <1318246714-5015-1-git-send-email-Joakim.Tjernlund@transmode.se>
Use the new _PAGE_PSE to map all kernel RAM with 8 MB TLBs
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
---
arch/ppc/mm/pgtable.c | 4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/arch/ppc/mm/pgtable.c b/arch/ppc/mm/pgtable.c
index 866ae43..56e847e 100644
--- a/arch/ppc/mm/pgtable.c
+++ b/arch/ppc/mm/pgtable.c
@@ -298,7 +298,9 @@ void __init mapin_ram(void)
/* On the MPC8xx, we want the page shared so we
* don't get ASID compares on kernel space.
*/
- f = _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_SHARED | _PAGE_HWEXEC;
+ f = _PAGE_PSE | _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_SHARED | _PAGE_HWEXEC;
+ if (_PAGE_PSE)
+ f |= _PAGE_WRENABLE;
#if defined(CONFIG_KGDB) || defined(CONFIG_XMON) || defined(CONFIG_BDI_SWITCH)
/* Allows stub to set breakpoints everywhere */
f |= _PAGE_WRENABLE;
--
1.7.3.4
^ permalink raw reply related
* Re: [PATCH 00/14] Backport 8xx TLB to 2.4
From: Willy Tarreau @ 2011-10-10 12:30 UTC (permalink / raw)
To: Joakim Tjernlund; +Cc: Scott Wood, linuxppc-dev, Dan Malek
In-Reply-To: <1318246220-4839-1-git-send-email-Joakim.Tjernlund@transmode.se>
Hi Joakim,
On Mon, Oct 10, 2011 at 01:30:06PM +0200, Joakim Tjernlund wrote:
> This is a backport from 2.6 which I did to overcome 8xx CPU
> bugs. 8xx does not update the DAR register when taking a TLB
> error caused by dcbX and icbi insns which makes it very
> tricky to use these insns. Also the dcbst wrongly sets the
> the store bit when faulting into DTLB error.
> A few more bugs very found during development.
>
> I know 2.4 is in strict maintenance mode and 8xx is obsolete
> but as it is still in use I wanted 8xx to age with grace.
Thank you. I must admit I was hoping those patches would come in
for a last release before the end of the year :-)
Unless there is any objection from anyone, I'll merge them when
kernel.org is back online.
Cheers,
Willy
^ permalink raw reply
* Re: [PATCH 1/3] [powerpc32] Process dynamic relocations for kernel
From: Scott Wood @ 2011-10-10 15:15 UTC (permalink / raw)
To: Suzuki K. Poulose
Cc: Michal Simek, tmarri, Mahesh Jagannath Salgaonkar, Dave Hansen,
David Laight, Paul Mackerras, Alan Modra, linux ppc dev,
Vivek Goyal
In-Reply-To: <20111010095432.16589.32935.stgit@suzukikp.in.ibm.com>
On 10/10/2011 04:55 AM, Suzuki K. Poulose wrote:
> The following patch implements the dynamic relocation processing for
> PPC32 kernel. relocate() accepts the target virtual address and relocates
> the kernel image to the same.
How much overhead is involved in a true relocatable kernel? Is it worth
preserving the old "relocatable" booke behavior under a different name?
-Scott
^ permalink raw reply
* Re: [PATCH 0/3] 8xx: Large page(8MB) support for 2.4
From: Willy Tarreau @ 2011-10-10 16:29 UTC (permalink / raw)
To: Dan Malek; +Cc: Scott Wood, linuxppc-dev
In-Reply-To: <39B024CE-70A2-4D32-A29C-5702C7D2D6CF@digitaldans.com>
Hi an,
On Mon, Oct 10, 2011 at 09:22:09AM -0700, Dan Malek wrote:
>
> Hi Joakim.
>
> On Oct 10, 2011, at 4:38 AM, Joakim Tjernlund wrote:
>
> >This adds Large page support for 8xx and uses it
> >for all kernel RAM....
>
> >- Dan, what do you think :)
>
> Since you asked, yes it looks great :-) Now, can we
> get this into a more contemporary kernel? I'm
> actually working on an 8xx project that may have
> a few years of life left.
At the pace of current 2.4, I'm sure the code won't have change
much a few years from now :-) It would be nice to know by now if
the current longterm branches work OK or not though.
Cheers,
Willy
^ permalink raw reply
* Re: [PATCH 0/3] 8xx: Large page(8MB) support for 2.4
From: Dan Malek @ 2011-10-10 16:22 UTC (permalink / raw)
To: Joakim Tjernlund; +Cc: Scott Wood, linuxppc-dev, Willy Tarreau
In-Reply-To: <1318246714-5015-1-git-send-email-Joakim.Tjernlund@transmode.se>
Hi Joakim.
On Oct 10, 2011, at 4:38 AM, Joakim Tjernlund wrote:
> This adds Large page support for 8xx and uses it
> for all kernel RAM....
> - Dan, what do you think :)
Since you asked, yes it looks great :-) Now, can we
get this into a more contemporary kernel? I'm
actually working on an 8xx project that may have
a few years of life left.
Thanks.
-- Dan
^ permalink raw reply
* [PATCH] mlx4_en: fix endianness with blue frame support
From: Thadeu Lima de Souza Cascardo @ 2011-10-10 16:42 UTC (permalink / raw)
To: netdev
Cc: Eli Cohen, linuxppc-dev, Thadeu Lima de Souza Cascardo,
Yevgeny Petrilin
In-Reply-To: <1318231920.29415.404.camel@pasglop>
The doorbell register was being unconditionally swapped. In x86, that
meant it was being swapped to BE and written to the descriptor and to
memory, depending on the case of blue frame support or writing to
doorbell register. On PPC, this meant it was being swapped to LE and
then swapped back to BE while writing to the register. But in the blue
frame case, it was being written as LE to the descriptor.
The fix is not to swap doorbell unconditionally, write it to the
register as BE and convert it to BE when writing it to the descriptor.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@linux.vnet.ibm.com>
Reported-by: Richard Hendrickson <richhend@us.ibm.com>
Cc: Eli Cohen <eli@dev.mellanox.co.il>
Cc: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
drivers/net/mlx4/en_tx.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/mlx4/en_tx.c b/drivers/net/mlx4/en_tx.c
index 6e03de0..f76ab6b 100644
--- a/drivers/net/mlx4/en_tx.c
+++ b/drivers/net/mlx4/en_tx.c
@@ -172,7 +172,7 @@ int mlx4_en_activate_tx_ring(struct mlx4_en_priv *priv,
memset(ring->buf, 0, ring->buf_size);
ring->qp_state = MLX4_QP_STATE_RST;
- ring->doorbell_qpn = swab32(ring->qp.qpn << 8);
+ ring->doorbell_qpn = ring->qp.qpn << 8;
mlx4_en_fill_qp_context(priv, ring->size, ring->stride, 1, 0, ring->qpn,
ring->cqn, &ring->context);
@@ -791,7 +791,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
skb_orphan(skb);
if (ring->bf_enabled && desc_size <= MAX_BF && !bounce && !vlan_tag) {
- *(u32 *) (&tx_desc->ctrl.vlan_tag) |= ring->doorbell_qpn;
+ *(__be32 *) (&tx_desc->ctrl.vlan_tag) |= cpu_to_be32(ring->doorbell_qpn);
op_own |= htonl((bf_index & 0xffff) << 8);
/* Ensure new descirptor hits memory
* before setting ownership of this descriptor to HW */
@@ -812,7 +812,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
wmb();
tx_desc->ctrl.owner_opcode = op_own;
wmb();
- writel(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
+ iowrite32be(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
}
/* Poll CQ here */
--
1.7.4.4
^ permalink raw reply related
* Re: [PATCH] mlx4_en: fix endianness with blue frame support
From: Thadeu Lima de Souza Cascardo @ 2011-10-10 16:46 UTC (permalink / raw)
To: netdev; +Cc: Eli Cohen, linuxppc-dev, Yevgeny Petrilin
In-Reply-To: <1318264943-10009-1-git-send-email-cascardo@linux.vnet.ibm.com>
On Mon, Oct 10, 2011 at 01:42:23PM -0300, Thadeu Lima de Souza Cascardo wrote:
> The doorbell register was being unconditionally swapped. In x86, that
> meant it was being swapped to BE and written to the descriptor and to
> memory, depending on the case of blue frame support or writing to
> doorbell register. On PPC, this meant it was being swapped to LE and
> then swapped back to BE while writing to the register. But in the blue
> frame case, it was being written as LE to the descriptor.
>
> The fix is not to swap doorbell unconditionally, write it to the
> register as BE and convert it to BE when writing it to the descriptor.
>
> Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@linux.vnet.ibm.com>
> Reported-by: Richard Hendrickson <richhend@us.ibm.com>
> Cc: Eli Cohen <eli@dev.mellanox.co.il>
> Cc: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> ---
So I tested this patch and it works for me. Thanks Ben and Eli for
finding out the problem with doorbell in the descriptor.
Regards,
Cascardo.
> drivers/net/mlx4/en_tx.c | 6 +++---
> 1 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/mlx4/en_tx.c b/drivers/net/mlx4/en_tx.c
> index 6e03de0..f76ab6b 100644
> --- a/drivers/net/mlx4/en_tx.c
> +++ b/drivers/net/mlx4/en_tx.c
> @@ -172,7 +172,7 @@ int mlx4_en_activate_tx_ring(struct mlx4_en_priv *priv,
> memset(ring->buf, 0, ring->buf_size);
>
> ring->qp_state = MLX4_QP_STATE_RST;
> - ring->doorbell_qpn = swab32(ring->qp.qpn << 8);
> + ring->doorbell_qpn = ring->qp.qpn << 8;
>
> mlx4_en_fill_qp_context(priv, ring->size, ring->stride, 1, 0, ring->qpn,
> ring->cqn, &ring->context);
> @@ -791,7 +791,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
> skb_orphan(skb);
>
> if (ring->bf_enabled && desc_size <= MAX_BF && !bounce && !vlan_tag) {
> - *(u32 *) (&tx_desc->ctrl.vlan_tag) |= ring->doorbell_qpn;
> + *(__be32 *) (&tx_desc->ctrl.vlan_tag) |= cpu_to_be32(ring->doorbell_qpn);
> op_own |= htonl((bf_index & 0xffff) << 8);
> /* Ensure new descirptor hits memory
> * before setting ownership of this descriptor to HW */
> @@ -812,7 +812,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
> wmb();
> tx_desc->ctrl.owner_opcode = op_own;
> wmb();
> - writel(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
> + iowrite32be(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
> }
>
> /* Poll CQ here */
> --
> 1.7.4.4
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox