Re: [PATCH] SLB shadow buffer

Re: [PATCH] SLB shadow buffer

[Milton Miller]

On Fri Aug  4 2006 12:53:19 AM CDT, Michael Neuling wrote:
> This adds a shadow buffer for the SLBs and regsiters it with PHYP.
> Only the bolted SLB entries (first 3) are saved.
> 
> Signed-off-by: Michael Neuling <mikey@neuling.org>
> 
> 
> Index: linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
> ===================================================================
> --- linux-2.6-ozlabs.orig/arch/powerpc/kernel/entry_64.S
> +++ linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
> @@ -323,6 +323,10 @@ _GLOBAL(ret_from_fork)
>   * The code which creates the new task context is in 'copy_thread'
>   * in arch/powerpc/kernel/process.c 
>   */
> +#define SHADOW_SLB_BOLTED_LAST_ESID \
> +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1))
> +#define SHADOW_SLB_BOLTED_LAST_VSID \
> +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1) + 8)

It's not because it's the last one but that it's the one with the stack.

>  	.align	7
>  _GLOBAL(_switch)
>  	mflr	r0
> @@ -375,6 +379,14 @@ BEGIN_FTR_SECTION
>  	ld	r7,KSP_VSID(r4)	/* Get new stack's VSID */
>  	oris	r0,r6,(SLB_ESID_V)@h
>  	ori	r0,r0,(SLB_NUM_BOLTED-1)@l
> +
> +	/* Update the last bolted SLB */
> +	ld	r9,PACA_SLBSHADOWPTR(r13)
> + 	li	r12,0
> +  	std     r12,SHADOW_SLB_BOLTED_LAST_ESID(r9) /* Clear ESID */
> +	std     r7,SHADOW_SLB_BOLTED_LAST_VSID(r9)  /* Save VSID */
> + 	std     r0,SHADOW_SLB_BOLTED_LAST_ESID(r9)  /* Save ESID */
> +

Tabs not spaces please

>  	slbie	r6
>  	slbie	r6		/* Workaround POWER5 < DD2.1 issue */
>  	slbmte	r7,r0
> Index: linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
> ===================================================================
> --- linux-2.6-ozlabs.orig/arch/powerpc/kernel/paca.c
> +++ linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
> @@ -17,6 +17,7 @@
>  #include <asm/lppaca.h>
>  #include <asm/iseries/it_lp_reg_save.h>
>  #include <asm/paca.h>
> +#include <asm/mmu.h>
>  
>  
>  /* This symbol is provided by the linker - let it fill in the paca
> @@ -45,6 +46,17 @@ struct lppaca lppaca[] = {
>  	},
>  };
>  
> +/*
> + * 3 persistent SLBs are registered here.  The buffer will be zero
> + * initially, hence will all be invaild until we actually write them.
> + */
> +struct slb_shadow_buffer slb_shadow_buffer[] = {
> +	[0 ... (NR_CPUS-1)] = {
> +		.persistent = SLB_NUM_BOLTED,
> +		.buffer_length = sizeof(struct slb_shadow_buffer),
> +	},
> +};

How about making this per-cpu and setting the paca pointer at runtime?
It would save NR_CPUS-1 cachelines of data.   Otherwise, put  in
cacheline_aligned will potentially save some space.

> +
>  /* The Paca is an array with one entry per processor.  Each contains an
>   * lppaca, which contains the information shared between the
>   * hypervisor and Linux.
> @@ -59,7 +71,8 @@ struct lppaca lppaca[] = {
>  	.lock_token = 0x8000,						    \
>  	.paca_index = (number),		/* Paca Index */		    \
>  	.kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL,		    \
> -	.hw_cpu_id = 0xffff,
> +	.hw_cpu_id = 0xffff,						    \
> +	.slb_shadow_buffer_ptr = &slb_shadow_buffer[number]

Trailing , (yeah, still have to add the \ )

>  
>  #ifdef CONFIG_PPC_ISERIES
>  #define PACA_INIT_ISERIES(number)					    \
> Index: linux-2.6-ozlabs/arch/powerpc/mm/slb.c
> ===================================================================
> --- linux-2.6-ozlabs.orig/arch/powerpc/mm/slb.c
> +++ linux-2.6-ozlabs/arch/powerpc/mm/slb.c
> @@ -22,6 +22,8 @@
>  #include <asm/paca.h>
>  #include <asm/cputable.h>
>  #include <asm/cacheflush.h>
> +#include <asm/smp.h>
> +#include <linux/compiler.h>
>  
>  #ifdef DEBUG
>  #define DBG(fmt...) udbg_printf(fmt)
> @@ -50,9 +52,29 @@ static inline unsigned long mk_vsid_data
>  	return (get_kernel_vsid(ea) << SLB_VSID_SHIFT) | flags;
>  }
>  
> +static inline void slb_shadow_update(unsigned long esid, unsigned long vsid,
> +				     unsigned long entry)
> +{
> +	/* Clear the ESID first so the entry is not valid while we are
> +	 * updating it.  Then write the VSID before the real ESID. */

Multi-line comments get the  */ on it's own line

> +	get_slb_shadow_buffer()->save_area[2*entry] = 0;
> +	barrier();
> +	get_slb_shadow_buffer()->save_area[2*entry+1] =	vsid;
> +	barrier();
> +	get_slb_shadow_buffer()->save_area[2*entry] = esid;
> +
> +}

This 2* seems magic.  How about an array of structs?

> +
>  static inline void create_slbe(unsigned long ea, unsigned long flags,
>  			       unsigned long entry)

Should we rename to create_shadowed_slbe?

>  {
> +	/* Updating the shadow buffer before writing the SLB ensures
> +	 * we don't get a stale entry here if we get preempted by PHYP
> +	 * between these two statements. */

own line

> +	slb_shadow_update(mk_esid_data(ea, entry),
> +			  mk_vsid_data(ea, flags),
> +			  entry);

Do we need the third line?

> +
>  	asm volatile("slbmte  %0,%1" :
>  		     : "r" (mk_vsid_data(ea, flags)),
>  		       "r" (mk_esid_data(ea, entry))

Hopefully the compiler only calculates these once.

> @@ -77,6 +99,11 @@ void slb_flush_and_rebolt(void)
>  	if ((ksp_esid_data & ESID_MASK) == PAGE_OFFSET)
>  		ksp_esid_data &= ~SLB_ESID_V;
>  
> +	/* Only second entry may change here so only resave that */
> +	slb_shadow_update(ksp_esid_data,
> +			  mk_vsid_data(ksp_esid_data, lflags),
> +			  2);
> +

1) it's the third entry 2) it's the stack slot

>  	/* We need to do this all in asm, so we're sure we don't touch
>  	 * the stack between the slbia and rebolting it. */

own line

>  	asm volatile("isync\n"
> Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
> ===================================================================
> --- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/lpar.c
> +++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
> @@ -254,18 +254,32 @@ out:
>  void vpa_init(int cpu)
>  {
>  	int hwcpu = get_hard_smp_processor_id(cpu);
> -	unsigned long vpa = __pa(&lppaca[cpu]);
> +	unsigned long vpa;

This is used for something  other than vpa, rename.

>  	long ret;
>  
>  	if (cpu_has_feature(CPU_FTR_ALTIVEC))
>  		lppaca[cpu].vmxregs_in_use = 1;
>  
> +	vpa = __pa(&lppaca[cpu]);
>  	ret = register_vpa(hwcpu, vpa);
>  
> -	if (ret)
> +	if (ret) {
>  		printk(KERN_ERR "WARNING: vpa_init: VPA registration for "
>  				"cpu %d (hw %d) of area %lx returns %ld\n",
>  				cpu, hwcpu, vpa, ret);
> +		return;
> +	}
> +	/* PAPR says this feature is SLB-Buffer but firmware never
> +	 * reports that.  All SPLPAR support SLB shadow buffer. */

own line

> +	vpa = __pa(&slb_shadow_buffer[cpu]);
> +	if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
> +		ret = register_slb_shadow(hwcpu, vpa);
> +		if (ret)
> +			printk(KERN_ERR
> +			       "WARNING: vpa_init: SLB shadow buffer "
> +			       "registration for cpu %d (hw %d) of area %lx "
> +			       "returns %ld\n", cpu, hwcpu, vpa, ret);
> +	}
>  }
>  
>  long pSeries_lpar_hpte_insert(unsigned long hpte_group,
...
> Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
> ===================================================================
> --- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/setup.c
> +++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
> @@ -234,8 +234,16 @@ static void pseries_kexec_cpu_down_xics(
>  {
>  	/* Don't risk a hypervisor call if we're crashing */
>  	if (firmware_has_feature(FW_FEATURE_SPLPAR) && !crash_shutdown) {
> -		unsigned long vpa = __pa(get_lppaca());
> +		unsigned long vpa;

Rename.

>  
> +		vpa = __pa(get_slb_shadow_buffer());
> +		if (unregister_slb_shadow(hard_smp_processor_id(), vpa))
> +			printk("SLB shadow buffer deregistration of "
> +			       "cpu %u (hw_cpu_id %d) failed\n",
> +			       smp_processor_id(),
> +			       hard_smp_processor_id());
> +
> +		vpa = __pa(get_lppaca());
>  		if (unregister_vpa(hard_smp_processor_id(), vpa)) {
>  			printk("VPA deregistration of cpu %u (hw_cpu_id %d) "
>  					"failed\n", smp_processor_id(),


Side comment: this seems like it should be in pseeries/lpar.c, that would
be a seperate cleanup.

> Index: linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
> ===================================================================
> --- linux-2.6-ozlabs.orig/include/asm-powerpc/lppaca.h
> +++ linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
> @@ -28,6 +28,7 @@
>  //
>  //----------------------------------------------------------------------------
>  #include <asm/types.h>
> +#include <asm/mmu.h>
>  
>  /* The Hypervisor barfs if the lppaca crosses a page boundary.  A 1k
>   * alignment is sufficient to prevent this */
> @@ -133,5 +134,19 @@ struct lppaca {
>  
>  extern struct lppaca lppaca[];
>  
> +/* SLB shadow buffer structure as defined in the PAPR.  The save_area
> + * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
> + * ESID is stored in the lower 64bits, then the VSID.  NOTE: This
> + * structure is 0x40 bytes long (with 3 bolted SLBs), but PHYP
> + * complaints if we're not 0x80 (cache line?) aligned.  */

Own line

Less explaination needed with array of structs.

> +struct slb_shadow_buffer {
> +	u32	persistent;		// Number of persistent SLBs	x00-x03
> +	u32	buffer_length;		// Total shadow buffer length	x04-x07
> +	u64	reserved;		// Alignment			x08-x0f
> +	u64     save_area[SLB_NUM_BOLTED * 2];	//			x10-x40
> +} __attribute__((__aligned__(0x80)));
> +
> +extern struct slb_shadow_buffer slb_shadow_buffer[];
> +

Remove buffer from these names? They seem quite long for linux.
 
>  #endif /* __KERNEL__ */
>  #endif /* _ASM_POWERPC_LPPACA_H */
> Index: linux-2.6-ozlabs/include/asm-powerpc/paca.h
> ===================================================================
> --- linux-2.6-ozlabs.orig/include/asm-powerpc/paca.h
> +++ linux-2.6-ozlabs/include/asm-powerpc/paca.h
> @@ -23,6 +23,7 @@
>  register struct paca_struct *local_paca asm("r13");
>  #define get_paca()	local_paca
>  #define get_lppaca()	(get_paca()->lppaca_ptr)
> +#define get_slb_shadow_buffer()	(get_paca()->slb_shadow_buffer_ptr)
>  
>  struct task_struct;
>  
> @@ -98,6 +99,9 @@ struct paca_struct {
>  	u64 user_time;			/* accumulated usermode TB ticks */
>  	u64 system_time;		/* accumulated system TB ticks */
>  	u64 startpurr;			/* PURR/TB value snapshot */
> +
> +	/* Pointer to SLB shadow buffer */
> +	struct slb_shadow_buffer *slb_shadow_buffer_ptr;
>  };

With the long name the comment doesn't add much.

>  
>  extern struct paca_struct paca[];
> 

Re: [PATCH] SLB shadow buffer

[Michael Neuling]

> > +#define SHADOW_SLB_BOLTED_LAST_ESID \
> > +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1))
> > +#define SHADOW_SLB_BOLTED_LAST_VSID \
> > +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1) + 8)
> 
> It's not because it's the last one but that it's the one with the stack.

Yep changed to SHADOW_SLB_BOLTED_STACK_<blah>

> > +  	std     r12,SHADOW_SLB_BOLTED_LAST_ESID(r9) /* Clear ESID */
> > +	std     r7,SHADOW_SLB_BOLTED_LAST_VSID(r9)  /* Save VSID */
> > + 	std     r0,SHADOW_SLB_BOLTED_LAST_ESID(r9)  /* Save ESID */
> > +
> 
> Tabs not spaces please

Oops.. done

> > +/*
> > + * 3 persistent SLBs are registered here.  The buffer will be zero
> > + * initially, hence will all be invaild until we actually write them.
> > + */
> > +struct slb_shadow_buffer slb_shadow_buffer[] = {
> > +	[0 ... (NR_CPUS-1)] = {
> > +		.persistent = SLB_NUM_BOLTED,
> > +		.buffer_length = sizeof(struct slb_shadow_buffer),
> > +	},
> > +};
> 
> How about making this per-cpu and setting the paca pointer at runtime?
> It would save NR_CPUS-1 cachelines of data.   Otherwise, put  in
> cacheline_aligned will potentially save some space.

The PAPR requirement is that it's cache aligned so I've added the
necessary  __cacheline_aligned foo.

I did some experiments and without the alignment.  Alignment costs
around 8KB when NR_CPUS=128 so there is some space to be saved if we
play around.

I also tried putting the structure directly in the PACA but I couldn't
get any savings.  I tired in a few different locations in the paca
struct, but it didn't make any difference.  I may have been doing
something stupid though :-)

> 
> > +
> >  /* The Paca is an array with one entry per processor.  Each contains an
> >   * lppaca, which contains the information shared between the
> >   * hypervisor and Linux.
> > @@ -59,7 +71,8 @@ struct lppaca lppaca[] = {
> >  	.lock_token = 0x8000,						    \
> >  	.paca_index = (number),		/* Paca Index */		    \
> >  	.kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL,		    \
> > -	.hw_cpu_id = 0xffff,
> > +	.hw_cpu_id = 0xffff,						    \
> > +	.slb_shadow_buffer_ptr = &slb_shadow_buffer[number]
> 
> Trailing , (yeah, still have to add the \ )

Oops.. done

> 
> >  
> >  #ifdef CONFIG_PPC_ISERIES
> >  #define PACA_INIT_ISERIES(number)					    \
> > Index: linux-2.6-ozlabs/arch/powerpc/mm/slb.c
> > ===================================================================
> > --- linux-2.6-ozlabs.orig/arch/powerpc/mm/slb.c
> > +++ linux-2.6-ozlabs/arch/powerpc/mm/slb.c
> > @@ -22,6 +22,8 @@
> >  #include <asm/paca.h>
> >  #include <asm/cputable.h>
> >  #include <asm/cacheflush.h>
> > +#include <asm/smp.h>
> > +#include <linux/compiler.h>
> >  
> >  #ifdef DEBUG
> >  #define DBG(fmt...) udbg_printf(fmt)
> > @@ -50,9 +52,29 @@ static inline unsigned long mk_vsid_data
> >  	return (get_kernel_vsid(ea) << SLB_VSID_SHIFT) | flags;
> >  }
> >  
> > +static inline void slb_shadow_update(unsigned long esid, unsigned long vsi
d,
> > +				     unsigned long entry)
> > +{
> > +	/* Clear the ESID first so the entry is not valid while we are
> > +	 * updating it.  Then write the VSID before the real ESID. */
> 
> Multi-line comments get the  */ on it's own line

Ok... (most comments in that file are like this though)

> > +	get_slb_shadow_buffer()->save_area[2*entry] = 0;
> > +	barrier();
> > +	get_slb_shadow_buffer()->save_area[2*entry+1] =	vsid;
> > +	barrier();
> > +	get_slb_shadow_buffer()->save_area[2*entry] = esid;
> > +
> > +}
> 
> This 2* seems magic.  How about an array of structs?

OK, that's two people I've confused.  Changed.

> 
> > +
> >  static inline void create_slbe(unsigned long ea, unsigned long flags,
> >  			       unsigned long entry)
> 
> Should we rename to create_shadowed_slbe?

Good point. Changed


> > +	slb_shadow_update(mk_esid_data(ea, entry),
> > +			  mk_vsid_data(ea, flags),
> > +			  entry);
> 
> Do we need the third line?

Nope.

> 
> > +
> >  	asm volatile("slbmte  %0,%1" :
> >  		     : "r" (mk_vsid_data(ea, flags)),
> >  		       "r" (mk_esid_data(ea, entry))
> 
> Hopefully the compiler only calculates these once.

I've been keeping all my fingers and toes crossed...

> > @@ -77,6 +99,11 @@ void slb_flush_and_rebolt(void)
> >  	if ((ksp_esid_data & ESID_MASK) == PAGE_OFFSET)
> >  		ksp_esid_data &= ~SLB_ESID_V;
> >  
> > +	/* Only second entry may change here so only resave that */
> > +	slb_shadow_update(ksp_esid_data,
> > +			  mk_vsid_data(ksp_esid_data, lflags),
> > +			  2);
> > +
> 
> 1) it's the third entry 2) it's the stack slot

Yep.  Updated comment.

> 
> >  	asm volatile("isync\n"
> > Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
> > ===================================================================
> > --- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/lpar.c
> > +++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
> > @@ -254,18 +254,32 @@ out:
> >  void vpa_init(int cpu)
> >  {
> >  	int hwcpu = get_hard_smp_processor_id(cpu);
> > -	unsigned long vpa = __pa(&lppaca[cpu]);
> > +	unsigned long vpa;
> 
> This is used for something  other than vpa, rename.

Another complaint about this, I'll changed.

BTW It's called vpa in the PAPR for both calls.

> > +/* SLB shadow buffer structure as defined in the PAPR.  The save_area
> > + * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
> > + * ESID is stored in the lower 64bits, then the VSID.  NOTE: This
> > + * structure is 0x40 bytes long (with 3 bolted SLBs), but PHYP
> > + * complaints if we're not 0x80 (cache line?) aligned.  */
> 
> Own line
> 
> Less explaination needed with array of structs.

Yep and yep.

> 
> > +struct slb_shadow_buffer {
> > +	u32	persistent;		// Number of persistent SLBs	x00-x03
> > +	u32	buffer_length;		// Total shadow buffer length	x04-x07
> > +	u64	reserved;		// Alignment			x08-x0f
> > +	u64     save_area[SLB_NUM_BOLTED * 2];	//			x10-x40
> > +} __attribute__((__aligned__(0x80)));
> > +
> > +extern struct slb_shadow_buffer slb_shadow_buffer[];
> > +
> 
> Remove buffer from these names? They seem quite long for linux.

Ok.  Changed.

> >  	u64 user_time;			/* accumulated usermode TB ticks */
> >  	u64 system_time;		/* accumulated system TB ticks */
> >  	u64 startpurr;			/* PURR/TB value snapshot */
> > +
> > +	/* Pointer to SLB shadow buffer */
> > +	struct slb_shadow_buffer *slb_shadow_buffer_ptr;
> >  };
> 
> With the long name the comment doesn't add much.

True.  The comment was only there since most other items in that struct
were documented.

Thanks for the review.  I'll repost the updated patch.

Mikey

[PATCH] SLB shadow buffer

[Michael Neuling]

This adds a shadow buffer for the SLBs and regsiters it with PHYP.
Only the bolted SLB entries (top 3) are shadowed.

Signed-off-by: Michael Neuling <mikey@neuling.org>
---
Updated with more comments from Milton

 arch/powerpc/kernel/asm-offsets.c               |    7 ++++
 arch/powerpc/kernel/entry_64.S                  |    8 +++++
 arch/powerpc/kernel/paca.c                      |   15 +++++++++
 arch/powerpc/mm/slb.c                           |   37 +++++++++++++++++++++---
 arch/powerpc/platforms/pseries/lpar.c           |   24 ++++++++++++---
 arch/powerpc/platforms/pseries/plpar_wrappers.h |   10 ++++++
 arch/powerpc/platforms/pseries/setup.c          |   12 ++++++-
 include/asm-powerpc/lppaca.h                    |   19 ++++++++++++
 include/asm-powerpc/paca.h                      |    3 +
 9 files changed, 124 insertions(+), 11 deletions(-)

Index: linux-2.6-ozlabs/arch/powerpc/kernel/asm-offsets.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/asm-offsets.c
+++ linux-2.6-ozlabs/arch/powerpc/kernel/asm-offsets.c
@@ -42,6 +42,7 @@
 #include <asm/lppaca.h>
 #include <asm/cache.h>
 #include <asm/compat.h>
+#include <asm/mmu.h>
 #endif
 
 #define DEFINE(sym, val) \
@@ -135,11 +136,17 @@ int main(void)
 	DEFINE(PACA_STARTPURR, offsetof(struct paca_struct, startpurr));
 	DEFINE(PACA_USER_TIME, offsetof(struct paca_struct, user_time));
 	DEFINE(PACA_SYSTEM_TIME, offsetof(struct paca_struct, system_time));
+	DEFINE(PACA_SLBSHADOWPTR, offsetof(struct paca_struct, slb_shadow_ptr));
 
 	DEFINE(LPPACASRR0, offsetof(struct lppaca, saved_srr0));
 	DEFINE(LPPACASRR1, offsetof(struct lppaca, saved_srr1));
 	DEFINE(LPPACAANYINT, offsetof(struct lppaca, int_dword.any_int));
 	DEFINE(LPPACADECRINT, offsetof(struct lppaca, int_dword.fields.decr_int));
+
+	DEFINE(SLBSHADOW_STACKVSID,
+	       offsetof(struct slb_shadow, save_area[SLB_NUM_BOLTED - 1].vsid));
+	DEFINE(SLBSHADOW_STACKESID,
+	       offsetof(struct slb_shadow, save_area[SLB_NUM_BOLTED - 1].esid));
 #endif /* CONFIG_PPC64 */
 
 	/* RTAS */
Index: linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/entry_64.S
+++ linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
@@ -375,6 +375,14 @@ BEGIN_FTR_SECTION
 	ld	r7,KSP_VSID(r4)	/* Get new stack's VSID */
 	oris	r0,r6,(SLB_ESID_V)@h
 	ori	r0,r0,(SLB_NUM_BOLTED-1)@l
+
+	/* Update the last bolted SLB */
+	ld	r9,PACA_SLBSHADOWPTR(r13)
+	li	r12,0
+	std	r12,SLBSHADOW_STACKESID(r9) /* Clear ESID */
+	std	r7,SLBSHADOW_STACKVSID(r9)  /* Save VSID */
+	std	r0,SLBSHADOW_STACKESID(r9)  /* Save ESID */
+
 	slbie	r6
 	slbie	r6		/* Workaround POWER5 < DD2.1 issue */
 	slbmte	r7,r0
Index: linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/paca.c
+++ linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
@@ -17,6 +17,7 @@
 #include <asm/lppaca.h>
 #include <asm/iseries/it_lp_reg_save.h>
 #include <asm/paca.h>
+#include <asm/mmu.h>
 
 
 /* This symbol is provided by the linker - let it fill in the paca
@@ -45,6 +46,17 @@ struct lppaca lppaca[] = {
 	},
 };
 
+/*
+ * 3 persistent SLBs are registered here.  The buffer will be zero
+ * initially, hence will all be invaild until we actually write them.
+ */
+struct slb_shadow slb_shadow[] __cacheline_aligned = {
+	[0 ... (NR_CPUS-1)] = {
+		.persistent = SLB_NUM_BOLTED,
+		.buffer_length = sizeof(struct slb_shadow),
+	},
+};
+
 /* The Paca is an array with one entry per processor.  Each contains an
  * lppaca, which contains the information shared between the
  * hypervisor and Linux.
@@ -59,7 +71,8 @@ struct lppaca lppaca[] = {
 	.lock_token = 0x8000,						    \
 	.paca_index = (number),		/* Paca Index */		    \
 	.kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL,		    \
-	.hw_cpu_id = 0xffff,
+	.hw_cpu_id = 0xffff,						    \
+	.slb_shadow_ptr = &slb_shadow[number],
 
 #ifdef CONFIG_PPC_ISERIES
 #define PACA_INIT_ISERIES(number)					    \
Index: linux-2.6-ozlabs/arch/powerpc/mm/slb.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/mm/slb.c
+++ linux-2.6-ozlabs/arch/powerpc/mm/slb.c
@@ -22,6 +22,8 @@
 #include <asm/paca.h>
 #include <asm/cputable.h>
 #include <asm/cacheflush.h>
+#include <asm/smp.h>
+#include <linux/compiler.h>
 
 #ifdef DEBUG
 #define DBG(fmt...) udbg_printf(fmt)
@@ -50,9 +52,32 @@ static inline unsigned long mk_vsid_data
 	return (get_kernel_vsid(ea) << SLB_VSID_SHIFT) | flags;
 }
 
-static inline void create_slbe(unsigned long ea, unsigned long flags,
-			       unsigned long entry)
+static inline void slb_shadow_update(unsigned long esid, unsigned long vsid,
+				     unsigned long entry)
 {
+	/*
+	 * Clear the ESID first so the entry is not valid while we are
+	 * updating it.
+	 */
+	get_slb_shadow()->save_area[entry].esid = 0;
+	barrier();
+	get_slb_shadow()->save_area[entry].vsid = vsid;
+	barrier();
+	get_slb_shadow()->save_area[entry].esid = esid;
+
+}
+
+static inline void create_shadowed_slbe(unsigned long ea, unsigned long flags,
+					unsigned long entry)
+{
+	/*
+	 * Updating the shadow buffer before writing the SLB ensures
+	 * we don't get a stale entry here if we get preempted by PHYP
+	 * between these two statements.
+	 */
+	slb_shadow_update(mk_esid_data(ea, entry), mk_vsid_data(ea, flags),
+			  entry);
+
 	asm volatile("slbmte  %0,%1" :
 		     : "r" (mk_vsid_data(ea, flags)),
 		       "r" (mk_esid_data(ea, entry))
@@ -77,6 +102,10 @@ void slb_flush_and_rebolt(void)
 	if ((ksp_esid_data & ESID_MASK) == PAGE_OFFSET)
 		ksp_esid_data &= ~SLB_ESID_V;
 
+	/* Only third entry (stack) may change here so only resave that */
+	slb_shadow_update(ksp_esid_data,
+			  mk_vsid_data(ksp_esid_data, lflags), 2);
+
 	/* We need to do this all in asm, so we're sure we don't touch
 	 * the stack between the slbia and rebolting it. */
 	asm volatile("isync\n"
@@ -209,9 +238,9 @@ void slb_initialize(void)
 	asm volatile("isync":::"memory");
 	asm volatile("slbmte  %0,%0"::"r" (0) : "memory");
 	asm volatile("isync; slbia; isync":::"memory");
-	create_slbe(PAGE_OFFSET, lflags, 0);
+	create_shadowed_slbe(PAGE_OFFSET, lflags, 0);
 
-	create_slbe(VMALLOC_START, vflags, 1);
+	create_shadowed_slbe(VMALLOC_START, vflags, 1);
 
 	/* We don't bolt the stack for the time being - we're in boot,
 	 * so the stack is in the bolted segment.  By the time it goes
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/lpar.c
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
@@ -254,18 +254,34 @@ out:
 void vpa_init(int cpu)
 {
 	int hwcpu = get_hard_smp_processor_id(cpu);
-	unsigned long vpa = __pa(&lppaca[cpu]);
+	unsigned long addr;
 	long ret;
 
 	if (cpu_has_feature(CPU_FTR_ALTIVEC))
 		lppaca[cpu].vmxregs_in_use = 1;
 
-	ret = register_vpa(hwcpu, vpa);
+	addr = __pa(&lppaca[cpu]);
+	ret = register_vpa(hwcpu, addr);
 
-	if (ret)
+	if (ret) {
 		printk(KERN_ERR "WARNING: vpa_init: VPA registration for "
 				"cpu %d (hw %d) of area %lx returns %ld\n",
-				cpu, hwcpu, vpa, ret);
+				cpu, hwcpu, addr, ret);
+		return;
+	}
+	/*
+	 * PAPR says this feature is SLB-Buffer but firmware never
+	 * reports that.  All SPLPAR support SLB shadow buffer.
+	 */
+	addr = __pa(&slb_shadow[cpu]);
+	if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
+		ret = register_slb_shadow(hwcpu, addr);
+		if (ret)
+			printk(KERN_ERR
+			       "WARNING: vpa_init: SLB shadow buffer "
+			       "registration for cpu %d (hw %d) of area %lx "
+			       "returns %ld\n", cpu, hwcpu, addr, ret);
+	}
 }
 
 long pSeries_lpar_hpte_insert(unsigned long hpte_group,
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/plpar_wrappers.h
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/plpar_wrappers.h
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/plpar_wrappers.h
@@ -40,6 +40,16 @@ static inline long register_vpa(unsigned
 	return vpa_call(0x1, cpu, vpa);
 }
 
+static inline long unregister_slb_shadow(unsigned long cpu, unsigned long vpa)
+{
+	return vpa_call(0x7, cpu, vpa);
+}
+
+static inline long register_slb_shadow(unsigned long cpu, unsigned long vpa)
+{
+	return vpa_call(0x3, cpu, vpa);
+}
+
 extern void vpa_init(int cpu);
 
 static inline long plpar_pte_remove(unsigned long flags, unsigned long ptex,
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/setup.c
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
@@ -234,9 +234,17 @@ static void pseries_kexec_cpu_down_xics(
 {
 	/* Don't risk a hypervisor call if we're crashing */
 	if (firmware_has_feature(FW_FEATURE_SPLPAR) && !crash_shutdown) {
-		unsigned long vpa = __pa(get_lppaca());
+		unsigned long addr;
 
-		if (unregister_vpa(hard_smp_processor_id(), vpa)) {
+		addr = __pa(get_slb_shadow());
+		if (unregister_slb_shadow(hard_smp_processor_id(), addr))
+			printk("SLB shadow buffer deregistration of "
+			       "cpu %u (hw_cpu_id %d) failed\n",
+			       smp_processor_id(),
+			       hard_smp_processor_id());
+
+		addr = __pa(get_lppaca());
+		if (unregister_vpa(hard_smp_processor_id(), addr)) {
 			printk("VPA deregistration of cpu %u (hw_cpu_id %d) "
 					"failed\n", smp_processor_id(),
 					hard_smp_processor_id());
Index: linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
===================================================================
--- linux-2.6-ozlabs.orig/include/asm-powerpc/lppaca.h
+++ linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
@@ -27,7 +27,9 @@
 //
 //
 //----------------------------------------------------------------------------
+#include <linux/cache.h>
 #include <asm/types.h>
+#include <asm/mmu.h>
 
 /* The Hypervisor barfs if the lppaca crosses a page boundary.  A 1k
  * alignment is sufficient to prevent this */
@@ -133,5 +135,22 @@ struct lppaca {
 
 extern struct lppaca lppaca[];
 
+/*
+ * SLB shadow buffer structure as defined in the PAPR.  The save_area
+ * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
+ * ESID is stored in the lower 64bits, then the VSID.
+ */
+struct slb_shadow {
+	u32	persistent;	       /* Number of persistent SLBs  x00-x03 */
+	u32	buffer_length;	       /* Total shadow buffer length x04-x07 */
+	u64	reserved;	       /* Alignment		     x08-x0f */
+	struct	{
+		u64     esid;
+		u64	vsid;
+	} save_area[SLB_NUM_BOLTED];   /* Actual SLB Save buffer     x10-x40 */
+} ____cacheline_aligned;
+
+extern struct slb_shadow slb_shadow[];
+
 #endif /* __KERNEL__ */
 #endif /* _ASM_POWERPC_LPPACA_H */
Index: linux-2.6-ozlabs/include/asm-powerpc/paca.h
===================================================================
--- linux-2.6-ozlabs.orig/include/asm-powerpc/paca.h
+++ linux-2.6-ozlabs/include/asm-powerpc/paca.h
@@ -23,6 +23,7 @@
 register struct paca_struct *local_paca asm("r13");
 #define get_paca()	local_paca
 #define get_lppaca()	(get_paca()->lppaca_ptr)
+#define get_slb_shadow()	(get_paca()->slb_shadow_ptr)
 
 struct task_struct;
 
@@ -98,6 +99,8 @@ struct paca_struct {
 	u64 user_time;			/* accumulated usermode TB ticks */
 	u64 system_time;		/* accumulated system TB ticks */
 	u64 startpurr;			/* PURR/TB value snapshot */
+
+	struct slb_shadow *slb_shadow_ptr;
 };
 
 extern struct paca_struct paca[];

Re: [PATCH] SLB shadow buffer

[Milton Miller]

On Mon Aug  7 2006 01:19:19 AM CDT, Michael Neuling wrote:

> Updated with comments from Milton.

Better

> +#define SHADOW_SLB_BOLTED_STACK_ESID \
> +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1))
> +#define SHADOW_SLB_BOLTED_STACK_VSID \
> +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1) + 8)
> +

Still a bit of magic size, but not likely to change.

Maybe an expicit define for the slot?


> +	/* Update the last bolted SLB */
> +	ld	r9,PACA_SLBSHADOWPTR(r13)
> + 	li	r12,0
> +  	std	r12,SHADOW_SLB_BOLTED_STACK_ESID(r9) /* Clear ESID */
> +	std	r7,SHADOW_SLB_BOLTED_STACK_VSID(r9)  /* Save VSID */
> + 	std	r0,SHADOW_SLB_BOLTED_STACK_ESID(r9)  /* Save ESID */
> +

 Still some leading spaces hiding

>  Index: linux-2.6-ozlabs/include/asm-powerpc/lppaca.h

> @@ -133,5 +135,22 @@ struct > > > lppaca {
>  
>  extern struct lppaca lppaca[];
>  
> +/*
> + * SLB shadow buffer structure as defined in the PAPR.  The save_area
> + * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
> + * ESID is stored in the lower 64bits, then the VSID.
> + */
> +struct slb_shadow {
> +	u32	persistent;		// Number of persistent SLBs	x00-x03
> +	u32	buffer_length;		// Total shadow buffer length	x04-x07
> +	u64	reserved;		// Alignment			x08-x0f
> +	struct	{
> +		u64     esid;
> +		u64	vsid;
> +	} save_area[SLB_NUM_BOLTED];	//				x10-x40
> +} ____cacheline_aligned;
> +

C comments ... or in the separate cleanup patch

> +extern struct slb_shadow slb_shadow[];
> +
>  #endif /* __KERNEL__ */
>  #endif /* _ASM_POWERPC_LPPACA_H */
> Index: linux-2.6-ozlabs/include/asm-powerpc/paca.h
> ===================================================================
> 
>  	/* accumulated system TB ticks */
>  	u64 startpurr;			/* PURR/TB value snapshot */
> +
> +	struct slb_shadow *slb_shadow_ptr;
>  };
 
Do we need the extra line?  (I haven't looked at the result)

milton

Re: [PATCH] SLB shadow buffer

[Michael Neuling]

> > +#define SHADOW_SLB_BOLTED_STACK_ESID \
> > +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1))
> > +#define SHADOW_SLB_BOLTED_STACK_VSID \
> > +		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1) + 8)
> > +
> Still a bit of magic size, but not likely to change.
> 
> Maybe an expicit define for the slot?

Arrh nice, I can do this all in asm-offsets.  Much less magic.

> > +	/* Update the last bolted SLB */
> > +	ld	r9,PACA_SLBSHADOWPTR(r13)
> > + 	li	r12,0
> > +  	std	r12,SHADOW_SLB_BOLTED_STACK_ESID(r9) /* Clear ESID */
> > +	std	r7,SHADOW_SLB_BOLTED_STACK_VSID(r9)  /* Save VSID */
> > + 	std	r0,SHADOW_SLB_BOLTED_STACK_ESID(r9)  /* Save ESID */
> > +
> 
>  Still some leading spaces hiding

Yep... oops.  

> 
> >  Index: linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
> 
> > @@ -133,5 +135,22 @@ struct > > > lppaca {
> >  
> >  extern struct lppaca lppaca[];
> >  
> > +/*
> > + * SLB shadow buffer structure as defined in the PAPR.  The save_area
> > + * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
> > + * ESID is stored in the lower 64bits, then the VSID.
> > + */
> > +struct slb_shadow {
> > +	u32	persistent;		// Number of persistent SLBs	x00-x03
> > +	u32	buffer_length;		// Total shadow buffer length	x04-x07
> > +	u64	reserved;		// Alignment			x08-x0f
> > +	struct	{
> > +		u64     esid;
> > +		u64	vsid;
> > +	} save_area[SLB_NUM_BOLTED];	//				x10-x40
> > +} ____cacheline_aligned;
> > +
> 
> C comments ... or in the separate cleanup patch

I think I'll post a full lppaca cleanup patch in a bit.  

> > +extern struct slb_shadow slb_shadow[];
> > +
> >  #endif /* __KERNEL__ */
> >  #endif /* _ASM_POWERPC_LPPACA_H */
> > Index: linux-2.6-ozlabs/include/asm-powerpc/paca.h
> > ===================================================================
> > 
> >  	/* accumulated system TB ticks */
> >  	u64 startpurr;			/* PURR/TB value snapshot */
> > +
> > +	struct slb_shadow *slb_shadow_ptr;
> >  };
>  
> Do we need the extra line?  (I haven't looked at the result)

They are broken into logical groups, so no.

Updated patch to come.

Michael

[PATCH] SLB shadow buffer

[Michael Neuling]

This adds a shadow buffer for the SLBs and regsiters it with PHYP.
Only the bolted SLB entries (top 3) are shadowed.

Signed-off-by: Michael Neuling <mikey@neuling.org>
---
Updated with comments from Milton.

 arch/powerpc/kernel/asm-offsets.c               |    2 +
 arch/powerpc/kernel/entry_64.S                  |   13 ++++++++
 arch/powerpc/kernel/paca.c                      |   15 +++++++++
 arch/powerpc/mm/slb.c                           |   37 +++++++++++++++++++++---
 arch/powerpc/platforms/pseries/lpar.c           |   24 ++++++++++++---
 arch/powerpc/platforms/pseries/plpar_wrappers.h |   10 ++++++
 arch/powerpc/platforms/pseries/setup.c          |   12 ++++++-
 include/asm-powerpc/lppaca.h                    |   19 ++++++++++++
 include/asm-powerpc/paca.h                      |    3 +
 9 files changed, 124 insertions(+), 11 deletions(-)

Index: linux-2.6-ozlabs/arch/powerpc/kernel/asm-offsets.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/asm-offsets.c
+++ linux-2.6-ozlabs/arch/powerpc/kernel/asm-offsets.c
@@ -135,11 +135,13 @@ int main(void)
 	DEFINE(PACA_STARTPURR, offsetof(struct paca_struct, startpurr));
 	DEFINE(PACA_USER_TIME, offsetof(struct paca_struct, user_time));
 	DEFINE(PACA_SYSTEM_TIME, offsetof(struct paca_struct, system_time));
+	DEFINE(PACA_SLBSHADOWPTR, offsetof(struct paca_struct, slb_shadow_ptr));
 
 	DEFINE(LPPACASRR0, offsetof(struct lppaca, saved_srr0));
 	DEFINE(LPPACASRR1, offsetof(struct lppaca, saved_srr1));
 	DEFINE(LPPACAANYINT, offsetof(struct lppaca, int_dword.any_int));
 	DEFINE(LPPACADECRINT, offsetof(struct lppaca, int_dword.fields.decr_int));
+	DEFINE(SLBSHADOW_SAVEAREA, offsetof(struct slb_shadow, save_area));
 #endif /* CONFIG_PPC64 */
 
 	/* RTAS */
Index: linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/entry_64.S
+++ linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
@@ -323,6 +323,11 @@ _GLOBAL(ret_from_fork)
  * The code which creates the new task context is in 'copy_thread'
  * in arch/powerpc/kernel/process.c 
  */
+#define SHADOW_SLB_BOLTED_STACK_ESID \
+		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1))
+#define SHADOW_SLB_BOLTED_STACK_VSID \
+		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1) + 8)
+
 	.align	7
 _GLOBAL(_switch)
 	mflr	r0
@@ -375,6 +380,14 @@ BEGIN_FTR_SECTION
 	ld	r7,KSP_VSID(r4)	/* Get new stack's VSID */
 	oris	r0,r6,(SLB_ESID_V)@h
 	ori	r0,r0,(SLB_NUM_BOLTED-1)@l
+
+	/* Update the last bolted SLB */
+	ld	r9,PACA_SLBSHADOWPTR(r13)
+ 	li	r12,0
+  	std	r12,SHADOW_SLB_BOLTED_STACK_ESID(r9) /* Clear ESID */
+	std	r7,SHADOW_SLB_BOLTED_STACK_VSID(r9)  /* Save VSID */
+ 	std	r0,SHADOW_SLB_BOLTED_STACK_ESID(r9)  /* Save ESID */
+
 	slbie	r6
 	slbie	r6		/* Workaround POWER5 < DD2.1 issue */
 	slbmte	r7,r0
Index: linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/paca.c
+++ linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
@@ -17,6 +17,7 @@
 #include <asm/lppaca.h>
 #include <asm/iseries/it_lp_reg_save.h>
 #include <asm/paca.h>
+#include <asm/mmu.h>
 
 
 /* This symbol is provided by the linker - let it fill in the paca
@@ -45,6 +46,17 @@ struct lppaca lppaca[] = {
 	},
 };
 
+/*
+ * 3 persistent SLBs are registered here.  The buffer will be zero
+ * initially, hence will all be invaild until we actually write them.
+ */
+struct slb_shadow slb_shadow[] __cacheline_aligned = {
+	[0 ... (NR_CPUS-1)] = {
+		.persistent = SLB_NUM_BOLTED,
+		.buffer_length = sizeof(struct slb_shadow),
+	},
+};
+
 /* The Paca is an array with one entry per processor.  Each contains an
  * lppaca, which contains the information shared between the
  * hypervisor and Linux.
@@ -59,7 +71,8 @@ struct lppaca lppaca[] = {
 	.lock_token = 0x8000,						    \
 	.paca_index = (number),		/* Paca Index */		    \
 	.kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL,		    \
-	.hw_cpu_id = 0xffff,
+	.hw_cpu_id = 0xffff,						    \
+	.slb_shadow_ptr = &slb_shadow[number],
 
 #ifdef CONFIG_PPC_ISERIES
 #define PACA_INIT_ISERIES(number)					    \
Index: linux-2.6-ozlabs/arch/powerpc/mm/slb.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/mm/slb.c
+++ linux-2.6-ozlabs/arch/powerpc/mm/slb.c
@@ -22,6 +22,8 @@
 #include <asm/paca.h>
 #include <asm/cputable.h>
 #include <asm/cacheflush.h>
+#include <asm/smp.h>
+#include <linux/compiler.h>
 
 #ifdef DEBUG
 #define DBG(fmt...) udbg_printf(fmt)
@@ -50,9 +52,32 @@ static inline unsigned long mk_vsid_data
 	return (get_kernel_vsid(ea) << SLB_VSID_SHIFT) | flags;
 }
 
-static inline void create_slbe(unsigned long ea, unsigned long flags,
-			       unsigned long entry)
+static inline void slb_shadow_update(unsigned long esid, unsigned long vsid,
+				     unsigned long entry)
 {
+	/*
+	 * Clear the ESID first so the entry is not valid while we are
+	 * updating it.
+	 */
+	get_slb_shadow()->save_area[entry].esid = 0;
+	barrier();
+	get_slb_shadow()->save_area[entry].vsid = vsid;
+	barrier();
+	get_slb_shadow()->save_area[entry].esid = esid;
+
+}
+
+static inline void create_shadowed_slbe(unsigned long ea, unsigned long flags,
+					unsigned long entry)
+{
+	/*
+	 * Updating the shadow buffer before writing the SLB ensures
+	 * we don't get a stale entry here if we get preempted by PHYP
+	 * between these two statements.
+	 */
+	slb_shadow_update(mk_esid_data(ea, entry), mk_vsid_data(ea, flags),
+			  entry);
+
 	asm volatile("slbmte  %0,%1" :
 		     : "r" (mk_vsid_data(ea, flags)),
 		       "r" (mk_esid_data(ea, entry))
@@ -77,6 +102,10 @@ void slb_flush_and_rebolt(void)
 	if ((ksp_esid_data & ESID_MASK) == PAGE_OFFSET)
 		ksp_esid_data &= ~SLB_ESID_V;
 
+	/* Only third entry (stack) may change here so only resave that */
+	slb_shadow_update(ksp_esid_data,
+			  mk_vsid_data(ksp_esid_data, lflags), 2);
+
 	/* We need to do this all in asm, so we're sure we don't touch
 	 * the stack between the slbia and rebolting it. */
 	asm volatile("isync\n"
@@ -209,9 +238,9 @@ void slb_initialize(void)
 	asm volatile("isync":::"memory");
 	asm volatile("slbmte  %0,%0"::"r" (0) : "memory");
 	asm volatile("isync; slbia; isync":::"memory");
-	create_slbe(PAGE_OFFSET, lflags, 0);
+	create_shadowed_slbe(PAGE_OFFSET, lflags, 0);
 
-	create_slbe(VMALLOC_START, vflags, 1);
+	create_shadowed_slbe(VMALLOC_START, vflags, 1);
 
 	/* We don't bolt the stack for the time being - we're in boot,
 	 * so the stack is in the bolted segment.  By the time it goes
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/lpar.c
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
@@ -254,18 +254,34 @@ out:
 void vpa_init(int cpu)
 {
 	int hwcpu = get_hard_smp_processor_id(cpu);
-	unsigned long vpa = __pa(&lppaca[cpu]);
+	unsigned long addr;
 	long ret;
 
 	if (cpu_has_feature(CPU_FTR_ALTIVEC))
 		lppaca[cpu].vmxregs_in_use = 1;
 
-	ret = register_vpa(hwcpu, vpa);
+	addr = __pa(&lppaca[cpu]);
+	ret = register_vpa(hwcpu, addr);
 
-	if (ret)
+	if (ret) {
 		printk(KERN_ERR "WARNING: vpa_init: VPA registration for "
 				"cpu %d (hw %d) of area %lx returns %ld\n",
-				cpu, hwcpu, vpa, ret);
+				cpu, hwcpu, addr, ret);
+		return;
+	}
+	/*
+	 * PAPR says this feature is SLB-Buffer but firmware never
+	 * reports that.  All SPLPAR support SLB shadow buffer.
+	 */
+	addr = __pa(&slb_shadow[cpu]);
+	if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
+		ret = register_slb_shadow(hwcpu, addr);
+		if (ret)
+			printk(KERN_ERR
+			       "WARNING: vpa_init: SLB shadow buffer "
+			       "registration for cpu %d (hw %d) of area %lx "
+			       "returns %ld\n", cpu, hwcpu, addr, ret);
+	}
 }
 
 long pSeries_lpar_hpte_insert(unsigned long hpte_group,
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/plpar_wrappers.h
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/plpar_wrappers.h
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/plpar_wrappers.h
@@ -40,6 +40,16 @@ static inline long register_vpa(unsigned
 	return vpa_call(0x1, cpu, vpa);
 }
 
+static inline long unregister_slb_shadow(unsigned long cpu, unsigned long vpa)
+{
+	return vpa_call(0x7, cpu, vpa);
+}
+
+static inline long register_slb_shadow(unsigned long cpu, unsigned long vpa)
+{
+	return vpa_call(0x3, cpu, vpa);
+}
+
 extern void vpa_init(int cpu);
 
 static inline long plpar_pte_remove(unsigned long flags, unsigned long ptex,
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/setup.c
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
@@ -234,9 +234,17 @@ static void pseries_kexec_cpu_down_xics(
 {
 	/* Don't risk a hypervisor call if we're crashing */
 	if (firmware_has_feature(FW_FEATURE_SPLPAR) && !crash_shutdown) {
-		unsigned long vpa = __pa(get_lppaca());
+		unsigned long addr;
 
-		if (unregister_vpa(hard_smp_processor_id(), vpa)) {
+		addr = __pa(get_slb_shadow());
+		if (unregister_slb_shadow(hard_smp_processor_id(), addr))
+			printk("SLB shadow buffer deregistration of "
+			       "cpu %u (hw_cpu_id %d) failed\n",
+			       smp_processor_id(),
+			       hard_smp_processor_id());
+
+		addr = __pa(get_lppaca());
+		if (unregister_vpa(hard_smp_processor_id(), addr)) {
 			printk("VPA deregistration of cpu %u (hw_cpu_id %d) "
 					"failed\n", smp_processor_id(),
 					hard_smp_processor_id());
Index: linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
===================================================================
--- linux-2.6-ozlabs.orig/include/asm-powerpc/lppaca.h
+++ linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
@@ -27,7 +27,9 @@
 //
 //
 //----------------------------------------------------------------------------
+#include <linux/cache.h>
 #include <asm/types.h>
+#include <asm/mmu.h>
 
 /* The Hypervisor barfs if the lppaca crosses a page boundary.  A 1k
  * alignment is sufficient to prevent this */
@@ -133,5 +135,22 @@ struct lppaca {
 
 extern struct lppaca lppaca[];
 
+/*
+ * SLB shadow buffer structure as defined in the PAPR.  The save_area
+ * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
+ * ESID is stored in the lower 64bits, then the VSID.
+ */
+struct slb_shadow {
+	u32	persistent;		// Number of persistent SLBs	x00-x03
+	u32	buffer_length;		// Total shadow buffer length	x04-x07
+	u64	reserved;		// Alignment			x08-x0f
+	struct	{
+		u64     esid;
+		u64	vsid;
+	} save_area[SLB_NUM_BOLTED];	//				x10-x40
+} ____cacheline_aligned;
+
+extern struct slb_shadow slb_shadow[];
+
 #endif /* __KERNEL__ */
 #endif /* _ASM_POWERPC_LPPACA_H */
Index: linux-2.6-ozlabs/include/asm-powerpc/paca.h
===================================================================
--- linux-2.6-ozlabs.orig/include/asm-powerpc/paca.h
+++ linux-2.6-ozlabs/include/asm-powerpc/paca.h
@@ -23,6 +23,7 @@
 register struct paca_struct *local_paca asm("r13");
 #define get_paca()	local_paca
 #define get_lppaca()	(get_paca()->lppaca_ptr)
+#define get_slb_shadow()	(get_paca()->slb_shadow_ptr)
 
 struct task_struct;
 
@@ -98,6 +99,8 @@ struct paca_struct {
 	u64 user_time;			/* accumulated usermode TB ticks */
 	u64 system_time;		/* accumulated system TB ticks */
 	u64 startpurr;			/* PURR/TB value snapshot */
+
+	struct slb_shadow *slb_shadow_ptr;
 };
 
 extern struct paca_struct paca[];

Re: [PATCH] SLB shadow buffer

[Milton Miller]

On Sat Aug  5 2006 07:45:44 AM CDT, Christoph Hellwig wrote:
> On Fri, Aug 04, 2006 at 03:53:19PM +1000, Michael Neuling wrote:
> > This adds a shadow buffer for the SLBs and regsiters it with PHYP.
> > Only the bolted SLB entries (first 3) are saved.
> 
> What is a SLB shadow buffer and why do we need it?  (I don't want to
> question the patch but rather learn a little more about lowlevel ppc64
> mmu details)

The SLB is the segment lookaside buffer and holds mappings from the 
effective address space (the address you get adding the register and 
offset) to the virtual address space.  Each entry covers 256MB (2^28) 
and includes an access lookup switch for both problem states (a single 
bit for each user and kernel mode), allowing the same or different 
access permissions.  The virtual address space, 80 bits shared by all 
processes in the system (partition in lpar), is translated to a real address 
and page permissions via the hash table, which is cached in the TLB.  
The original 64 bit archiecture filled the SLB from a segment table 
buffer (STAB), but this was changed to expicit software management in 
version 2, used by POWER4 and later processors.  The 32 bit 
architecture uses 16 fixed segment registers to perfom this mapping to 
a 52 bit virtual space.  

The SLB shadow buffer is a pure software construct recording the required 
segment translations.  Registering a the shadow buffer informs the 
hypervisor which entries are required when switching partitions, and may 
allow it to recover the partition from a hardware error. 

milton

Re: [PATCH] SLB shadow buffer

[Michael Neuling]

> The SLB shadow buffer is a pure software construct recording the required 
> segment translations.  Registering a the shadow buffer informs the 
> hypervisor which entries are required when switching partitions, and may 
> allow it to recover the partition from a hardware error. 

What Milton said.  It's a high availability feature.

We can register volatile and persistent SLBs with PHYP.  PHYP will
invalidate volatile entries and replace persistent entries if it sees a
problem with them.  

Here I've only registered the 3 persistent (bolted) SLBs, as the
volatile SLBs can be handled with SLB faults in the kernel.

Mikey

[PATCH] SLB shadow buffer

[Michael Neuling]

This adds a shadow buffer for the SLBs and regsiters it with PHYP.
Only the bolted SLB entries (first 3) are saved.

Signed-off-by: Michael Neuling <mikey@neuling.org>
---
Thanks to sfr and mpe for their help.

 arch/powerpc/kernel/asm-offsets.c               |    2 +
 arch/powerpc/kernel/entry_64.S                  |   12 ++++++++++
 arch/powerpc/kernel/paca.c                      |   15 ++++++++++++-
 arch/powerpc/mm/slb.c                           |   27 ++++++++++++++++++++++++
 arch/powerpc/platforms/pseries/lpar.c           |   18 ++++++++++++++--
 arch/powerpc/platforms/pseries/plpar_wrappers.h |   10 ++++++++
 arch/powerpc/platforms/pseries/setup.c          |   10 ++++++++
 include/asm-powerpc/lppaca.h                    |   15 +++++++++++++
 include/asm-powerpc/paca.h                      |    4 +++
 9 files changed, 109 insertions(+), 4 deletions(-)

Index: linux-2.6-ozlabs/arch/powerpc/kernel/asm-offsets.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/asm-offsets.c
+++ linux-2.6-ozlabs/arch/powerpc/kernel/asm-offsets.c
@@ -135,11 +135,13 @@ int main(void)
 	DEFINE(PACA_STARTPURR, offsetof(struct paca_struct, startpurr));
 	DEFINE(PACA_USER_TIME, offsetof(struct paca_struct, user_time));
 	DEFINE(PACA_SYSTEM_TIME, offsetof(struct paca_struct, system_time));
+	DEFINE(PACA_SLBSHADOWPTR, offsetof(struct paca_struct, slb_shadow_buffer_ptr));
 
 	DEFINE(LPPACASRR0, offsetof(struct lppaca, saved_srr0));
 	DEFINE(LPPACASRR1, offsetof(struct lppaca, saved_srr1));
 	DEFINE(LPPACAANYINT, offsetof(struct lppaca, int_dword.any_int));
 	DEFINE(LPPACADECRINT, offsetof(struct lppaca, int_dword.fields.decr_int));
+	DEFINE(SLBSHADOW_SAVEAREA, offsetof(struct slb_shadow_buffer, save_area));
 #endif /* CONFIG_PPC64 */
 
 	/* RTAS */
Index: linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/entry_64.S
+++ linux-2.6-ozlabs/arch/powerpc/kernel/entry_64.S
@@ -323,6 +323,10 @@ _GLOBAL(ret_from_fork)
  * The code which creates the new task context is in 'copy_thread'
  * in arch/powerpc/kernel/process.c 
  */
+#define SHADOW_SLB_BOLTED_LAST_ESID \
+		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1))
+#define SHADOW_SLB_BOLTED_LAST_VSID \
+		(SLBSHADOW_SAVEAREA + 0x10*(SLB_NUM_BOLTED-1) + 8)
 	.align	7
 _GLOBAL(_switch)
 	mflr	r0
@@ -375,6 +379,14 @@ BEGIN_FTR_SECTION
 	ld	r7,KSP_VSID(r4)	/* Get new stack's VSID */
 	oris	r0,r6,(SLB_ESID_V)@h
 	ori	r0,r0,(SLB_NUM_BOLTED-1)@l
+
+	/* Update the last bolted SLB */
+	ld	r9,PACA_SLBSHADOWPTR(r13)
+ 	li	r12,0
+  	std     r12,SHADOW_SLB_BOLTED_LAST_ESID(r9) /* Clear ESID */
+	std     r7,SHADOW_SLB_BOLTED_LAST_VSID(r9)  /* Save VSID */
+ 	std     r0,SHADOW_SLB_BOLTED_LAST_ESID(r9)  /* Save ESID */
+
 	slbie	r6
 	slbie	r6		/* Workaround POWER5 < DD2.1 issue */
 	slbmte	r7,r0
Index: linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/kernel/paca.c
+++ linux-2.6-ozlabs/arch/powerpc/kernel/paca.c
@@ -17,6 +17,7 @@
 #include <asm/lppaca.h>
 #include <asm/iseries/it_lp_reg_save.h>
 #include <asm/paca.h>
+#include <asm/mmu.h>
 
 
 /* This symbol is provided by the linker - let it fill in the paca
@@ -45,6 +46,17 @@ struct lppaca lppaca[] = {
 	},
 };
 
+/*
+ * 3 persistent SLBs are registered here.  The buffer will be zero
+ * initially, hence will all be invaild until we actually write them.
+ */
+struct slb_shadow_buffer slb_shadow_buffer[] = {
+	[0 ... (NR_CPUS-1)] = {
+		.persistent = SLB_NUM_BOLTED,
+		.buffer_length = sizeof(struct slb_shadow_buffer),
+	},
+};
+
 /* The Paca is an array with one entry per processor.  Each contains an
  * lppaca, which contains the information shared between the
  * hypervisor and Linux.
@@ -59,7 +71,8 @@ struct lppaca lppaca[] = {
 	.lock_token = 0x8000,						    \
 	.paca_index = (number),		/* Paca Index */		    \
 	.kernel_toc = (unsigned long)(&__toc_start) + 0x8000UL,		    \
-	.hw_cpu_id = 0xffff,
+	.hw_cpu_id = 0xffff,						    \
+	.slb_shadow_buffer_ptr = &slb_shadow_buffer[number]
 
 #ifdef CONFIG_PPC_ISERIES
 #define PACA_INIT_ISERIES(number)					    \
Index: linux-2.6-ozlabs/arch/powerpc/mm/slb.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/mm/slb.c
+++ linux-2.6-ozlabs/arch/powerpc/mm/slb.c
@@ -22,6 +22,8 @@
 #include <asm/paca.h>
 #include <asm/cputable.h>
 #include <asm/cacheflush.h>
+#include <asm/smp.h>
+#include <linux/compiler.h>
 
 #ifdef DEBUG
 #define DBG(fmt...) udbg_printf(fmt)
@@ -50,9 +52,29 @@ static inline unsigned long mk_vsid_data
 	return (get_kernel_vsid(ea) << SLB_VSID_SHIFT) | flags;
 }
 
+static inline void slb_shadow_update(unsigned long esid, unsigned long vsid,
+				     unsigned long entry)
+{
+	/* Clear the ESID first so the entry is not valid while we are
+	 * updating it.  Then write the VSID before the real ESID. */
+	get_slb_shadow_buffer()->save_area[2*entry] = 0;
+	barrier();
+	get_slb_shadow_buffer()->save_area[2*entry+1] =	vsid;
+	barrier();
+	get_slb_shadow_buffer()->save_area[2*entry] = esid;
+
+}
+
 static inline void create_slbe(unsigned long ea, unsigned long flags,
 			       unsigned long entry)
 {
+	/* Updating the shadow buffer before writing the SLB ensures
+	 * we don't get a stale entry here if we get preempted by PHYP
+	 * between these two statements. */
+	slb_shadow_update(mk_esid_data(ea, entry),
+			  mk_vsid_data(ea, flags),
+			  entry);
+
 	asm volatile("slbmte  %0,%1" :
 		     : "r" (mk_vsid_data(ea, flags)),
 		       "r" (mk_esid_data(ea, entry))
@@ -77,6 +99,11 @@ void slb_flush_and_rebolt(void)
 	if ((ksp_esid_data & ESID_MASK) == PAGE_OFFSET)
 		ksp_esid_data &= ~SLB_ESID_V;
 
+	/* Only second entry may change here so only resave that */
+	slb_shadow_update(ksp_esid_data,
+			  mk_vsid_data(ksp_esid_data, lflags),
+			  2);
+
 	/* We need to do this all in asm, so we're sure we don't touch
 	 * the stack between the slbia and rebolting it. */
 	asm volatile("isync\n"
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/lpar.c
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/lpar.c
@@ -254,18 +254,32 @@ out:
 void vpa_init(int cpu)
 {
 	int hwcpu = get_hard_smp_processor_id(cpu);
-	unsigned long vpa = __pa(&lppaca[cpu]);
+	unsigned long vpa;
 	long ret;
 
 	if (cpu_has_feature(CPU_FTR_ALTIVEC))
 		lppaca[cpu].vmxregs_in_use = 1;
 
+	vpa = __pa(&lppaca[cpu]);
 	ret = register_vpa(hwcpu, vpa);
 
-	if (ret)
+	if (ret) {
 		printk(KERN_ERR "WARNING: vpa_init: VPA registration for "
 				"cpu %d (hw %d) of area %lx returns %ld\n",
 				cpu, hwcpu, vpa, ret);
+		return;
+	}
+	/* PAPR says this feature is SLB-Buffer but firmware never
+	 * reports that.  All SPLPAR support SLB shadow buffer. */
+	vpa = __pa(&slb_shadow_buffer[cpu]);
+	if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
+		ret = register_slb_shadow(hwcpu, vpa);
+		if (ret)
+			printk(KERN_ERR
+			       "WARNING: vpa_init: SLB shadow buffer "
+			       "registration for cpu %d (hw %d) of area %lx "
+			       "returns %ld\n", cpu, hwcpu, vpa, ret);
+	}
 }
 
 long pSeries_lpar_hpte_insert(unsigned long hpte_group,
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/plpar_wrappers.h
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/plpar_wrappers.h
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/plpar_wrappers.h
@@ -40,6 +40,16 @@ static inline long register_vpa(unsigned
 	return vpa_call(0x1, cpu, vpa);
 }
 
+static inline long unregister_slb_shadow(unsigned long cpu, unsigned long vpa)
+{
+	return vpa_call(0x7, cpu, vpa);
+}
+
+static inline long register_slb_shadow(unsigned long cpu, unsigned long vpa)
+{
+	return vpa_call(0x3, cpu, vpa);
+}
+
 extern void vpa_init(int cpu);
 
 static inline long plpar_pte_remove(unsigned long flags, unsigned long ptex,
Index: linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
===================================================================
--- linux-2.6-ozlabs.orig/arch/powerpc/platforms/pseries/setup.c
+++ linux-2.6-ozlabs/arch/powerpc/platforms/pseries/setup.c
@@ -234,8 +234,16 @@ static void pseries_kexec_cpu_down_xics(
 {
 	/* Don't risk a hypervisor call if we're crashing */
 	if (firmware_has_feature(FW_FEATURE_SPLPAR) && !crash_shutdown) {
-		unsigned long vpa = __pa(get_lppaca());
+		unsigned long vpa;
 
+		vpa = __pa(get_slb_shadow_buffer());
+		if (unregister_slb_shadow(hard_smp_processor_id(), vpa))
+			printk("SLB shadow buffer deregistration of "
+			       "cpu %u (hw_cpu_id %d) failed\n",
+			       smp_processor_id(),
+			       hard_smp_processor_id());
+
+		vpa = __pa(get_lppaca());
 		if (unregister_vpa(hard_smp_processor_id(), vpa)) {
 			printk("VPA deregistration of cpu %u (hw_cpu_id %d) "
 					"failed\n", smp_processor_id(),
Index: linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
===================================================================
--- linux-2.6-ozlabs.orig/include/asm-powerpc/lppaca.h
+++ linux-2.6-ozlabs/include/asm-powerpc/lppaca.h
@@ -28,6 +28,7 @@
 //
 //----------------------------------------------------------------------------
 #include <asm/types.h>
+#include <asm/mmu.h>
 
 /* The Hypervisor barfs if the lppaca crosses a page boundary.  A 1k
  * alignment is sufficient to prevent this */
@@ -133,5 +134,19 @@ struct lppaca {
 
 extern struct lppaca lppaca[];
 
+/* SLB shadow buffer structure as defined in the PAPR.  The save_area
+ * contains adjacent ESID and VSID pairs for each shadowed SLB.  The
+ * ESID is stored in the lower 64bits, then the VSID.  NOTE: This
+ * structure is 0x40 bytes long (with 3 bolted SLBs), but PHYP
+ * complaints if we're not 0x80 (cache line?) aligned.  */
+struct slb_shadow_buffer {
+	u32	persistent;		// Number of persistent SLBs	x00-x03
+	u32	buffer_length;		// Total shadow buffer length	x04-x07
+	u64	reserved;		// Alignment			x08-x0f
+	u64     save_area[SLB_NUM_BOLTED * 2];	//			x10-x40
+} __attribute__((__aligned__(0x80)));
+
+extern struct slb_shadow_buffer slb_shadow_buffer[];
+
 #endif /* __KERNEL__ */
 #endif /* _ASM_POWERPC_LPPACA_H */
Index: linux-2.6-ozlabs/include/asm-powerpc/paca.h
===================================================================
--- linux-2.6-ozlabs.orig/include/asm-powerpc/paca.h
+++ linux-2.6-ozlabs/include/asm-powerpc/paca.h
@@ -23,6 +23,7 @@
 register struct paca_struct *local_paca asm("r13");
 #define get_paca()	local_paca
 #define get_lppaca()	(get_paca()->lppaca_ptr)
+#define get_slb_shadow_buffer()	(get_paca()->slb_shadow_buffer_ptr)
 
 struct task_struct;
 
@@ -98,6 +99,9 @@ struct paca_struct {
 	u64 user_time;			/* accumulated usermode TB ticks */
 	u64 system_time;		/* accumulated system TB ticks */
 	u64 startpurr;			/* PURR/TB value snapshot */
+
+	/* Pointer to SLB shadow buffer */
+	struct slb_shadow_buffer *slb_shadow_buffer_ptr;
 };
 
 extern struct paca_struct paca[];

Re: [PATCH] SLB shadow buffer

[Christoph Hellwig]

On Fri, Aug 04, 2006 at 03:53:19PM +1000, Michael Neuling wrote:
> This adds a shadow buffer for the SLBs and regsiters it with PHYP.
> Only the bolted SLB entries (first 3) are saved.

What is a SLB shadow buffer and why do we need it?  (I don't want to
question the patch but rather learn a little more about lowlevel ppc64
mmu details)

Re: [PATCH] SLB shadow buffer

[Benjamin Herrenschmidt]

On Sat, 2006-08-05 at 14:45 +0200, Christoph Hellwig wrote:
> On Fri, Aug 04, 2006 at 03:53:19PM +1000, Michael Neuling wrote:
> > This adds a shadow buffer for the SLBs and regsiters it with PHYP.
> > Only the bolted SLB entries (first 3) are saved.
> 
> What is a SLB shadow buffer and why do we need it?  (I don't want to
> question the patch but rather learn a little more about lowlevel ppc64
> mmu details)

Yeah, same question :) I don't have my PAPR at hand and I'm still
travellling but I'm curious about that one... 

I would suspect it allows the HV to save/restore SLBs when sharing a
processor between partitions but then... the kernel requires that anyway
so either the current code is bogus and that's pretty bad (which I don't
think so) or maybe the fact that we didn't register the shadow caused HV
to always save/restore the entire SLB, while by using the shadow, we let
it save/restore only the bolted entries we put in there (and just
re-fault the other ones). If it's not that, then what ? :)

Ben.

Re: [PATCH] SLB shadow buffer

[Segher Boessenkool]

>> What is a SLB shadow buffer and why do we need it?  (I don't want to
>> question the patch but rather learn a little more about lowlevel  
>> ppc64
>> mmu details)
>
> Yeah, same question :) I don't have my PAPR at hand and I'm still
> travellling but I'm curious about that one...
>
> I would suspect it allows the HV to save/restore SLBs when sharing a
> processor between partitions but then... the kernel requires that  
> anyway
> so either the current code is bogus and that's pretty bad (which I  
> don't
> think so) or maybe the fact that we didn't register the shadow  
> caused HV
> to always save/restore the entire SLB, while by using the shadow,  
> we let
> it save/restore only the bolted entries we put in there (and just
> re-fault the other ones). If it's not that, then what ? :)

Yep, that's it (and I do have a PAPR at hand ;-) )

Note that we could also store "volatile" entries in the shadow buffer;
that's supposed to improve performance.


Segher