* [PATCH] powerpc: exported functions __clear_user and copy_page use r2 so need _GLOBAL_TOC()
From: Anton Blanchard @ 2014-06-04 22:04 UTC (permalink / raw)
To: benh, paulus, mjw; +Cc: linuxppc-dev
__clear_user and copy_page load from the TOC and are also exported
to modules. This means we have to use _GLOBAL_TOC() so that we
create the global entry point that sets up the TOC.
Signed-off-by: Anton Blanchard <anton@samba.org>
---
Index: b/arch/powerpc/lib/copypage_64.S
===================================================================
--- a/arch/powerpc/lib/copypage_64.S
+++ b/arch/powerpc/lib/copypage_64.S
@@ -16,7 +16,7 @@ PPC64_CACHES:
.tc ppc64_caches[TC],ppc64_caches
.section ".text"
-_GLOBAL(copy_page)
+_GLOBAL_TOC(copy_page)
BEGIN_FTR_SECTION
lis r5,PAGE_SIZE@h
FTR_SECTION_ELSE
Index: b/arch/powerpc/lib/string_64.S
===================================================================
--- a/arch/powerpc/lib/string_64.S
+++ b/arch/powerpc/lib/string_64.S
@@ -77,7 +77,7 @@ err3; stb r0,0(r3)
mr r3,r4
blr
-_GLOBAL(__clear_user)
+_GLOBAL_TOC(__clear_user)
cmpdi r4,32
neg r6,r3
li r0,0
^ permalink raw reply
* Re: [RESEND PATCH] slub: search partial list on numa_mem_id(), instead of numa_node_id()
From: David Rientjes @ 2014-06-05 0:13 UTC (permalink / raw)
To: Andrew Morton
Cc: Joonsoo Kim, Han Pingtian, Nishanth Aravamudan, linux-kernel,
Pekka Enberg, linux-mm, paulus, Anton Blanchard, mpm, Joonsoo Kim,
linuxppc-dev, Christoph Lameter, Wanpeng Li
In-Reply-To: <1391674026-20092-1-git-send-email-iamjoonsoo.kim@lge.com>
On Wed, 21 May 2014, Joonsoo Kim wrote:
> Currently, if allocation constraint to node is NUMA_NO_NODE, we search
> a partial slab on numa_node_id() node. This doesn't work properly on the
> system having memoryless node, since it can have no memory on that node and
> there must be no partial slab on that node.
>
> On that node, page allocation always fallback to numa_mem_id() first. So
> searching a partial slab on numa_node_id() in that case is proper solution
> for memoryless node case.
>
> Acked-by: Nishanth Aravamudan <nacc@linux.vnet.ibm.com>
> Acked-by: David Rientjes <rientjes@google.com>
> Acked-by: Christoph Lameter <cl@linux.com>
> Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> diff --git a/mm/slub.c b/mm/slub.c
> index 545a170..cc1f995 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -1698,7 +1698,7 @@ static void *get_partial(struct kmem_cache *s, gfp_t flags, int node,
> struct kmem_cache_cpu *c)
> {
> void *object;
> - int searchnode = (node == NUMA_NO_NODE) ? numa_node_id() : node;
> + int searchnode = (node == NUMA_NO_NODE) ? numa_mem_id() : node;
>
> object = get_partial_node(s, get_node(s, searchnode), c, flags);
> if (object || node != NUMA_NO_NODE)
Andrew, can you merge this please? It's still not in linux-next.
^ permalink raw reply
* Re: [PATCH 1/1] powerpc: correct DSCR during TM context switch
From: Sam Bobroff @ 2014-06-05 1:26 UTC (permalink / raw)
To: Michael Neuling, Michael Ellerman; +Cc: aik, benh, linuxppc-dev, khandual
In-Reply-To: <1401876215.32587.58.camel@ale.ozlabs.ibm.com>
On 04/06/14 20:03, Michael Neuling wrote:
> On Wed, 2014-06-04 at 17:31 +1000, Michael Ellerman wrote:
>> Hi Sam,
>>
>> Comments inline ..
>
> Ditto....
Responses inline...
>> On Wed, 2014-06-04 at 13:33 +1000, Sam Bobroff wrote:
>>> Correct the DSCR SPR becoming temporarily corrupted when a task is
>>> context switched when within a transaction. It is corrected when
>>> the transaction is aborted (which will happen after a context switch)
>>> but if the task has suspended (TSUSPEND) the transaction the incorrect
>>> value can be seen.
>>
>> I don't quite follow this description. How is it corrected when the transaction
>> is aborted, and when does that usually happen? If that happens the task can't
>> ever see the corrupted value?
>>
>> To hit the suspended case, the task starts a transaction, suspends it, is then
>> context switched out and back in, and at that point it can see the wrong value?
>
> Yep, that's it and it's corrupted until the transaction is rolled back
> (normally at the tresume). At the tresume it gets rolled back to the
> checkpointed value at tbegin and is no longer corrupt.
>
I'll re-work the explanation to be clearer about how it becomes corrupt and how it is corrected.
>>> The problem is caused by saving a thread's DSCR afterNo it's lost at that point as we've not saved it and it was overwritten when we did the treclaim. it has already
>>> been reverted to the CPU's default value:
>>>
>>> __switch_to() calls __switch_to_tm()
>>> which calls tm_reclaim_task()
>>> which calls tm_reclaim_thread()
>>> which calls tm_reclaim() where the DSCR is reset
>>
>> Where the DSCR is set to DSCR_DEFAULT ? Or now PACA_DSCR since your previous
>> patches?
>>
>> Could we instead fix the bug there by reverting to the thread's DSCR value?
>
> We really need to save it earlier, before the treclaim which will
> override it.
I'll try to improve this explanation as well.
>>> __switch_to() calls _switch
>>> _switch() saves the DSCR to thread.dscrTBEGIN
>>>
>>> The fix is to treat the DSCR similarly to the TAR and save it early
>>> in __switch_to().
>>>
>>> The program below will expose the problem:
>>
>>
>> Can you drop this in tools/testing/selftests/powerpc/tm ?
>>
>> You'll need to create that directory, you can ape the Makefile from the pmu
>> directory, it should be fairly obvious. See the pmu tests for how to integrate
>> with the test harness etc., or bug me if it's not straight forward.
Will do :-)
>>> diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
>>> index 2737f46..3efd0e5 100644
>>> --- a/arch/powerpc/include/asm/switch_to.h
>>> +++ b/arch/powerpc/include/asm/switch_to.h
>>> @@ -16,13 +16,15 @@ struct thread_struct;
>>> extern struct task_struct *_switch(struct thread_struct *prev,
>>> struct thread_struct *next);
>>> #ifdef CONFIG_PPC_BOOK3S_64
>>> -static inline void save_tar(struct thread_struct *prev)
>>> +static inline void save_early_sprs(struct thread_struct *prev)
>>> {
>>> if (cpu_has_feature(CPU_FTR_ARCH_207S))
>>> prev->tar = mfspr(SPRN_TAR);
>>> + if (cpu_has_feature(CPU_FTR_DSCR))
>>> + prev->dscr = mfspr(SPRN_DSCR);
>>> }
>>
>> Are we going to end up saving more SPRs in this code? What makes the TAR & DSCR
>> special vs everything else?
>
> There are only a limited set of SPRs that TM checkpoints. The full list
> is CR, LR, CTR, FPSCR, AMR, PPR, VRSAVE, VSCR, DSCR, and TAR.
>
> http://www.scribd.com/doc/142877680/PowerISA-v2-07#outer_page_826
>
> CR, LR, CTR, PPR are handled really early in the exception handler
>
> FPSCR, VSCR are done in the FP/VMX/VSX code.
>
> AMR we don't care about.
>
> That just leaves the DSCR and the TAR for here....
>
> ... and the VRSAVE. Sam: did you have a patch to save that one early
> too? I think we talked about it but forgot, or did we decide that it's
> always broken anyway so we don't care? :-D
I thought we'd decided that VRSAVE was already probably broken ;-)
I haven't tested VRSAVE yet so we don't know if it's actually getting corrupted in this situation (although it seems likely), and from a quick look at the code it's not being treated like DSCR or TAR at the moment so I would need to investigate it separately.
> Mikey
>
>> The nice thing about doing this in asm is it's nop'ed out for cpus that don't
>> have the DSCR. What does the generated code for this look like?
>>
>>> diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
>>> index 662c6dd..a107f4a 100644
>>> --- a/arch/powerpc/kernel/entry_64.S
>>> +++ b/arch/powerpc/kernel/entry_64.S
>>> @@ -432,12 +432,6 @@ BEGIN_FTR_SECTION
>>> std r24,THREAD_VRSAVE(r3)
>>> END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
>>> #endif /* CONFIG_ALTIVEC */
>>> -#ifdef CONFIG_PPC64
>>> -BEGIN_FTR_SECTION
>>> - mfspr r25,SPRN_DSCR
>>> - std r25,THREAD_DSCR(r3)
>>> -END_FTR_SECTION_IFSET(CPU_FTR_DSCR)
>>> -#endif
>>> and. r0,r0,r22
>>> beq+ 1f
>>> andc r22,r22,r0
>>
I wondered a little about this as well. The C code calls this function:
static inline int cpu_has_feature(unsigned long feature)
{
return (CPU_FTRS_ALWAYS & feature) ||
(CPU_FTRS_POSSIBLE
& cur_cpu_spec->cpu_features
& feature);
}
And if I'm extracting it correctly, it ends up like this (17 == SPRN_DSCR):
0x0000000000000bac <__switch_to+108>: rldicl. r8,r9,20,63
0x0000000000000bb0 <__switch_to+112>: beq 0xbc0 <__switch_to+128>
0x0000000000000bb4 <__switch_to+116>: mfspr r9,17
0x0000000000000bb8 <__switch_to+120>: std r9,4280(r31)
0x0000000000000bbc <__switch_to+124>: ld r9,16(r10)
I don't think I can comment on it's performance myself; I'll wait for someone more knowledgeable :-)
Converting this to ASM could be out of scope for this fix, but it doesn't seem like it would be hard. (I assume we'd want to convert both the TAR and DSCR parts.)
>> cheers
Cheers,
Sam.
^ permalink raw reply
* Re: [PATCH] powerpc: Remove platforms/wsp and associated pieces
From: Michael Ellerman @ 2014-06-05 3:04 UTC (permalink / raw)
To: Paul Bolle; +Cc: linuxppc-dev, Jimi Xenidis, Stephen Rothwell
In-Reply-To: <1401696244.14959.6.camel@x220>
On Mon, 2014-06-02 at 10:04 +0200, Paul Bolle wrote:
> On Mon, 2014-06-02 at 11:20 +1000, Michael Ellerman wrote:
> > __attribute__ ((unused))
> >
> > WSP is the last user of CONFIG_PPC_A2, so we remove that as well.
> >
> > Although CONFIG_PPC_ICSWX still exists, it's no longer selectable for
> > any Book3E platform, so we can remove the code in mmu-book3e.h that
> > depended on it.
> >
> > Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> > ---
> > arch/powerpc/Kconfig.debug | 5 -
> > arch/powerpc/configs/chroma_defconfig | 307 ---------
> > arch/powerpc/include/asm/mmu-book3e.h | 4 -
> > arch/powerpc/include/asm/reg_a2.h | 9 -
> > arch/powerpc/include/asm/wsp.h | 14 -
> > arch/powerpc/kernel/Makefile | 1 -
> > arch/powerpc/kernel/cpu_setup_a2.S | 120 ----
> > arch/powerpc/kernel/cputable.c | 38 --
> > arch/powerpc/kernel/exceptions-64e.S | 16 -
> > arch/powerpc/kernel/udbg.c | 2 -
> > arch/powerpc/kernel/udbg_16550.c | 11 -
> > arch/powerpc/platforms/Kconfig | 1 -
> > arch/powerpc/platforms/Kconfig.cputype | 6 +-
> > arch/powerpc/platforms/Makefile | 1 -
> > arch/powerpc/platforms/wsp/Kconfig | 30 -
> > arch/powerpc/platforms/wsp/Makefile | 10 -
> > arch/powerpc/platforms/wsp/chroma.c | 56 --
> > arch/powerpc/platforms/wsp/h8.c | 135 ----
> > arch/powerpc/platforms/wsp/ics.c | 762 ---------------------
> > arch/powerpc/platforms/wsp/ics.h | 25 -
> > arch/powerpc/platforms/wsp/msi.c | 102 ---
> > arch/powerpc/platforms/wsp/msi.h | 19 -
> > arch/powerpc/platforms/wsp/opb_pic.c | 321 ---------
> > arch/powerpc/platforms/wsp/psr2.c | 67 --
> > arch/powerpc/platforms/wsp/scom_smp.c | 434 ------------
>
> This (trivially) conflicts with commit 2751b628c97e ("powerpc: Fix SMP
> issues with ppc64le ABIv2") in next-20140530.
Ah thanks. I'll assume Ben can handle it unless he yells at me.
> For what it's worth, according to my scripts this doesn't introduce
> Kconfig related regressions (eg, you've removed all references to the
> Kconfig symbols that were removed). And, of course, that puzzling check
> for CONFIG_WSP_DD1_WORKAROUND_DD1_TCE_BUGS is now gone.
Thanks for checking.
cheers
^ permalink raw reply
* Re: [PATCH 1/2] powerpc/cpuidle: Only clear LPCR decrementer wakeup bit on fast sleep entry
From: Preeti U Murthy @ 2014-06-05 3:05 UTC (permalink / raw)
To: Michael Neuling; +Cc: linuxppc-dev, Anton Blanchard, Paul Mackerras
In-Reply-To: <1401692223-28337-1-git-send-email-mikey@neuling.org>
On 06/02/2014 12:27 PM, Michael Neuling wrote:
> Currently when entering fastsleep we clear all LPCR PECE bits.
>
> This patch changes it to only clear the decrementer bit (ie. PECE1), which is
> the only bit we really need to clear here. This is needed if we want to set
> other wakeup causes like the PECEDH bit so we can use hypervisor doorbells on
> powernv.
>
> Signed-off-by: Michael Neuling <mikey@neuling.org>
> ---
> drivers/cpuidle/cpuidle-powernv.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/cpuidle/cpuidle-powernv.c b/drivers/cpuidle/cpuidle-powernv.c
> index 719f6fb..7f7798e 100644
> --- a/drivers/cpuidle/cpuidle-powernv.c
> +++ b/drivers/cpuidle/cpuidle-powernv.c
> @@ -73,7 +73,7 @@ static int fastsleep_loop(struct cpuidle_device *dev,
> return index;
>
> new_lpcr = old_lpcr;
> - new_lpcr &= ~(LPCR_MER | LPCR_PECE); /* lpcr[mer] must be 0 */
> + new_lpcr &= ~(LPCR_MER | LPCR_PECE1); /* lpcr[mer] must be 0 */
>
> /* exit powersave upon external interrupt, but not decrementer
> * interrupt.
You might want to remove this comment and instead simply add
/* Do not exit powersave upon decrementer interrupt */
Besides this, the following line which clears the wakeup from external
interrupt bit can be removed as well, since we are not clearing it anyway ?
new_lpcr |= LPCR_PECE0
^^^
Regards
Preeti U Murthy
>
^ permalink raw reply
* Re: [PATCH 2/6] powerpc, powernv, CPU hotplug: Put offline CPUs in Fast-Sleep instead of Nap
From: Paul Mackerras @ 2014-06-05 4:23 UTC (permalink / raw)
To: Preeti U Murthy; +Cc: svaidyan, linuxppc-dev, linux-kernel, srivatsa.bhat
In-Reply-To: <20140528043856.15676.35844.stgit@preeti.in.ibm.com>
On Wed, May 28, 2014 at 10:08:56AM +0530, Preeti U Murthy wrote:
> From: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
>
> The offline cpus are put to fast sleep if the idle state is discovered in the
> device tree. This is to gain maximum powersavings in the offline state.
...
> while (!generic_check_cpu_restart(cpu)) {
> ppc64_runlatch_off();
> - power7_nap();
> +
> + /* If sleep is supported, go to sleep, instead of nap */
> + if (idle_states & IDLE_USE_SLEEP)
> + power7_sleep();
> + else
> + power7_nap();
> +
> ppc64_runlatch_on();
> if (!generic_check_cpu_restart(cpu)) {
> DBG("CPU%d Unexpected exit while offline !\n", cpu);
What is the latency for waking up from fast sleep state? I'm concerned
this will increase the latency for entering KVM guests.
Paul.
^ permalink raw reply
* [PATCH] powerpc/powernv: Pass buffer size to OPAL validate flash call
From: Vasant Hegde @ 2014-06-05 4:43 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Anton Blanchard
We pass actual buffer size to opal_validate_flash() OPAL API call
and in return it contains output buffer size.
Commit cc146d1d (Fix little endian issues) missed to set the size
param before making OPAL call. So firmware image validation fails.
This patch sets size variable before making OPAL call.
Signed-off-by: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
Tested-by: Thomas Falcon <tlfalcon@linux.vnet.ibm.com>
---
arch/powerpc/platforms/powernv/opal-flash.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/opal-flash.c b/arch/powerpc/platforms/powernv/opal-flash.c
index dc487ff..4e541e6 100644
--- a/arch/powerpc/platforms/powernv/opal-flash.c
+++ b/arch/powerpc/platforms/powernv/opal-flash.c
@@ -130,7 +130,8 @@ static inline void opal_flash_validate(void)
{
long ret;
void *buf = validate_flash_data.buf;
- __be32 size, result;
+ __be32 size = cpu_to_be32(validate_flash_data.buf_size);
+ __be32 result;
ret = opal_validate_flash(__pa(buf), &size, &result);
^ permalink raw reply related
* [PATCH 1/1 v2] powerpc: Correct DSCR during TM context switch
From: Sam Bobroff @ 2014-06-05 6:19 UTC (permalink / raw)
To: benh; +Cc: aik, mikey, linuxppc-dev, khandual
In-Reply-To: <f1ddc3bc4b970a480367a45cdca51e879cb14df0.1401852531.git.sam.bobroff@au1.ibm.com>
Correct the DSCR SPR becoming temporarily corrupted if a task is
context switched during a transaction.
The problem occurs while suspending the task and is caused by saving
the DSCR to thread.dscr after it has already been set to the CPU's
default value:
__switch_to() calls __switch_to_tm()
which calls tm_reclaim_task()
which calls tm_reclaim_thread()
which calls tm_reclaim()
where the DSCR is set to the CPU's default
__switch_to() calls _switch()
where thread.dscr is set to the DSCR
When the task is resumed, it's transaction will be doomed (as usual)
and the DSCR SPR will be corrupted, although the checkpointed value
will be correct. Therefore the DSCR will be immediately corrected by
the transaction aborting, unless it has been suspended. In that case
the incorrect value can be seen by the task until it resumes the
transaction.
The fix is to treat the DSCR similarly to the TAR and save it early
in __switch_to().
A program exposing the problem is added to the kernel self tests as:
tools/testing/selftests/powerpc/tm/tm-resched-dscr.
Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
---
Changes:
v2:
* Reworked commit message.
* Adjusted test code and added it to kernel self tests.
---
arch/powerpc/include/asm/switch_to.h | 6 +-
arch/powerpc/kernel/entry_64.S | 6 --
arch/powerpc/kernel/process.c | 8 +-
tools/testing/selftests/powerpc/Makefile | 2 +-
tools/testing/selftests/powerpc/tm/Makefile | 15 ++++
.../testing/selftests/powerpc/tm/tm-resched-dscr.c | 90 ++++++++++++++++++++
6 files changed, 114 insertions(+), 13 deletions(-)
create mode 100644 tools/testing/selftests/powerpc/tm/Makefile
create mode 100644 tools/testing/selftests/powerpc/tm/tm-resched-dscr.c
diff --git a/arch/powerpc/include/asm/switch_to.h b/arch/powerpc/include/asm/switch_to.h
index 2737f46..3efd0e5 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -16,13 +16,15 @@ struct thread_struct;
extern struct task_struct *_switch(struct thread_struct *prev,
struct thread_struct *next);
#ifdef CONFIG_PPC_BOOK3S_64
-static inline void save_tar(struct thread_struct *prev)
+static inline void save_early_sprs(struct thread_struct *prev)
{
if (cpu_has_feature(CPU_FTR_ARCH_207S))
prev->tar = mfspr(SPRN_TAR);
+ if (cpu_has_feature(CPU_FTR_DSCR))
+ prev->dscr = mfspr(SPRN_DSCR);
}
#else
-static inline void save_tar(struct thread_struct *prev) {}
+static inline void save_early_sprs(struct thread_struct *prev) {}
#endif
extern void enable_kernel_fp(void);
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 662c6dd..a107f4a 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -432,12 +432,6 @@ BEGIN_FTR_SECTION
std r24,THREAD_VRSAVE(r3)
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
#endif /* CONFIG_ALTIVEC */
-#ifdef CONFIG_PPC64
-BEGIN_FTR_SECTION
- mfspr r25,SPRN_DSCR
- std r25,THREAD_DSCR(r3)
-END_FTR_SECTION_IFSET(CPU_FTR_DSCR)
-#endif
and. r0,r0,r22
beq+ 1f
andc r22,r22,r0
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index e247898..8d2065e 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -771,15 +771,15 @@ struct task_struct *__switch_to(struct task_struct *prev,
WARN_ON(!irqs_disabled());
- /* Back up the TAR across context switches.
+ /* Back up the TAR and DSCR across context switches.
* Note that the TAR is not available for use in the kernel. (To
* provide this, the TAR should be backed up/restored on exception
* entry/exit instead, and be in pt_regs. FIXME, this should be in
* pt_regs anyway (for debug).)
- * Save the TAR here before we do treclaim/trecheckpoint as these
- * will change the TAR.
+ * Save the TAR and DSCR here before we do treclaim/trecheckpoint as
+ * these will change them.
*/
- save_tar(&prev->thread);
+ save_early_sprs(&prev->thread);
__switch_to_tm(prev);
diff --git a/tools/testing/selftests/powerpc/Makefile b/tools/testing/selftests/powerpc/Makefile
index 316194f..e1544e8 100644
--- a/tools/testing/selftests/powerpc/Makefile
+++ b/tools/testing/selftests/powerpc/Makefile
@@ -13,7 +13,7 @@ CFLAGS := -Wall -O2 -flto -Wall -Werror -DGIT_VERSION='"$(GIT_VERSION)"' -I$(CUR
export CC CFLAGS
-TARGETS = pmu copyloops
+TARGETS = pmu copyloops tm
endif
diff --git a/tools/testing/selftests/powerpc/tm/Makefile b/tools/testing/selftests/powerpc/tm/Makefile
new file mode 100644
index 0000000..51267f4
--- /dev/null
+++ b/tools/testing/selftests/powerpc/tm/Makefile
@@ -0,0 +1,15 @@
+PROGS := tm-resched-dscr
+
+all: $(PROGS)
+
+$(PROGS):
+
+run_tests: all
+ @-for PROG in $(PROGS); do \
+ ./$$PROG; \
+ done;
+
+clean:
+ rm -f $(PROGS) *.o
+
+.PHONY: all run_tests clean
diff --git a/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c b/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c
new file mode 100644
index 0000000..ee98e38
--- /dev/null
+++ b/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c
@@ -0,0 +1,90 @@
+/* Test context switching to see if the DSCR SPR is correctly preserved
+ * when within a transaction.
+ *
+ * Note: We assume that the DSCR has been left at the default value (0)
+ * for all CPUs.
+ *
+ * Method:
+ *
+ * Set a value into the DSCR.
+ *
+ * Start a transaction, and suspend it (*).
+ *
+ * Hard loop checking to see if the transaction has become doomed.
+ *
+ * Now that we *may* have been preempted, record the DSCR and TEXASR SPRS.
+ *
+ * If the abort was because of a context switch, check the DSCR value.
+ * Otherwise, try again.
+ *
+ * (*) If the transaction is not suspended we can't see the problem because
+ * the transaction abort handler will restore the DSCR to it's checkpointed
+ * value before we regain control.
+ */
+
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <asm/tm.h>
+
+#define TBEGIN ".long 0x7C00051D ;"
+#define TEND ".long 0x7C00055D ;"
+#define TCHECK ".long 0x7C00059C ;"
+#define TSUSPEND ".long 0x7C0005DD ;"
+#define TRESUME ".long 0x7C2005DD ;"
+#define SPRN_TEXASR 0x82
+#define SPRN_DSCR 0x03
+
+int main(void) {
+ uint64_t rv, dscr1 = 1, dscr2, texasr;
+
+ printf("Check DSCR TM context switch: ");
+ fflush(stdout);
+ for (;;) {
+ rv = 1;
+ asm __volatile__ (
+ /* set a known value into the DSCR */
+ "ld 3, %[dscr1];"
+ "mtspr %[sprn_dscr], 3;"
+
+ /* start and suspend a transaction */
+ TBEGIN
+ "beq 1f;"
+ TSUSPEND
+
+ /* hard loop until the transaction becomes doomed */
+ "2: ;"
+ TCHECK
+ "bc 4, 0, 2b;"
+
+ /* record DSCR and TEXASR */
+ "mfspr 3, %[sprn_dscr];"
+ "std 3, %[dscr2];"
+ "mfspr 3, %[sprn_texasr];"
+ "std 3, %[texasr];"
+
+ TRESUME
+ TEND
+ "li %[rv], 0;"
+ "1: ;"
+ : [rv]"=r"(rv), [dscr2]"=m"(dscr2), [texasr]"=m"(texasr)
+ : [dscr1]"m"(dscr1)
+ , [sprn_dscr]"i"(SPRN_DSCR), [sprn_texasr]"i"(SPRN_TEXASR)
+ : "memory", "r3"
+ );
+ assert(rv); /* make sure the transaction aborted */
+ if ((texasr >> 56) != TM_CAUSE_RESCHED) {
+ putchar('.');
+ fflush(stdout);
+ continue;
+ }
+ if (dscr2 != dscr1) {
+ printf(" FAIL\n");
+ exit(EXIT_FAILURE);
+ } else {
+ printf(" OK\n");
+ exit(EXIT_SUCCESS);
+ }
+ }
+}
--
1.7.10.4
^ permalink raw reply related
* [PATCH v8 1/3] powerpc/eeh: Avoid event on passed PE
From: Gavin Shan @ 2014-06-05 6:36 UTC (permalink / raw)
To: kvm-ppc, linuxppc-dev; +Cc: aik, agraf, Gavin Shan, alex.williamson, qiudayu
In-Reply-To: <1401950169-30206-1-git-send-email-gwshan@linux.vnet.ibm.com>
We must not handle EEH error on devices which are passed to somebody
else. Instead, we expect that the frozen device owner detects an EEH
error and recovers from it.
This avoids EEH error handling on passed through devices so the device
owner gets a chance to handle them.
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 7 +++++++
arch/powerpc/kernel/eeh.c | 8 ++++++++
arch/powerpc/platforms/powernv/eeh-ioda.c | 3 ++-
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 7782056..653d981 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -25,6 +25,7 @@
#include <linux/list.h>
#include <linux/string.h>
#include <linux/time.h>
+#include <linux/atomic.h>
struct pci_dev;
struct pci_bus;
@@ -84,6 +85,7 @@ struct eeh_pe {
int freeze_count; /* Times of froze up */
struct timeval tstamp; /* Time on first-time freeze */
int false_positives; /* Times of reported #ff's */
+ atomic_t pass_dev_cnt; /* Count of passed through devs */
struct eeh_pe *parent; /* Parent PE */
struct list_head child_list; /* Link PE to the child list */
struct list_head edevs; /* Link list of EEH devices */
@@ -93,6 +95,11 @@ struct eeh_pe {
#define eeh_pe_for_each_dev(pe, edev, tmp) \
list_for_each_entry_safe(edev, tmp, &pe->edevs, list)
+static inline bool eeh_pe_passed(struct eeh_pe *pe)
+{
+ return pe ? !!atomic_read(&pe->pass_dev_cnt) : false;
+}
+
/*
* The struct is used to trace EEH state for the associated
* PCI device node or PCI device. In future, it might
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index 9c6b899..3bc8b12 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -400,6 +400,14 @@ int eeh_dev_check_failure(struct eeh_dev *edev)
if (ret > 0)
return ret;
+ /*
+ * If the PE isn't owned by us, we shouldn't check the
+ * state. Instead, let the owner handle it if the PE has
+ * been frozen.
+ */
+ if (eeh_pe_passed(pe))
+ return 0;
+
/* If we already have a pending isolation event for this
* slot, we know it's bad already, we don't need to check.
* Do this checking under a lock; as multiple PCI devices
diff --git a/arch/powerpc/platforms/powernv/eeh-ioda.c b/arch/powerpc/platforms/powernv/eeh-ioda.c
index cab3e62..79193eb 100644
--- a/arch/powerpc/platforms/powernv/eeh-ioda.c
+++ b/arch/powerpc/platforms/powernv/eeh-ioda.c
@@ -892,7 +892,8 @@ static int ioda_eeh_next_error(struct eeh_pe **pe)
opal_pci_eeh_freeze_clear(phb->opal_id, frozen_pe_no,
OPAL_EEH_ACTION_CLEAR_FREEZE_ALL);
ret = EEH_NEXT_ERR_NONE;
- } else if ((*pe)->state & EEH_PE_ISOLATED) {
+ } else if ((*pe)->state & EEH_PE_ISOLATED ||
+ eeh_pe_passed(*pe)) {
ret = EEH_NEXT_ERR_NONE;
} else {
pr_err("EEH: Frozen PHB#%x-PE#%x (%s) detected\n",
--
1.8.3.2
^ permalink raw reply related
* [PATCH v8 2/3] powerpc/eeh: EEH support for VFIO PCI device
From: Gavin Shan @ 2014-06-05 6:36 UTC (permalink / raw)
To: kvm-ppc, linuxppc-dev; +Cc: aik, agraf, Gavin Shan, alex.williamson, qiudayu
In-Reply-To: <1401950169-30206-1-git-send-email-gwshan@linux.vnet.ibm.com>
The patch exports functions to be used by new VFIO ioctl command,
which will be introduced in subsequent patch, to support EEH
functinality for VFIO PCI devices.
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 14 +++
arch/powerpc/kernel/eeh.c | 268 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 282 insertions(+)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 653d981..5b4cc4e 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -166,6 +166,8 @@ enum {
#define EEH_OPT_ENABLE 1 /* EEH enable */
#define EEH_OPT_THAW_MMIO 2 /* MMIO enable */
#define EEH_OPT_THAW_DMA 3 /* DMA enable */
+#define EEH_OPT_GET_PE_ADDR 0 /* Get PE addr */
+#define EEH_OPT_GET_PE_MODE 1 /* Get PE mode */
#define EEH_STATE_UNAVAILABLE (1 << 0) /* State unavailable */
#define EEH_STATE_NOT_SUPPORT (1 << 1) /* EEH not supported */
#define EEH_STATE_RESET_ACTIVE (1 << 2) /* Active reset */
@@ -173,6 +175,11 @@ enum {
#define EEH_STATE_DMA_ACTIVE (1 << 4) /* Active DMA */
#define EEH_STATE_MMIO_ENABLED (1 << 5) /* MMIO enabled */
#define EEH_STATE_DMA_ENABLED (1 << 6) /* DMA enabled */
+#define EEH_PE_STATE_NORMAL 0 /* Normal state */
+#define EEH_PE_STATE_RESET 1 /* PE reset asserted */
+#define EEH_PE_STATE_STOPPED_IO_DMA 2 /* Frozen PE */
+#define EEH_PE_STATE_STOPPED_DMA 4 /* Stopped DMA, Enabled IO */
+#define EEH_PE_STATE_UNAVAIL 5 /* Unavailable */
#define EEH_RESET_DEACTIVATE 0 /* Deactivate the PE reset */
#define EEH_RESET_HOT 1 /* Hot reset */
#define EEH_RESET_FUNDAMENTAL 3 /* Fundamental reset */
@@ -280,6 +287,13 @@ void eeh_add_device_late(struct pci_dev *);
void eeh_add_device_tree_late(struct pci_bus *);
void eeh_add_sysfs_files(struct pci_bus *);
void eeh_remove_device(struct pci_dev *);
+int eeh_dev_open(struct pci_dev *pdev);
+void eeh_dev_release(struct pci_dev *pdev);
+struct eeh_pe *eeh_iommu_group_to_pe(struct iommu_group *group);
+int eeh_pe_set_option(struct eeh_pe *pe, int option);
+int eeh_pe_get_state(struct eeh_pe *pe);
+int eeh_pe_reset(struct eeh_pe *pe, int option);
+int eeh_pe_configure(struct eeh_pe *pe);
/**
* EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index 3bc8b12..fc90df0 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -40,6 +40,7 @@
#include <asm/eeh.h>
#include <asm/eeh_event.h>
#include <asm/io.h>
+#include <asm/iommu.h>
#include <asm/machdep.h>
#include <asm/ppc-pci.h>
#include <asm/rtas.h>
@@ -108,6 +109,9 @@ struct eeh_ops *eeh_ops = NULL;
/* Lock to avoid races due to multiple reports of an error */
DEFINE_RAW_SPINLOCK(confirm_error_lock);
+/* Lock to protect passed flags */
+static DEFINE_MUTEX(eeh_dev_mutex);
+
/* Buffer for reporting pci register dumps. Its here in BSS, and
* not dynamically alloced, so that it ends up in RMO where RTAS
* can access it.
@@ -1106,6 +1110,270 @@ void eeh_remove_device(struct pci_dev *dev)
edev->mode &= ~EEH_DEV_SYSFS;
}
+/**
+ * eeh_dev_open - Increase count of pass through devices for PE
+ * @pdev: PCI device
+ *
+ * Increase count of passed through devices for the indicated
+ * PE. In the result, the EEH errors detected on the PE won't be
+ * reported. The PE owner will be responsible for detection
+ * and recovery.
+ */
+int eeh_dev_open(struct pci_dev *pdev)
+{
+ struct eeh_dev *edev;
+
+ mutex_lock(&eeh_dev_mutex);
+
+ /* No PCI device ? */
+ if (!pdev)
+ goto out;
+
+ /* No EEH device or PE ? */
+ edev = pci_dev_to_eeh_dev(pdev);
+ if (!edev || !edev->pe)
+ goto out;
+
+ /* Increase PE's pass through count */
+ atomic_inc(&edev->pe->pass_dev_cnt);
+ mutex_unlock(&eeh_dev_mutex);
+
+ return 0;
+out:
+ mutex_unlock(&eeh_dev_mutex);
+ return -ENODEV;
+}
+EXPORT_SYMBOL_GPL(eeh_dev_open);
+
+/**
+ * eeh_dev_release - Decrease count of pass through devices for PE
+ * @pdev: PCI device
+ *
+ * Decrease count of pass through devices for the indicated PE. If
+ * there is no passed through device in PE, the EEH errors detected
+ * on the PE will be reported and handled as usual.
+ */
+void eeh_dev_release(struct pci_dev *pdev)
+{
+ struct eeh_dev *edev;
+
+ mutex_lock(&eeh_dev_mutex);
+
+ /* No PCI device ? */
+ if (!pdev)
+ goto out;
+
+ /* No EEH device ? */
+ edev = pci_dev_to_eeh_dev(pdev);
+ if (!edev || !edev->pe || !eeh_pe_passed(edev->pe))
+ goto out;
+
+ /* Decrease PE's pass through count */
+ atomic_dec(&edev->pe->pass_dev_cnt);
+ WARN_ON(atomic_read(&edev->pe->pass_dev_cnt) < 0);
+out:
+ mutex_unlock(&eeh_dev_mutex);
+}
+EXPORT_SYMBOL(eeh_dev_release);
+
+/**
+ * eeh_iommu_group_to_pe - Convert IOMMU group to EEH PE
+ * @group: IOMMU group
+ *
+ * The routine is called to convert IOMMU group to EEH PE.
+ */
+struct eeh_pe *eeh_iommu_group_to_pe(struct iommu_group *group)
+{
+ struct iommu_table *tbl;
+ struct pci_dev *pdev = NULL;
+ struct eeh_dev *edev;
+ bool found = false;
+
+ /* No IOMMU group ? */
+ if (!group)
+ return NULL;
+
+ /* No PCI device ? */
+ for_each_pci_dev(pdev) {
+ tbl = get_iommu_table_base(&pdev->dev);
+ if (tbl && tbl->it_group == group) {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ return NULL;
+
+ /* No EEH device or PE ? */
+ edev = pci_dev_to_eeh_dev(pdev);
+ if (!edev || !edev->pe)
+ return NULL;
+
+ return edev->pe;
+}
+
+/**
+ * eeh_pe_set_option - Set options for the indicated PE
+ * @pe: EEH PE
+ * @option: requested option
+ *
+ * The routine is called to enable or disable EEH functionality
+ * on the indicated PE, to enable IO or DMA for the frozen PE.
+ */
+int eeh_pe_set_option(struct eeh_pe *pe, int option)
+{
+ int ret = 0;
+
+ /* Invalid PE ? */
+ if (!pe)
+ return -ENODEV;
+
+ /*
+ * EEH functionality could possibly be disabled, just
+ * return error for the case. And the EEH functinality
+ * isn't expected to be disabled on one specific PE.
+ */
+ switch (option) {
+ case EEH_OPT_ENABLE:
+ if (eeh_enabled())
+ break;
+ ret = -EIO;
+ break;
+ case EEH_OPT_DISABLE:
+ break;
+ case EEH_OPT_THAW_MMIO:
+ case EEH_OPT_THAW_DMA:
+ if (!eeh_ops || !eeh_ops->set_option) {
+ ret = -ENOENT;
+ break;
+ }
+
+ ret = eeh_ops->set_option(pe, option);
+ break;
+ default:
+ pr_debug("%s: Option %d out of range (%d, %d)\n",
+ __func__, option, EEH_OPT_DISABLE, EEH_OPT_THAW_DMA);
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(eeh_pe_set_option);
+
+/**
+ * eeh_pe_get_state - Retrieve PE's state
+ * @pe: EEH PE
+ *
+ * Retrieve the PE's state, which includes 3 aspects: enabled
+ * DMA, enabled IO and asserted reset.
+ */
+int eeh_pe_get_state(struct eeh_pe *pe)
+{
+ int result, ret = 0;
+ bool rst_active, dma_en, mmio_en;
+
+ /* Existing PE ? */
+ if (!pe)
+ return -ENODEV;
+
+ if (!eeh_ops || !eeh_ops->get_state)
+ return -ENOENT;
+
+ result = eeh_ops->get_state(pe, NULL);
+ rst_active = !!(result & EEH_STATE_RESET_ACTIVE);
+ dma_en = !!(result & EEH_STATE_DMA_ENABLED);
+ mmio_en = !!(result & EEH_STATE_MMIO_ENABLED);
+
+ if (rst_active)
+ ret = EEH_PE_STATE_RESET;
+ else if (dma_en && mmio_en)
+ ret = EEH_PE_STATE_NORMAL;
+ else if (!dma_en && !mmio_en)
+ ret = EEH_PE_STATE_STOPPED_IO_DMA;
+ else if (!dma_en && mmio_en)
+ ret = EEH_PE_STATE_STOPPED_DMA;
+ else
+ ret = EEH_PE_STATE_UNAVAIL;
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(eeh_pe_get_state);
+
+/**
+ * eeh_pe_reset - Issue PE reset according to specified type
+ * @pe: EEH PE
+ * @option: reset type
+ *
+ * The routine is called to reset the specified PE with the
+ * indicated type, either fundamental reset or hot reset.
+ * PE reset is the most important part for error recovery.
+ */
+int eeh_pe_reset(struct eeh_pe *pe, int option)
+{
+ int ret = 0;
+
+ /* Invalid PE ? */
+ if (!pe)
+ return -ENODEV;
+
+ if (!eeh_ops || !eeh_ops->set_option || !eeh_ops->reset)
+ return -ENOENT;
+
+ switch (option) {
+ case EEH_RESET_DEACTIVATE:
+ ret = eeh_ops->reset(pe, option);
+ if (ret)
+ break;
+
+ /*
+ * The PE is still in frozen state and we need to clear
+ * that. It's good to clear frozen state after deassert
+ * to avoid messy IO access during reset, which might
+ * cause recursive frozen PE.
+ */
+ ret = eeh_ops->set_option(pe, EEH_OPT_THAW_MMIO);
+ if (!ret)
+ ret = eeh_ops->set_option(pe, EEH_OPT_THAW_DMA);
+ if (!ret)
+ eeh_pe_state_clear(pe, EEH_PE_ISOLATED);
+ break;
+ case EEH_RESET_HOT:
+ case EEH_RESET_FUNDAMENTAL:
+ ret = eeh_ops->reset(pe, option);
+ break;
+ default:
+ pr_debug("%s: Unsupported option %d\n",
+ __func__, option);
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(eeh_pe_reset);
+
+/**
+ * eeh_pe_configure - Configure PCI bridges after PE reset
+ * @pe: EEH PE
+ *
+ * The routine is called to restore the PCI config space for
+ * those PCI devices, especially PCI bridges affected by PE
+ * reset issued previously.
+ */
+int eeh_pe_configure(struct eeh_pe *pe)
+{
+ int ret = 0;
+
+ /* Invalid PE ? */
+ if (!pe)
+ return -ENODEV;
+
+ /* Restore config space for the affected devices */
+ eeh_pe_restore_bars(pe);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(eeh_pe_configure);
+
static int proc_eeh_show(struct seq_file *m, void *v)
{
if (!eeh_enabled()) {
--
1.8.3.2
^ permalink raw reply related
* [PATCH v8 0/3] EEH Support for VFIO PCI Device
From: Gavin Shan @ 2014-06-05 6:36 UTC (permalink / raw)
To: kvm-ppc, linuxppc-dev; +Cc: aik, agraf, Gavin Shan, alex.williamson, qiudayu
The series of patches adds support EEH for PCI devices, which are passed
through to PowerKVM based guest via VFIO. The implementation is straightforward
based on the issues or problems we have to resolve to support EEH for PowerKVM
based guest.
- Emulation for EEH RTAS requests. All EEH RTAS requests goes to QEMU firstly.
If QEMU can't handle it, the request will be sent to host via newly introduced
VFIO container IOCTL command (VFIO_EEH_OP) and gets handled in host kernel.
The series of patches requires corresponding QEMU changes.
Change log
==========
v1 -> v2:
* EEH RTAS requests are routed to QEMU, and then possiblly to host kerenl.
The mechanism KVM in-kernel handling is dropped.
* Error injection is reimplemented based syscall, instead of KVM in-kerenl
handling. The logic for error injection token management is moved to
QEMU. The error injection request is routed to QEMU and then possiblly
to host kernel.
v2 -> v3:
* Make the fields in struct eeh_vfio_pci_addr, struct vfio_eeh_info based
on the comments from Alexey.
* Define macros for EEH VFIO operations (Alexey).
* Clear frozen state after successful PE reset.
* Merge original [PATCH 1/2/3] to one.
v3 -> v4:
* Remove the error injection from the patchset. Mike or I will work on that
later.
* Rename CONFIG_VFIO_EEH to VFIO_PCI_EEH.
* Rename the IOCTL command to VFIO_EEH_OP and it's handled by VFIO-PCI device
instead of VFIO container.
* Rename the IOCTL argument structure to "vfio_eeh_op" accordingly. Also, more
fields added to hold return values for RTAS requests.
* The address mapping stuff is totally removed. When opening or releasing VFIO
PCI device, notification sent to EEH to update the flags indicates the device
is passed to guest or not.
* Change pr_warn() to pr_debug() to avoid DOS as pointed by Alex.W
* Argument size check issue pointed by Alex.W.
v4 -> v5:
* Functions for VFIO PCI EEH support are moved to eeh.c and exported from there.
VFIO PCI driver just uses those functions to tackle IOCTL command VFIO_EEH_OP.
All of this is to make the code organized in a good way as suggested by Alex.G.
Another potential benefit is PowerNV/pSeries are sharing "eeh_ops" and same
infrastructure could possiblly work for KVM_PR and KVM_HV mode at the same time.
* Don't clear error injection registers after finishing PE reset as the patchset
is doing nothing related to error injection.
* Amending Documentation/vfio.txt, which was missed in last revision.
* No QEMU changes for this revision. "v4" works well. Also, remove "RFC" from the
subject as the design is basically recognized.
v5 -> v6:
* CONFIG_VFIO_PCI_EEH removed. Instead to use CONFIG_EEH.
* Split one ioctl command to 5.
* In eeh.c, description has been added for those exported functions. Also, the
functions have negative return values for error and information with other values.
All digital numbers have been replaced by macros defined in eeh.h. The comments,
including the function names have been amended not to mention "guest" or "vfio".
* Add one mutex to protect flag in eeh_dev_open()/release().
* More information on how to use those ioctl commands to Documentation/vfio.txt.
v6 -> v7:
* Remove ioctl command VFIO_EEH_PE_GET_ADDR, the PE address will be figured out
in userland (e.g. QEMU) as Alex.G suggested.
* Let sPAPR VFIO container process the ioctl commands as VFIO container is naturally
corresponds to IOMMU group (aka PE on sPAPR platform).
* All VFIO PCI EEH ioctl commands have "argsz+flags" for its companion data struct.
* For VFIO PCI EEH ioctl commands, ioctl() returns negative number to indicate error
or zero for success. Additinal output information is transported by the companion
data struct.
* Explaining PE in Documentation/vfio.txt, typo fixes, more comments suggested by
Alex.G.
* Split/merge patches according to suggestions from Alex.G and Alex.W.
* To have EEH stub in drivers/vfio/pci/, which was suggested by Alex.W.
* Define various EEH options as macros in vfio.h for userland to use.
v7 -> v8:
* Change ioctl commands back to combined one.
* EEH related logic was put into drivers/vfio/vfio_eeh.c, which is only built with
CONFIG_EEH. Otherwise, inline functions defined in include/linux/vfio.h
* Change vfio.txt according to the source code changes.
* Fix various comments from internal reviews by Alexey. Thanks to Alexey.
Gavin Shan (3):
powerpc/eeh: Avoid event on passed PE
powerpc/eeh: EEH support for VFIO PCI device
drivers/vfio: EEH support for VFIO PCI device
Documentation/vfio.txt | 87 +++++++++-
arch/powerpc/include/asm/eeh.h | 21 +++
arch/powerpc/kernel/eeh.c | 276 ++++++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/eeh-ioda.c | 3 +-
drivers/vfio/Makefile | 1 +
drivers/vfio/pci/vfio_pci.c | 20 ++-
drivers/vfio/vfio_iommu_spapr_tce.c | 17 +-
drivers/vfio/vfio_spapr_eeh.c | 89 ++++++++++
include/linux/vfio.h | 23 +++
include/uapi/linux/vfio.h | 35 ++++
10 files changed, 561 insertions(+), 11 deletions(-)
create mode 100644 drivers/vfio/vfio_spapr_eeh.c
--
1.8.3.2
^ permalink raw reply
* [PATCH v8 3/3] drivers/vfio: EEH support for VFIO PCI device
From: Gavin Shan @ 2014-06-05 6:36 UTC (permalink / raw)
To: kvm-ppc, linuxppc-dev; +Cc: aik, agraf, Gavin Shan, alex.williamson, qiudayu
In-Reply-To: <1401950169-30206-1-git-send-email-gwshan@linux.vnet.ibm.com>
The patch adds new IOCTL commands for sPAPR VFIO container device
to support EEH functionality for PCI devices, which have been passed
through from host to somebody else via VFIO.
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
---
Documentation/vfio.txt | 87 ++++++++++++++++++++++++++++++++++--
drivers/vfio/Makefile | 1 +
drivers/vfio/pci/vfio_pci.c | 20 ++++++---
drivers/vfio/vfio_iommu_spapr_tce.c | 17 ++++++-
drivers/vfio/vfio_spapr_eeh.c | 89 +++++++++++++++++++++++++++++++++++++
include/linux/vfio.h | 23 ++++++++++
include/uapi/linux/vfio.h | 35 +++++++++++++++
7 files changed, 262 insertions(+), 10 deletions(-)
create mode 100644 drivers/vfio/vfio_spapr_eeh.c
diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt
index b9ca023..3fa4538 100644
--- a/Documentation/vfio.txt
+++ b/Documentation/vfio.txt
@@ -305,7 +305,15 @@ faster, the map/unmap handling has been implemented in real mode which provides
an excellent performance which has limitations such as inability to do
locked pages accounting in real time.
-So 3 additional ioctls have been added:
+4) According to sPAPR specification, A Partitionable Endpoint (PE) is an I/O
+subtree that can be treated as a unit for the purposes of partitioning and
+error recovery. A PE may be a single or multi-function IOA (IO Adapter), a
+function of a multi-function IOA, or multiple IOAs (possibly including switch
+and bridge structures above the multiple IOAs). PPC64 guests detect PCI errors
+and recover from them via EEH RTAS services, which works on the basis of
+additional ioctl commands.
+
+So 4 additional ioctls have been added:
VFIO_IOMMU_SPAPR_TCE_GET_INFO - returns the size and the start
of the DMA window on the PCI bus.
@@ -316,9 +324,12 @@ So 3 additional ioctls have been added:
VFIO_IOMMU_DISABLE - disables the container.
+ VFIO_EEH_PE_OP - provides an API for EEH setup, error detection and recovery.
The code flow from the example above should be slightly changed:
+ struct vfio_eeh_pe_op pe_op = { .argsz = sizeof(pe_op) };
+
.....
/* Add the group to the container */
ioctl(group, VFIO_GROUP_SET_CONTAINER, &container);
@@ -342,9 +353,79 @@ The code flow from the example above should be slightly changed:
dma_map.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE;
/* Check here is .iova/.size are within DMA window from spapr_iommu_info */
-
ioctl(container, VFIO_IOMMU_MAP_DMA, &dma_map);
- .....
+
+ /* Get a file descriptor for the device */
+ device = ioctl(group, VFIO_GROUP_GET_DEVICE_FD, "0000:06:0d.0");
+
+ ....
+
+ /* Gratuitous device reset and go... */
+ ioctl(device, VFIO_DEVICE_RESET);
+
+ /* Make sure EEH is supported */
+ ioctl(container, VFIO_CHECK_EXTENSION, VFIO_EEH);
+
+ /* Enable the EEH functionality on the device */
+ pe_op.op = VFIO_EEH_PE_ENABLE;
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+
+ /* You're suggested to create additional data struct to represent
+ * PE, and put child devices belonging to same IOMMU group to the
+ * PE instance for later reference.
+ */
+
+ /* Check the PE's state and make sure it's in functional state */
+ pe_op.op = VFIO_EEH_PE_GET_STATE;
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+
+ /* Save device state using pci_save_state().
+ * EEH should be enabled on the specified device.
+ */
+
+ ....
+
+ /* When 0xFF's returned from reading PCI config space or IO BARs
+ * of the PCI device. Check the PE's state to see if that has been
+ * frozen.
+ */
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+
+ /* Waiting for pending PCI transactions to be completed and don't
+ * produce any more PCI traffic from/to the affected PE until
+ * recovery is finished.
+ */
+
+ /* Enable IO for the affected PE and collect logs. Usually, the
+ * standard part of PCI config space, AER registers are dumped
+ * as logs for further analysis.
+ */
+ pe_op.op = VFIO_EEH_PE_UNFREEZE_IO;
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+
+ /*
+ * Issue PE reset: hot or fundamental reset. Usually, hot reset
+ * is enough. However, the firmware of some PCI adapters would
+ * require fundamental reset.
+ */
+ pe_op.op = VFIO_EEH_PE_RESET_HOT;
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+ pe_op.op = VFIO_EEH_PE_RESET_DEACTIVATE;
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+
+ /* Configure the PCI bridges for the affected PE */
+ pe_op.op = VFIO_EEH_PE_CONFIGURE;
+ ioctl(container, VFIO_EEH_PE_OP, &pe_op);
+
+ /* Restored state we saved at initialization time. pci_restore_state()
+ * is good enough as an example.
+ */
+
+ /* Hopefully, error is recovered successfully. Now, you can resume to
+ * start PCI traffic to/from the affected PE.
+ */
+
+ ....
-------------------------------------------------------------------------------
diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile
index 72bfabc..50e30bc 100644
--- a/drivers/vfio/Makefile
+++ b/drivers/vfio/Makefile
@@ -1,4 +1,5 @@
obj-$(CONFIG_VFIO) += vfio.o
obj-$(CONFIG_VFIO_IOMMU_TYPE1) += vfio_iommu_type1.o
obj-$(CONFIG_VFIO_IOMMU_SPAPR_TCE) += vfio_iommu_spapr_tce.o
+obj-$(CONFIG_EEH) += vfio_spapr_eeh.o
obj-$(CONFIG_VFIO_PCI) += pci/
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 7ba0424..26f289d 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -156,8 +156,10 @@ static void vfio_pci_release(void *device_data)
{
struct vfio_pci_device *vdev = device_data;
- if (atomic_dec_and_test(&vdev->refcnt))
+ if (atomic_dec_and_test(&vdev->refcnt)) {
+ vfio_spapr_pci_eeh_release(vdev->pdev);
vfio_pci_disable(vdev);
+ }
module_put(THIS_MODULE);
}
@@ -165,19 +167,25 @@ static void vfio_pci_release(void *device_data)
static int vfio_pci_open(void *device_data)
{
struct vfio_pci_device *vdev = device_data;
+ int ret;
if (!try_module_get(THIS_MODULE))
return -ENODEV;
if (atomic_inc_return(&vdev->refcnt) == 1) {
- int ret = vfio_pci_enable(vdev);
- if (ret) {
- module_put(THIS_MODULE);
- return ret;
- }
+ ret = vfio_pci_enable(vdev);
+ if (ret)
+ goto error;
+
+ ret = vfio_spapr_pci_eeh_open(vdev->pdev);
+ if (ret)
+ goto error;
}
return 0;
+error:
+ module_put(THIS_MODULE);
+ return ret;
}
static int vfio_pci_get_irq_count(struct vfio_pci_device *vdev, int irq_type)
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index a84788b..730b4ef 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -156,7 +156,16 @@ static long tce_iommu_ioctl(void *iommu_data,
switch (cmd) {
case VFIO_CHECK_EXTENSION:
- return (arg == VFIO_SPAPR_TCE_IOMMU) ? 1 : 0;
+ switch (arg) {
+ case VFIO_SPAPR_TCE_IOMMU:
+ ret = 1;
+ break;
+ default:
+ ret = vfio_spapr_iommu_eeh_ioctl(NULL, cmd, arg);
+ break;
+ }
+
+ return (ret < 0) ? 0 : ret;
case VFIO_IOMMU_SPAPR_TCE_GET_INFO: {
struct vfio_iommu_spapr_tce_info info;
@@ -283,6 +292,12 @@ static long tce_iommu_ioctl(void *iommu_data,
tce_iommu_disable(container);
mutex_unlock(&container->lock);
return 0;
+ case VFIO_EEH_PE_OP:
+ if (!container->tbl || !container->tbl->it_group)
+ return -ENODEV;
+
+ return vfio_spapr_iommu_eeh_ioctl(container->tbl->it_group,
+ cmd, arg);
}
return -ENOTTY;
diff --git a/drivers/vfio/vfio_spapr_eeh.c b/drivers/vfio/vfio_spapr_eeh.c
new file mode 100644
index 0000000..691f8d8
--- /dev/null
+++ b/drivers/vfio/vfio_spapr_eeh.c
@@ -0,0 +1,89 @@
+/*
+ * EEH functionality support for VFIO devices. The feature is only
+ * available on sPAPR compatible platforms.
+ *
+ * Copyright Gavin Shan, IBM Corporation 2014.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/iommu.h>
+#include <linux/pci.h>
+#include <linux/uaccess.h>
+#include <linux/vfio.h>
+#include <asm/eeh.h>
+
+/* We might build address mapping here for "fast" path later */
+int vfio_spapr_pci_eeh_open(struct pci_dev *pdev)
+{
+ return eeh_dev_open(pdev);
+}
+
+void vfio_spapr_pci_eeh_release(struct pci_dev *pdev)
+{
+ eeh_dev_release(pdev);
+}
+
+long vfio_spapr_iommu_eeh_ioctl(struct iommu_group *group,
+ unsigned int cmd, unsigned long arg)
+{
+ struct eeh_pe *pe;
+ struct vfio_eeh_pe_op op;
+ unsigned long minsz;
+ long ret = -EINVAL;
+
+ switch (cmd) {
+ case VFIO_CHECK_EXTENSION:
+ if (arg == VFIO_EEH)
+ ret = eeh_enabled() ? 1 : 0;
+ else
+ ret = 0;
+ break;
+ case VFIO_EEH_PE_OP:
+ pe = eeh_iommu_group_to_pe(group);
+ if (!pe)
+ return -ENODEV;
+
+ minsz = offsetofend(struct vfio_eeh_pe_op, op);
+ if (copy_from_user(&op, (void __user *)arg, minsz))
+ return -EFAULT;
+ if (op.argsz < minsz)
+ return -EINVAL;
+
+ switch (op.op) {
+ case VFIO_EEH_PE_DISABLE:
+ ret = eeh_pe_set_option(pe, EEH_OPT_DISABLE);
+ break;
+ case VFIO_EEH_PE_ENABLE:
+ ret = eeh_pe_set_option(pe, EEH_OPT_ENABLE);
+ break;
+ case VFIO_EEH_PE_UNFREEZE_IO:
+ ret = eeh_pe_set_option(pe, EEH_OPT_THAW_MMIO);
+ break;
+ case VFIO_EEH_PE_UNFREEZE_DMA:
+ ret = eeh_pe_set_option(pe, EEH_OPT_THAW_DMA);
+ break;
+ case VFIO_EEH_PE_GET_STATE:
+ ret = eeh_pe_get_state(pe);
+ break;
+ case VFIO_EEH_PE_RESET_DEACTIVATE:
+ ret = eeh_pe_reset(pe, EEH_RESET_DEACTIVATE);
+ break;
+ case VFIO_EEH_PE_RESET_HOT:
+ ret = eeh_pe_reset(pe, EEH_RESET_HOT);
+ break;
+ case VFIO_EEH_PE_RESET_FUNDAMENTAL:
+ ret = eeh_pe_reset(pe, EEH_RESET_FUNDAMENTAL);
+ break;
+ case VFIO_EEH_PE_CONFIGURE:
+ ret = eeh_pe_configure(pe);
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ }
+
+ return ret;
+}
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 81022a52..0d3bb8f 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -99,4 +99,27 @@ extern int vfio_external_user_iommu_id(struct vfio_group *group);
extern long vfio_external_check_extension(struct vfio_group *group,
unsigned long arg);
+#ifdef CONFIG_EEH
+extern int vfio_spapr_pci_eeh_open(struct pci_dev *pdev);
+extern void vfio_spapr_pci_eeh_release(struct pci_dev *pdev);
+extern long vfio_spapr_iommu_eeh_ioctl(struct iommu_group *group,
+ unsigned int cmd,
+ unsigned long arg);
+#else
+static inline int vfio_spapr_pci_eeh_open(struct pci_dev *pdev)
+{
+ return 0;
+}
+
+static inline void vfio_spapr_pci_eeh_release(struct pci_dev *pdev)
+{
+}
+
+static inline long vfio_spapr_iommu_eeh_ioctl(struct iommu_group *group,
+ unsigned int cmd,
+ unsigned long arg)
+{
+ return -ENOTTY;
+}
+#endif /* CONFIG_EEH */
#endif /* VFIO_H */
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index cb9023d..b93c619 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -30,6 +30,9 @@
*/
#define VFIO_DMA_CC_IOMMU 4
+/* Check if EEH is supported */
+#define VFIO_EEH 5
+
/*
* The IOCTL interface is designed for extensibility by embedding the
* structure length (argsz) and flags into structures passed between
@@ -455,6 +458,38 @@ struct vfio_iommu_spapr_tce_info {
#define VFIO_IOMMU_SPAPR_TCE_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
+/*
+ * EEH PE operation struct provides ways to:
+ * - enable/disable EEH functionality;
+ * - unfreeze IO/DMA for frozen PE;
+ * - read PE state;
+ * - reset PE;
+ * - configure PE.
+ */
+struct vfio_eeh_pe_op {
+ __u32 argsz;
+ __u32 flags;
+ __u32 op;
+};
+
+#define VFIO_EEH_PE_DISABLE 0 /* Disable EEH functionality */
+#define VFIO_EEH_PE_ENABLE 1 /* Enable EEH functionality */
+#define VFIO_EEH_PE_UNFREEZE_IO 2 /* Enable IO for frozen PE */
+#define VFIO_EEH_PE_UNFREEZE_DMA 3 /* Enable DMA for frozen PE */
+#define VFIO_EEH_PE_GET_STATE 4 /* PE state retrieval */
+#define VFIO_EEH_PE_RESET_DEACTIVATE 5 /* Deassert PE reset */
+#define VFIO_EEH_PE_RESET_HOT 6 /* Assert hot reset */
+#define VFIO_EEH_PE_RESET_FUNDAMENTAL 7 /* Assert fundamental reset */
+#define VFIO_EEH_PE_CONFIGURE 8 /* PE configuration */
+
+#define VFIO_EEH_PE_STATE_NORMAL 0 /* PE in functional state */
+#define VFIO_EEH_PE_STATE_RESET 1 /* PE reset in progress */
+#define VFIO_EEH_PE_STATE_STOPPED 2 /* Stopped DMA and IO */
+#define VFIO_EEH_PE_STATE_STOPPED_DMA 4 /* Stopped DMA only */
+#define VFIO_EEH_PE_STATE_UNAVAIL 5 /* State unavailable */
+
+#define VFIO_EEH_PE_OP _IO(VFIO_TYPE, VFIO_BASE + 21)
+
/* ***************************************************************** */
#endif /* _UAPIVFIO_H */
--
1.8.3.2
^ permalink raw reply related
* Re: [PATCH v8 2/3] powerpc/eeh: EEH support for VFIO PCI device
From: Benjamin Herrenschmidt @ 2014-06-05 6:50 UTC (permalink / raw)
To: Gavin Shan; +Cc: aik, agraf, kvm-ppc, alex.williamson, qiudayu, linuxppc-dev
In-Reply-To: <1401950169-30206-3-git-send-email-gwshan@linux.vnet.ibm.com>
On Thu, 2014-06-05 at 16:36 +1000, Gavin Shan wrote:
> +#define EEH_OPT_GET_PE_ADDR 0 /* Get PE addr */
> +#define EEH_OPT_GET_PE_MODE 1 /* Get PE mode */
I assume that's just some leftover from the previous patches :-)
Don't respin just yet, let's see what other comments come in.
Cheers,
Ben.
^ permalink raw reply
* [PATCH 2/3] PPC: KVM: Reserve KVM_CAP_SPAPR_TCE_64 capability number
From: Alexey Kardashevskiy @ 2014-06-05 7:25 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Alexander Graf, kvm-ppc, linux-kernel,
Gleb Natapov, Paul Mackerras, Paolo Bonzini
In-Reply-To: <1401953144-19186-1-git-send-email-aik@ozlabs.ru>
This adds a capability number for 64-bit TCE tables support.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
include/uapi/linux/kvm.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 944cd21..e6972bf 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -744,6 +744,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_ENABLE_CAP_VM 98
#define KVM_CAP_S390_IRQCHIP 99
#define KVM_CAP_SPAPR_TCE_VFIO 100
+#define KVM_CAP_SPAPR_TCE_64 101
#ifdef KVM_CAP_IRQ_ROUTING
--
2.0.0
^ permalink raw reply related
* [PATCH 1/3] PPC: KVM: Reserve KVM_CAP_SPAPR_TCE_VFIO capability number
From: Alexey Kardashevskiy @ 2014-06-05 7:25 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Alexander Graf, kvm-ppc, linux-kernel,
Gleb Natapov, Paul Mackerras, Paolo Bonzini
In-Reply-To: <1401953144-19186-1-git-send-email-aik@ozlabs.ru>
This adds a capability number for in-kernel support for VFIO on
SPAPR platform.
The capability will tell the user space whether in-kernel handlers of
H_PUT_TCE can handle VFIO-targeted requests or not. If not, the user space
must not attempt allocating a TCE table in the host kernel via
the KVM_CREATE_SPAPR_TCE KVM ioctl because in that case TCE requests
will not be passed to the user space which is desired action in
the situation like that.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
include/uapi/linux/kvm.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index a8f4ee5..944cd21 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -743,6 +743,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_IOAPIC_POLARITY_IGNORED 97
#define KVM_CAP_ENABLE_CAP_VM 98
#define KVM_CAP_S390_IRQCHIP 99
+#define KVM_CAP_SPAPR_TCE_VFIO 100
#ifdef KVM_CAP_IRQ_ROUTING
--
2.0.0
^ permalink raw reply related
* [PATCH 3/3] PPC: KVM: Add support for 64bit TCE windows
From: Alexey Kardashevskiy @ 2014-06-05 7:25 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Alexander Graf, kvm-ppc, linux-kernel,
Gleb Natapov, Paul Mackerras, Paolo Bonzini
In-Reply-To: <1401953144-19186-1-git-send-email-aik@ozlabs.ru>
The existing KVM_CREATE_SPAPR_TCE only supports 32bit windows which is not
enough for directly mapped windows as the guest can get more than 4GB.
This adds KVM_CREATE_SPAPR_TCE_64 ioctl and advertises it
via KVM_CAP_SPAPR_TCE_64 capability.
Since 64bit windows are to support Dynamic DMA windows (DDW), let's add
@bus_offset and @page_shift which are also required by DDW.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Documentation/virtual/kvm/api.txt | 46 +++++++++++++++++++++++++++++++++++++
arch/powerpc/include/asm/kvm_host.h | 4 +++-
arch/powerpc/include/asm/kvm_ppc.h | 2 +-
arch/powerpc/include/uapi/asm/kvm.h | 9 ++++++++
arch/powerpc/kvm/book3s_64_vio.c | 4 +++-
arch/powerpc/kvm/powerpc.c | 24 ++++++++++++++++++-
include/uapi/linux/kvm.h | 2 ++
7 files changed, 87 insertions(+), 4 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index b4f5365..8a2a2da 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2484,6 +2484,52 @@ calls by the guest for that service will be passed to userspace to be
handled.
+4.87 KVM_CREATE_SPAPR_TCE_64
+
+Capability: KVM_CAP_SPAPR_TCE_64
+Architectures: powerpc
+Type: vm ioctl
+Parameters: struct kvm_create_spapr_tce_64 (in)
+Returns: file descriptor for manipulating the created TCE table
+
+This is an extension for KVM_CAP_SPAPR_TCE which only supports 32bit
+windows.
+
+This creates a virtual TCE (translation control entry) table, which
+is an IOMMU for PAPR-style virtual I/O. It is used to translate
+logical addresses used in virtual I/O into guest physical addresses,
+and provides a scatter/gather capability for PAPR virtual I/O.
+
+/* for KVM_CAP_SPAPR_TCE_64 */
+struct kvm_create_spapr_tce_64 {
+ __u64 liobn;
+ __u64 window_size;
+ __u64 bus_offset;
+ __u32 page_shift;
+ __u32 flags;
+};
+
+The liobn field gives the logical IO bus number for which to create a
+TCE table. The window_size field specifies the size of the DMA window
+which this TCE table will translate - the table will contain one 64
+bit TCE entry for every IOMMU page. The bus_offset field tells where
+this window is mapped on the IO bus. The page_size field tells a size
+of the pages in this window, can be 4K, 64K, 16MB, etc. The flags field
+is not used at the moment but provides the room for extensions.
+
+When the guest issues an H_PUT_TCE/H_PUT_TCE_INDIRECT/H_STUFF_TCE hcall
+on a liobn for which a TCE table has been created using this ioctl(),
+the kernel will handle it in real or virtual mode, updating the TCE table.
+If liobn has not been registered with this ioctl, H_PUT_TCE/etc calls
+will cause a vm exit and must be handled by userspace.
+
+The return value is a file descriptor which can be passed to mmap(2)
+to map the created TCE table into userspace. This lets userspace read
+the entries written by kernel-handled H_PUT_TCE calls, and also lets
+userspace update the TCE table directly which is useful in some
+circumstances.
+
+
5. The kvm_run structure
------------------------
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 1eaea2d..260a810 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -179,7 +179,9 @@ struct kvmppc_spapr_tce_table {
struct list_head list;
struct kvm *kvm;
u64 liobn;
- u32 window_size;
+ u64 window_size;
+ u64 bus_offset;
+ u32 page_shift;
struct page *pages[0];
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 4096f16..b472fd3 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -126,7 +126,7 @@ extern void kvmppc_map_vrma(struct kvm_vcpu *vcpu,
extern int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu);
extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
- struct kvm_create_spapr_tce *args);
+ struct kvm_create_spapr_tce_64 *args);
extern long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
unsigned long ioba, unsigned long tce);
extern long kvmppc_h_get_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
index a6665be..0ada7b4 100644
--- a/arch/powerpc/include/uapi/asm/kvm.h
+++ b/arch/powerpc/include/uapi/asm/kvm.h
@@ -333,6 +333,15 @@ struct kvm_create_spapr_tce {
__u32 window_size;
};
+/* for KVM_CAP_SPAPR_TCE_64 */
+struct kvm_create_spapr_tce_64 {
+ __u64 liobn;
+ __u64 window_size;
+ __u64 bus_offset;
+ __u32 page_shift;
+ __u32 flags;
+};
+
/* for KVM_ALLOCATE_RMA */
struct kvm_allocate_rma {
__u64 rma_size;
diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c
index 54cf9bc..230fa5f 100644
--- a/arch/powerpc/kvm/book3s_64_vio.c
+++ b/arch/powerpc/kvm/book3s_64_vio.c
@@ -98,7 +98,7 @@ static const struct file_operations kvm_spapr_tce_fops = {
};
long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
- struct kvm_create_spapr_tce *args)
+ struct kvm_create_spapr_tce_64 *args)
{
struct kvmppc_spapr_tce_table *stt = NULL;
long npages;
@@ -120,6 +120,8 @@ long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
stt->liobn = args->liobn;
stt->window_size = args->window_size;
+ stt->bus_offset = args->bus_offset;
+ stt->page_shift = args->page_shift;
stt->kvm = kvm;
for (i = 0; i < npages; i++) {
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 3cf541a..3b78b8d 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -33,6 +33,7 @@
#include <asm/tlbflush.h>
#include <asm/cputhreads.h>
#include <asm/irqflags.h>
+#include <asm/iommu.h>
#include "timing.h"
#include "irq.h"
#include "../mm/mmu_decl.h"
@@ -373,6 +374,7 @@ int kvm_dev_ioctl_check_extension(long ext)
#ifdef CONFIG_PPC_BOOK3S_64
case KVM_CAP_SPAPR_TCE:
+ case KVM_CAP_SPAPR_TCE_64:
case KVM_CAP_PPC_ALLOC_HTAB:
case KVM_CAP_PPC_RTAS:
#ifdef CONFIG_KVM_XICS
@@ -1077,13 +1079,33 @@ long kvm_arch_vm_ioctl(struct file *filp,
break;
}
#ifdef CONFIG_PPC_BOOK3S_64
+ case KVM_CREATE_SPAPR_TCE_64: {
+ struct kvm_create_spapr_tce_64 create_tce_64;
+
+ r = -EFAULT;
+ if (copy_from_user(&create_tce_64, argp, sizeof(create_tce_64)))
+ goto out;
+ if (create_tce_64.flags) {
+ r = -EINVAL;
+ goto out;
+ }
+ r = kvm_vm_ioctl_create_spapr_tce(kvm, &create_tce_64);
+ goto out;
+ }
case KVM_CREATE_SPAPR_TCE: {
struct kvm_create_spapr_tce create_tce;
+ struct kvm_create_spapr_tce_64 create_tce_64;
r = -EFAULT;
if (copy_from_user(&create_tce, argp, sizeof(create_tce)))
goto out;
- r = kvm_vm_ioctl_create_spapr_tce(kvm, &create_tce);
+
+ create_tce_64.liobn = create_tce.liobn;
+ create_tce_64.window_size = create_tce.window_size;
+ create_tce_64.bus_offset = 0;
+ create_tce_64.page_shift = IOMMU_PAGE_SHIFT_4K;
+ create_tce_64.flags = 0;
+ r = kvm_vm_ioctl_create_spapr_tce(kvm, &create_tce_64);
goto out;
}
case KVM_PPC_GET_SMMU_INFO: {
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index e6972bf..c435cbb 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1018,6 +1018,8 @@ struct kvm_s390_ucas_mapping {
/* Available with KVM_CAP_PPC_ALLOC_HTAB */
#define KVM_PPC_ALLOCATE_HTAB _IOWR(KVMIO, 0xa7, __u32)
#define KVM_CREATE_SPAPR_TCE _IOW(KVMIO, 0xa8, struct kvm_create_spapr_tce)
+#define KVM_CREATE_SPAPR_TCE_64 _IOW(KVMIO, 0xa8, \
+ struct kvm_create_spapr_tce_64)
/* Available with KVM_CAP_RMA */
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
/* Available with KVM_CAP_PPC_HTAB_FD */
--
2.0.0
^ permalink raw reply related
* [PATCH 0/3] Prepare for in-kernel VFIO DMA operations acceleration
From: Alexey Kardashevskiy @ 2014-06-05 7:25 UTC (permalink / raw)
To: linuxppc-dev
Cc: kvm, Alexey Kardashevskiy, Alexander Graf, kvm-ppc, linux-kernel,
Gleb Natapov, Paul Mackerras, Paolo Bonzini
This reserves 2 capability numbers.
This implements an extended version of KVM_CREATE_SPAPR_TCE_64 ioctl.
Please advise how to proceed with these patches as I suspect that
first two should go via Paolo's tree while the last one via Alex Graf's tree
(correct?).
Thanks!
Alexey Kardashevskiy (3):
PPC: KVM: Reserve KVM_CAP_SPAPR_TCE_VFIO capability number
PPC: KVM: Reserve KVM_CAP_SPAPR_TCE_64 capability number
PPC: KVM: Add support for 64bit TCE windows
Documentation/virtual/kvm/api.txt | 46 +++++++++++++++++++++++++++++++++++++
arch/powerpc/include/asm/kvm_host.h | 4 +++-
arch/powerpc/include/asm/kvm_ppc.h | 2 +-
arch/powerpc/include/uapi/asm/kvm.h | 9 ++++++++
arch/powerpc/kvm/book3s_64_vio.c | 4 +++-
arch/powerpc/kvm/powerpc.c | 24 ++++++++++++++++++-
include/uapi/linux/kvm.h | 4 ++++
7 files changed, 89 insertions(+), 4 deletions(-)
--
2.0.0
^ permalink raw reply
* Re: [PATCH 3/3] PPC: KVM: Add support for 64bit TCE windows
From: Benjamin Herrenschmidt @ 2014-06-05 7:38 UTC (permalink / raw)
To: Alexey Kardashevskiy
Cc: kvm, Gleb Natapov, Alexander Graf, kvm-ppc, linux-kernel,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
In-Reply-To: <1401953144-19186-4-git-send-email-aik@ozlabs.ru>
On Thu, 2014-06-05 at 17:25 +1000, Alexey Kardashevskiy wrote:
> +This creates a virtual TCE (translation control entry) table, which
> +is an IOMMU for PAPR-style virtual I/O. It is used to translate
> +logical addresses used in virtual I/O into guest physical addresses,
> +and provides a scatter/gather capability for PAPR virtual I/O.
> +
> +/* for KVM_CAP_SPAPR_TCE_64 */
> +struct kvm_create_spapr_tce_64 {
> + __u64 liobn;
> + __u64 window_size;
> + __u64 bus_offset;
> + __u32 page_shift;
> + __u32 flags;
> +};
> +
> +The liobn field gives the logical IO bus number for which to create a
> +TCE table. The window_size field specifies the size of the DMA window
> +which this TCE table will translate - the table will contain one 64
> +bit TCE entry for every IOMMU page. The bus_offset field tells where
> +this window is mapped on the IO bus.
Hrm, the bus_offset cannot be set arbitrarily, it has some pretty strong
HW limits depending on the type of bridge & architecture version...
Do you plan to have that knowledge in qemu ? Or do you have some other
mechanism to query it ? (I might be missing a piece of the puzzle here).
Also one thing I've been pondering ...
We'll end up wasting a ton of memory with those TCE tables. If you have
3 PEs mapped into a guest, it will try to create 3 DDW's mapping the
entire guest memory and so 3 TCE tables large enough for that ... and
which will contain exactly the same entries !
We really want to look into extending PAPR to allow the creation of
table "aliases" so that the guest can essentially create one table and
associate it with multiple PEs. We might still decide to do multiple
copies for NUMA reasons but no more than one per node for example... at
least we can have the policy in qemu/kvm.
Also, do you currently require allocating a single physically contiguous
table or do you support TCE trees in your implementation ?
Cheers,
Ben.
^ permalink raw reply
* Re: [PATCH 3/3] PPC: KVM: Add support for 64bit TCE windows
From: Alexey Kardashevskiy @ 2014-06-05 9:26 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: kvm, Gleb Natapov, Alexander Graf, kvm-ppc, linux-kernel,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
In-Reply-To: <1401953908.3247.121.camel@pasglop>
On 06/05/2014 05:38 PM, Benjamin Herrenschmidt wrote:
> On Thu, 2014-06-05 at 17:25 +1000, Alexey Kardashevskiy wrote:
>> +This creates a virtual TCE (translation control entry) table, which
>> +is an IOMMU for PAPR-style virtual I/O. It is used to translate
>> +logical addresses used in virtual I/O into guest physical addresses,
>> +and provides a scatter/gather capability for PAPR virtual I/O.
>> +
>> +/* for KVM_CAP_SPAPR_TCE_64 */
>> +struct kvm_create_spapr_tce_64 {
>> + __u64 liobn;
>> + __u64 window_size;
>> + __u64 bus_offset;
>> + __u32 page_shift;
>> + __u32 flags;
>> +};
>> +
>> +The liobn field gives the logical IO bus number for which to create a
>> +TCE table. The window_size field specifies the size of the DMA window
>> +which this TCE table will translate - the table will contain one 64
>> +bit TCE entry for every IOMMU page. The bus_offset field tells where
>> +this window is mapped on the IO bus.
>
> Hrm, the bus_offset cannot be set arbitrarily, it has some pretty strong
> HW limits depending on the type of bridge & architecture version...
>
> Do you plan to have that knowledge in qemu ? Or do you have some other
> mechanism to query it ? (I might be missing a piece of the puzzle here).
Yes. QEMU will have this knowledge as it has to implement
ibm,create-pe-dma-window and return this address to the guest. There will
be a container API to receive it from powernv code via funky ppc_md callback.
There are 2 steps:
1. query + create window
2. enable in-kernel KVM acceleration for it.
Everything will work without step2 and, frankly speaking, we do not need it
too much for DDW but it does not cost much.
By having bus_offset in ioctl which is only used for step2, I reduce
dependance from powernv.
> Also one thing I've been pondering ...
>
> We'll end up wasting a ton of memory with those TCE tables. If you have
> 3 PEs mapped into a guest, it will try to create 3 DDW's mapping the
> entire guest memory and so 3 TCE tables large enough for that ... and
> which will contain exactly the same entries !
This is in the plan too, do not rush :)
> We really want to look into extending PAPR to allow the creation of
> table "aliases" so that the guest can essentially create one table and
> associate it with multiple PEs. We might still decide to do multiple
> copies for NUMA reasons but no more than one per node for example... at
> least we can have the policy in qemu/kvm.
>
> Also, do you currently require allocating a single physically contiguous
> table or do you support TCE trees in your implementation ?
No trees yet. For 64GB window we need (64<<30)/(16<<20)*8 = 32K TCE table.
Do we really need trees?
--
Alexey
^ permalink raw reply
* Re: [PATCH 3/3] PPC: KVM: Add support for 64bit TCE windows
From: Benjamin Herrenschmidt @ 2014-06-05 10:27 UTC (permalink / raw)
To: Alexey Kardashevskiy
Cc: kvm, Gleb Natapov, Alexander Graf, kvm-ppc, linux-kernel,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
In-Reply-To: <539037DB.5080706@ozlabs.ru>
On Thu, 2014-06-05 at 19:26 +1000, Alexey Kardashevskiy wrote:
>
> No trees yet. For 64GB window we need (64<<30)/(16<<20)*8 = 32K TCE table.
> Do we really need trees?
The above is assuming hugetlbfs backed guests. These are the least of my worry
indeed. But we need to deal with 4k and 64k guests.
Cheers,
Ben
^ permalink raw reply
* Re: [PATCH 3/3] PPC: KVM: Add support for 64bit TCE windows
From: Alexander Graf @ 2014-06-05 11:56 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Alexey Kardashevskiy
Cc: kvm, Gleb Natapov, linux-kernel, kvm-ppc, Paul Mackerras,
Paolo Bonzini, linuxppc-dev
In-Reply-To: <1401964037.3247.129.camel@pasglop>
On 05.06.14 12:27, Benjamin Herrenschmidt wrote:
> On Thu, 2014-06-05 at 19:26 +1000, Alexey Kardashevskiy wrote:
>> No trees yet. For 64GB window we need (64<<30)/(16<<20)*8 = 32K TCE table.
>> Do we really need trees?
> The above is assuming hugetlbfs backed guests. These are the least of my worry
> indeed. But we need to deal with 4k and 64k guests.
What if we ask user space to give us a pointer to user space allocated
memory along with the TCE registration? We would still ask user space to
only use the returned fd for TCE modifications, but would have some
nicely swappable memory we can store the TCE entries in.
In fact, the code as is today can allocate an arbitrary amount of pinned
kernel memory from within user space without any checks.
Alex
^ permalink raw reply
* Re: [PATCH 0/3] Prepare for in-kernel VFIO DMA operations acceleration
From: Alexander Graf @ 2014-06-05 11:57 UTC (permalink / raw)
To: Alexey Kardashevskiy, linuxppc-dev
Cc: kvm, Gleb Natapov, linux-kernel, kvm-ppc, Paul Mackerras,
Paolo Bonzini
In-Reply-To: <1401953144-19186-1-git-send-email-aik@ozlabs.ru>
On 05.06.14 09:25, Alexey Kardashevskiy wrote:
> This reserves 2 capability numbers.
>
> This implements an extended version of KVM_CREATE_SPAPR_TCE_64 ioctl.
>
> Please advise how to proceed with these patches as I suspect that
> first two should go via Paolo's tree while the last one via Alex Graf's tree
> (correct?).
They would just go via my tree, but only be actually allocated (read:
mergable to qemu) when they hit Paolo's tree.
In fact, I don't think it makes sense to split them off at all.
Alex
^ permalink raw reply
* KVM: PPC: BOOK3S: PR: P8 Support
From: Aneesh Kumar K.V @ 2014-06-05 12:08 UTC (permalink / raw)
To: agraf, benh, paulus; +Cc: linuxppc-dev, kvm, kvm-ppc
This patchset adds support for emulating VTB, IC and Doorbell features in P8.
Doorbell support is dummy since we don't support SMT cores with PR-KVM.
-aneesh
^ permalink raw reply
* [PATCH 2/4] KVM: PPC: BOOK3S: PR: Doorbell support
From: Aneesh Kumar K.V @ 2014-06-05 12:08 UTC (permalink / raw)
To: agraf, benh, paulus; +Cc: linuxppc-dev, kvm, kvm-ppc, Aneesh Kumar K.V
In-Reply-To: <1401970085-14493-1-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
We don't have SMT support yet, hence we should not find a doorbell
message generated
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
arch/powerpc/kvm/book3s_emulate.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c
index 1bb16a59dcbc..d6c87d085182 100644
--- a/arch/powerpc/kvm/book3s_emulate.c
+++ b/arch/powerpc/kvm/book3s_emulate.c
@@ -28,7 +28,9 @@
#define OP_19_XOP_RFI 50
#define OP_31_XOP_MFMSR 83
+#define OP_31_XOP_MSGSNDP 142
#define OP_31_XOP_MTMSR 146
+#define OP_31_XOP_MSGCLRP 174
#define OP_31_XOP_MTMSRD 178
#define OP_31_XOP_MTSR 210
#define OP_31_XOP_MTSRIN 242
@@ -303,6 +305,22 @@ int kvmppc_core_emulate_op_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
break;
}
+ case OP_31_XOP_MSGSNDP:
+ {
+ /*
+ * PR KVM still don't support SMT mode. So we should
+ * not see a MSGSNDP/MSGCLRP used with PR KVM
+ */
+ pr_info("KVM: MSGSNDP used in non SMT case\n");
+ emulated = EMULATE_FAIL;
+ break;
+ }
+ case OP_31_XOP_MSGCLRP:
+ {
+ pr_info("KVM: MSGCLRP used in non SMT case\n");
+ emulated = EMULATE_FAIL;
+ break;
+ }
default:
emulated = EMULATE_FAIL;
}
--
1.9.1
^ permalink raw reply related
* [PATCH 1/4] KVM: PPC: BOOK3S: PR: Emulate virtual timebase register
From: Aneesh Kumar K.V @ 2014-06-05 12:08 UTC (permalink / raw)
To: agraf, benh, paulus; +Cc: linuxppc-dev, kvm, kvm-ppc, Aneesh Kumar K.V
In-Reply-To: <1401970085-14493-1-git-send-email-aneesh.kumar@linux.vnet.ibm.com>
virtual time base register is a per VM, per cpu register that needs
to be saved and restored on vm exit and entry. Writing to VTB is not
allowed in the privileged mode.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/include/asm/reg.h | 15 +++++++++++++++
arch/powerpc/include/asm/time.h | 9 +++++++++
arch/powerpc/kvm/book3s.c | 6 ++++++
arch/powerpc/kvm/book3s_emulate.c | 3 +++
arch/powerpc/kvm/book3s_hv.c | 6 ------
arch/powerpc/kvm/book3s_pr.c | 3 ++-
7 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 4a58731a0a72..bd3caeaeebe1 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -505,6 +505,7 @@ struct kvm_vcpu_arch {
#endif
/* Time base value when we entered the guest */
u64 entry_tb;
+ u64 entry_vtb;
u32 tcr;
ulong tsr; /* we need to perform set/clr_bits() which requires ulong */
u32 ivor[64];
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 4852bcf270f3..3e7085d8af90 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -25,6 +25,7 @@
#ifdef CONFIG_8xx
#include <asm/reg_8xx.h>
#endif /* CONFIG_8xx */
+#include <asm/bug.h>
#define MSR_SF_LG 63 /* Enable 64 bit mode */
#define MSR_ISF_LG 61 /* Interrupt 64b mode valid on 630 */
@@ -1193,6 +1194,20 @@
: "r" ((unsigned long)(v)) \
: "memory")
+static inline unsigned long mfvtb (void)
+{
+#ifdef CONFIG_PPC_BOOK3S_64
+ if (cpu_has_feature(CPU_FTR_ARCH_207S))
+ return mfspr(SPRN_VTB);
+#endif
+ /*
+ * The above mfspr will be a no-op on anything before Power8
+ * That can result in random values returned. We need to
+ * capture that.
+ */
+ BUG();
+}
+
#ifdef __powerpc64__
#if defined(CONFIG_PPC_CELL) || defined(CONFIG_PPC_FSL_BOOK3E)
#define mftb() ({unsigned long rval; \
diff --git a/arch/powerpc/include/asm/time.h b/arch/powerpc/include/asm/time.h
index 1d428e6007ca..03cbada59d3a 100644
--- a/arch/powerpc/include/asm/time.h
+++ b/arch/powerpc/include/asm/time.h
@@ -102,6 +102,15 @@ static inline u64 get_rtc(void)
return (u64)hi * 1000000000 + lo;
}
+static inline u64 get_vtb(void)
+{
+#ifdef CONFIG_PPC_BOOK3S_64
+ if (cpu_has_feature(CPU_FTR_ARCH_207S))
+ return mfvtb();
+#endif
+ return 0;
+}
+
#ifdef CONFIG_PPC64
static inline u64 get_tb(void)
{
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index 52c654dbd41a..ae43e4178ecd 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -646,6 +646,9 @@ int kvm_vcpu_ioctl_get_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
case KVM_REG_PPC_BESCR:
val = get_reg_val(reg->id, vcpu->arch.bescr);
break;
+ case KVM_REG_PPC_VTB:
+ val = get_reg_val(reg->id, vcpu->arch.vtb);
+ break;
default:
r = -EINVAL;
break;
@@ -750,6 +753,9 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
case KVM_REG_PPC_BESCR:
vcpu->arch.bescr = set_reg_val(reg->id, val);
break;
+ case KVM_REG_PPC_VTB:
+ vcpu->arch.vtb = set_reg_val(reg->id, val);
+ break;
default:
r = -EINVAL;
break;
diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c
index 3565e775b61b..1bb16a59dcbc 100644
--- a/arch/powerpc/kvm/book3s_emulate.c
+++ b/arch/powerpc/kvm/book3s_emulate.c
@@ -577,6 +577,9 @@ int kvmppc_core_emulate_mfspr_pr(struct kvm_vcpu *vcpu, int sprn, ulong *spr_val
*/
*spr_val = vcpu->arch.spurr;
break;
+ case SPRN_VTB:
+ *spr_val = vcpu->arch.vtb;
+ break;
case SPRN_GQR0:
case SPRN_GQR1:
case SPRN_GQR2:
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index aba05bbb3e74..f6ac58336b3f 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -897,9 +897,6 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
case KVM_REG_PPC_IC:
*val = get_reg_val(id, vcpu->arch.ic);
break;
- case KVM_REG_PPC_VTB:
- *val = get_reg_val(id, vcpu->arch.vtb);
- break;
case KVM_REG_PPC_CSIGR:
*val = get_reg_val(id, vcpu->arch.csigr);
break;
@@ -1097,9 +1094,6 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
case KVM_REG_PPC_IC:
vcpu->arch.ic = set_reg_val(id, *val);
break;
- case KVM_REG_PPC_VTB:
- vcpu->arch.vtb = set_reg_val(id, *val);
- break;
case KVM_REG_PPC_CSIGR:
vcpu->arch.csigr = set_reg_val(id, *val);
break;
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index ff15f8e31a27..96cdf89a8c86 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -125,6 +125,7 @@ void kvmppc_copy_to_svcpu(struct kvmppc_book3s_shadow_vcpu *svcpu,
* to find the guest purr and spurr value.
*/
vcpu->arch.entry_tb = get_tb();
+ vcpu->arch.entry_vtb = get_vtb();
svcpu->in_use = true;
}
@@ -176,7 +177,7 @@ void kvmppc_copy_from_svcpu(struct kvm_vcpu *vcpu,
*/
vcpu->arch.purr += get_tb() - vcpu->arch.entry_tb;
vcpu->arch.spurr += get_tb() - vcpu->arch.entry_tb;
-
+ vcpu->arch.vtb += get_vtb() - vcpu->arch.entry_vtb;
svcpu->in_use = false;
out:
--
1.9.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox