* Re: [PATCH] powerpc: Fix definition of SIAR register
2016-04-25 8:15 ` Alexander Graf
@ 2016-04-25 9:16 ` Thomas Huth
2016-04-25 9:31 ` Alexander Graf
2016-04-25 11:15 ` Madhavan Srinivasan
2016-05-12 4:57 ` Paul Mackerras
2 siblings, 1 reply; 11+ messages in thread
From: Thomas Huth @ 2016-04-25 9:16 UTC (permalink / raw)
To: Alexander Graf, Madhavan Srinivasan
Cc: Michael Ellerman, Paul Mackerras, linuxppc-dev, kvm-ppc
On 25.04.2016 10:15, Alexander Graf wrote:
>
>
>> Am 25.04.2016 um 10:08 schrieb Madhavan Srinivasan <maddy@linux.vnet.ibm.com>:
>>
>>
>>
>>> On Friday 08 April 2016 09:24 PM, Thomas Huth wrote:
>>> The SIAR register is available twice, one time as SPR 780 (unprivileged,
>>> but read-only), and one time as SPR 796 (privileged, but read and write).
>>> The Linux kernel code currently uses SPR 780 - and while this is OK for
>>> reading, writing to that register of course does not work.
>>> Since the KVM code tries to write to this register, too (see the mtspr
>>> in book3s_hv_rmhandlers.S), the contents of this register sometimes get
>>> lost for the guests, e.g. during migration of a VM.
>>> To fix this issue, simply switch to the other SPR numer 796 instead.
>>
>> IIUC, SIAR and SDAR are updated by hardware when we take
>> a pmu exception with sampling mode enabled (based on instr).
>> And these register contents are mainly for OS consumption.
>> So, we dont need to restore these register values at all,
>> kindly correct me if I missing something here.
>
> What if you migrate between a pmu event firing and the os reading siar? Or what if the host gets pmu events? Or we migrate the guest to a different pcpu? In all those cases we need to ensure the register contents are consistent.
Right. Or a guest could use the SIAR as a temporary scratch register
while not using the performance monitoring stuff. In that case the
contents of the register of course have to be preserved, too.
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> Note: The perf code in core-book3s.c also seems to write to the SIAR
>>> SPR, so that might be affected by this issue, too - but I did
>>> not test the perf code, so I'm not sure about that part.
>
> Please write a small unit test that fires off pmu events constantly and checks whtether they arrive correctly. Run perf in parallel on the host to increase the chance for breakage.
I'm not very familiar with that PMU stuff yet, but I can have a try...
>>> arch/powerpc/include/asm/reg.h | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
>>> index f5f4c66..6630420 100644
>>> --- a/arch/powerpc/include/asm/reg.h
>>> +++ b/arch/powerpc/include/asm/reg.h
>>> @@ -752,13 +752,13 @@
>>> #define SPRN_PMC6 792
>>> #define SPRN_PMC7 793
>>> #define SPRN_PMC8 794
>>> -#define SPRN_SIAR 780
>>> #define SPRN_SDAR 781
>>> #define SPRN_SIER 784
>>> #define SIER_SIPR 0x2000000 /* Sampled MSR_PR */
>>> #define SIER_SIHV 0x1000000 /* Sampled MSR_HV */
>>> #define SIER_SIAR_VALID 0x0400000 /* SIAR contents valid */
>>> #define SIER_SDAR_VALID 0x0200000 /* SDAR contents valid */
>>> +#define SPRN_SIAR 796
>
> I'm sure there's a reason (iSeries?) we used the r/o version before. Better introduce a new constant that gives us rw access and use that in the kvm entry/exit code.
Sure. Any suggestions on the naming? I could either rename the current
SPRN_SIAR to SPRN_USIAR (so that it is named similar to other registers
that behave that way, like SPRN_USPRG3 - and also QEMU uses USIAR for
this already). Or I could leave the old name untouched and use something
like "SPRN_SIAR_WR" for the 796 register. What do you prefer?
By the way, I just noticed that SPRN_SDAR (781) seems to suffer from the
same problem, too!
Thomas
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] powerpc: Fix definition of SIAR register
2016-04-25 9:16 ` Thomas Huth
@ 2016-04-25 9:31 ` Alexander Graf
0 siblings, 0 replies; 11+ messages in thread
From: Alexander Graf @ 2016-04-25 9:31 UTC (permalink / raw)
To: Thomas Huth, Madhavan Srinivasan
Cc: Michael Ellerman, Paul Mackerras, linuxppc-dev, kvm-ppc
On 04/25/2016 11:16 AM, Thomas Huth wrote:
> On 25.04.2016 10:15, Alexander Graf wrote:
>>
>>> Am 25.04.2016 um 10:08 schrieb Madhavan Srinivasan <maddy@linux.vnet.ibm.com>:
>>>
>>>
>>>
>>>> On Friday 08 April 2016 09:24 PM, Thomas Huth wrote:
>>>> The SIAR register is available twice, one time as SPR 780 (unprivileged,
>>>> but read-only), and one time as SPR 796 (privileged, but read and write).
>>>> The Linux kernel code currently uses SPR 780 - and while this is OK for
>>>> reading, writing to that register of course does not work.
>>>> Since the KVM code tries to write to this register, too (see the mtspr
>>>> in book3s_hv_rmhandlers.S), the contents of this register sometimes get
>>>> lost for the guests, e.g. during migration of a VM.
>>>> To fix this issue, simply switch to the other SPR numer 796 instead.
>>> IIUC, SIAR and SDAR are updated by hardware when we take
>>> a pmu exception with sampling mode enabled (based on instr).
>>> And these register contents are mainly for OS consumption.
>>> So, we dont need to restore these register values at all,
>>> kindly correct me if I missing something here.
>> What if you migrate between a pmu event firing and the os reading siar? Or what if the host gets pmu events? Or we migrate the guest to a different pcpu? In all those cases we need to ensure the register contents are consistent.
> Right. Or a guest could use the SIAR as a temporary scratch register
> while not using the performance monitoring stuff. In that case the
> contents of the register of course have to be preserved, too.
>
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>> Note: The perf code in core-book3s.c also seems to write to the SIAR
>>>> SPR, so that might be affected by this issue, too - but I did
>>>> not test the perf code, so I'm not sure about that part.
>> Please write a small unit test that fires off pmu events constantly and checks whtether they arrive correctly. Run perf in parallel on the host to increase the chance for breakage.
> I'm not very familiar with that PMU stuff yet, but I can have a try...
>
>>>> arch/powerpc/include/asm/reg.h | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
>>>> index f5f4c66..6630420 100644
>>>> --- a/arch/powerpc/include/asm/reg.h
>>>> +++ b/arch/powerpc/include/asm/reg.h
>>>> @@ -752,13 +752,13 @@
>>>> #define SPRN_PMC6 792
>>>> #define SPRN_PMC7 793
>>>> #define SPRN_PMC8 794
>>>> -#define SPRN_SIAR 780
>>>> #define SPRN_SDAR 781
>>>> #define SPRN_SIER 784
>>>> #define SIER_SIPR 0x2000000 /* Sampled MSR_PR */
>>>> #define SIER_SIHV 0x1000000 /* Sampled MSR_HV */
>>>> #define SIER_SIAR_VALID 0x0400000 /* SIAR contents valid */
>>>> #define SIER_SDAR_VALID 0x0200000 /* SDAR contents valid */
>>>> +#define SPRN_SIAR 796
>> I'm sure there's a reason (iSeries?) we used the r/o version before. Better introduce a new constant that gives us rw access and use that in the kvm entry/exit code.
> Sure. Any suggestions on the naming? I could either rename the current
> SPRN_SIAR to SPRN_USIAR (so that it is named similar to other registers
> that behave that way, like SPRN_USPRG3 - and also QEMU uses USIAR for
> this already). Or I could leave the old name untouched and use something
> like "SPRN_SIAR_WR" for the 796 register. What do you prefer?
I'd defer that decision to Michael :).
> By the way, I just noticed that SPRN_SDAR (781) seems to suffer from the
> same problem, too!
Great! The more the merrier :)
Alex
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] powerpc: Fix definition of SIAR register
2016-04-25 8:15 ` Alexander Graf
2016-04-25 9:16 ` Thomas Huth
@ 2016-04-25 11:15 ` Madhavan Srinivasan
2016-05-12 4:57 ` Paul Mackerras
2 siblings, 0 replies; 11+ messages in thread
From: Madhavan Srinivasan @ 2016-04-25 11:15 UTC (permalink / raw)
To: Alexander Graf
Cc: Thomas Huth, Michael Ellerman, Paul Mackerras, linuxppc-dev,
kvm-ppc
On Monday 25 April 2016 01:45 PM, Alexander Graf wrote:
>
>> Am 25.04.2016 um 10:08 schrieb Madhavan Srinivasan <maddy@linux.vnet.ibm.com>:
>>
>>
>>
>>> On Friday 08 April 2016 09:24 PM, Thomas Huth wrote:
>>> The SIAR register is available twice, one time as SPR 780 (unprivileged,
>>> but read-only), and one time as SPR 796 (privileged, but read and write).
>>> The Linux kernel code currently uses SPR 780 - and while this is OK for
>>> reading, writing to that register of course does not work.
>>> Since the KVM code tries to write to this register, too (see the mtspr
>>> in book3s_hv_rmhandlers.S), the contents of this register sometimes get
>>> lost for the guests, e.g. during migration of a VM.
>>> To fix this issue, simply switch to the other SPR numer 796 instead.
>> IIUC, SIAR and SDAR are updated by hardware when we take
>> a pmu exception with sampling mode enabled (based on instr).
>> And these register contents are mainly for OS consumption.
>> So, we dont need to restore these register values at all,
>> kindly correct me if I missing something here.
> What if you migrate between a pmu event firing and the os reading siar? Or what if the host gets pmu events? Or we migrate the guest to a different pcpu? In all those cases we need to ensure the register contents are consistent.
Ok got it. Let me try perf record with sample_addr type.
Maddy
>
>> Maddy
>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>> Note: The perf code in core-book3s.c also seems to write to the SIAR
>>> SPR, so that might be affected by this issue, too - but I did
>>> not test the perf code, so I'm not sure about that part.
> Please write a small unit test that fires off pmu events constantly and checks whtether they arrive correctly. Run perf in parallel on the host to increase the chance for breakage.
>
>>> arch/powerpc/include/asm/reg.h | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
>>> index f5f4c66..6630420 100644
>>> --- a/arch/powerpc/include/asm/reg.h
>>> +++ b/arch/powerpc/include/asm/reg.h
>>> @@ -752,13 +752,13 @@
>>> #define SPRN_PMC6 792
>>> #define SPRN_PMC7 793
>>> #define SPRN_PMC8 794
>>> -#define SPRN_SIAR 780
>>> #define SPRN_SDAR 781
>>> #define SPRN_SIER 784
>>> #define SIER_SIPR 0x2000000 /* Sampled MSR_PR */
>>> #define SIER_SIHV 0x1000000 /* Sampled MSR_HV */
>>> #define SIER_SIAR_VALID 0x0400000 /* SIAR contents valid */
>>> #define SIER_SDAR_VALID 0x0200000 /* SDAR contents valid */
>>> +#define SPRN_SIAR 796
> I'm sure there's a reason (iSeries?) we used the r/o version before. Better introduce a new constant that gives us rw access and use that in the kvm entry/exit code.
>
> Alex
>
>>> #define SPRN_TACR 888
>>> #define SPRN_TCSCR 889
>>> #define SPRN_CSIGR 890
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] powerpc: Fix definition of SIAR register
2016-04-25 8:15 ` Alexander Graf
2016-04-25 9:16 ` Thomas Huth
2016-04-25 11:15 ` Madhavan Srinivasan
@ 2016-05-12 4:57 ` Paul Mackerras
2016-05-12 7:27 ` Thomas Huth
2 siblings, 1 reply; 11+ messages in thread
From: Paul Mackerras @ 2016-05-12 4:57 UTC (permalink / raw)
To: Alexander Graf
Cc: Madhavan Srinivasan, Thomas Huth, Michael Ellerman, linuxppc-dev,
kvm-ppc
On Mon, Apr 25, 2016 at 10:15:47AM +0200, Alexander Graf wrote:
>
> >> +#define SPRN_SIAR 796
>
> I'm sure there's a reason (iSeries?) we used the r/o version before. Better introduce a new constant that gives us rw access and use that in the kvm entry/exit code.
I don't think we ever did any performance monitoring on iSeries, and
in any case, we have removed the iSeries code. I think we should just
use the privileged, read/write number (i.e. 796).
Paul.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] powerpc: Fix definition of SIAR register
2016-05-12 4:57 ` Paul Mackerras
@ 2016-05-12 7:27 ` Thomas Huth
2016-05-12 10:40 ` Paul Mackerras
0 siblings, 1 reply; 11+ messages in thread
From: Thomas Huth @ 2016-05-12 7:27 UTC (permalink / raw)
To: Paul Mackerras, Alexander Graf
Cc: Madhavan Srinivasan, Michael Ellerman, linuxppc-dev, kvm-ppc
On 12.05.2016 06:57, Paul Mackerras wrote:
> On Mon, Apr 25, 2016 at 10:15:47AM +0200, Alexander Graf wrote:
>>
>>>> +#define SPRN_SIAR 796
>>
>> I'm sure there's a reason (iSeries?) we used the r/o version before. Better introduce a new constant that gives us rw access and use that in the kvm entry/exit code.
>
> I don't think we ever did any performance monitoring on iSeries, and
> in any case, we have removed the iSeries code. I think we should just
> use the privileged, read/write number (i.e. 796).
Ok, then I'll simply add the fix for SDAR to my patch as well, test it
and submit it again.
Thomas
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH] powerpc: Fix definition of SIAR register
2016-05-12 7:27 ` Thomas Huth
@ 2016-05-12 10:40 ` Paul Mackerras
0 siblings, 0 replies; 11+ messages in thread
From: Paul Mackerras @ 2016-05-12 10:40 UTC (permalink / raw)
To: Thomas Huth
Cc: Alexander Graf, Madhavan Srinivasan, Michael Ellerman,
linuxppc-dev, kvm-ppc
On Thu, May 12, 2016 at 09:27:40AM +0200, Thomas Huth wrote:
> On 12.05.2016 06:57, Paul Mackerras wrote:
> > On Mon, Apr 25, 2016 at 10:15:47AM +0200, Alexander Graf wrote:
> >>
> >>>> +#define SPRN_SIAR 796
> >>
> >> I'm sure there's a reason (iSeries?) we used the r/o version before. Better introduce a new constant that gives us rw access and use that in the kvm entry/exit code.
> >
> > I don't think we ever did any performance monitoring on iSeries, and
> > in any case, we have removed the iSeries code. I think we should just
> > use the privileged, read/write number (i.e. 796).
>
> Ok, then I'll simply add the fix for SDAR to my patch as well, test it
> and submit it again.
Great! Could you switch MMCR2 to the privileged number as well while
you're at it?
Thanks,
Paul.
^ permalink raw reply [flat|nested] 11+ messages in thread