* [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events
@ 2024-12-22 14:02 Vaibhav Jain
2024-12-22 14:02 ` [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters Vaibhav Jain
` (5 more replies)
0 siblings, 6 replies; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
This patch-series adds support for reporting Hostwide(L1-Lpar) counters via
perf-events. With the support for running KVM Guest in a PSeries-Lpar using
nested-APIv2 via [1], the underlying L0-PowerVM hypervisor holds some state
information pertaining to all running L2-KVM Guests in an L1-Lpar. This
state information is held in a pre-allocated memory thats owned by
L0-PowerVM and is termed as Guest-Management-Area(GMA). The GMA is
allocated per L1-LPAR and is only allocated if the lpar is KVM enabled. The
size of this area is a fixed percentage of the memory assigned to the KVM
enabled L1-lpar and is composed of two major components, Guest Management
Space(Host-Heap) and Guest Page Table Management Space(Host-Pagetable).
The Host-Heap holds the various data-structures allocated by L0-PowerVM for
L2-KVM Guests running in the L1-Lpar. The Host-Pagetable holds the Radix
pagetable[2] for the L2-KVM Guest which is used by L0-PowerVM to handle
page faults. Since the size of both of these areas is limited and fixed via
partition boot profile, it puts an upper bound on the number of L2-KVM
Guests that can be run in an LPAR. Also due limited size of Host-Pagetable
area, L0-PowerVM is at times forced to perform reclaim operation on
it. This reclaim operation is usually performed when running large number
of L2-KVM Guests which are memory bound and increases Host-Pagetable
utilization.
In light of the above its recommended to track usage of these areas to
ensure consistent L2-KVM Guest performance. Hence this patch-series
attempts to expose the max-size and current-usage of these areas as well as
cumulative amount of bytes reclaimed from Host-Pagetable as perf-events
that can be queried via perf-stat.
The patch series introduces a new 'kvm-hv' PMU which exports the
perf-events mentioned below. Since perf-events exported represents the
state of the whole L1-Lpar and not that of a specific L2-KVM guest hence
the 'kvm-hv' PMU's scope is set as PERF_PMU_SCOPE_SYS_WIDE(System-Wide).
New perf-events introduced
==========================
* kvm-hv/host_heap/ : The currently used bytes in the
Hypervisor's Guest Management Space
associated with the Host Partition.
* kvm-hv/host_heap_max/ : The maximum bytes available in the
Hypervisor's Guest Management Space
associated with the Host Partition.
* kvm-hv/host_pagetable/ : The currently used bytes in the
Hypervisor's Guest Page Table Management
Space associated with the Host Partition.
* kvm-hv/host_pagetable_max/ : The maximum bytes available in the
Hypervisor's Guest Page Table Management
Space associated with the Host Partition.
* kvm-hv/host_pagetable_reclaim/: The amount of space in bytes that has
been reclaimed due to overcommit in the
Hypervisor's Guest Page Table Management
Space associated with the Host Partition.
Structure of this patch series
==============================
Start with documenting and updating the KVM nested-APIv2 hcall
specifications for H_GUEST_GET_STATE hcall and Hostwide guest-state-buffer
elements.
Subsequent patches add support for adding and parsing Hostwide
guest-state-buffer elements in existing kvm-hv apiv2 infrastructure. Also
add a kunit test case to verify correctness of the changes introduced.
Next set of patches in the patch-set introduces a new PMU for kvm-hv on
pseries named as 'kvm-hv', implement plumbing between kvm-hv module and
initialization of this new PMU, necessary setup code in kvm-hv pmu to
create populate and parse a guest-state-buffer holding the Hostwide
counters returned from L0-PowerVM.
The final patch in the series creates the five new perf-events which then
leverage the kernel's perf-event infrastructure to report the Hostwide
counters returned from L0-PowerVM to perf tool.
Output
======
Once the patch-set is integrated, perf-stat should report the Hostwide
counters for a kvm-enabled pseries lpar as below:
$ sudo perf stat -e 'kvm-hv/host_heap/' -e 'kvm-hv/host_heap_max/' \
-e 'kvm-hv/host_pagetable/' -e 'kvm-hv/host_pagetable_max/' \
-e 'kvm-hv/host_pagetable_reclaim/' -- sleep 0
Performance counter stats for 'system wide':
0 kvm-hv/host_heap/
10,995,367,936 kvm-hv/host_heap_max/
2,178,304 kvm-hv/host_pagetable/
2,147,483,648 kvm-hv/host_pagetable_max/
0 kvm-hv/host_pagetable_reclaim/
References
==========
[1] - commit 19d31c5f1157 ("KVM: PPC: Add support for nestedv2 guests")
[2] - "KVM in a PowerVM LPAR: A Power user guide Part II"
https://ibm.biz/BdGHeY
Vaibhav Jain (6):
powerpc: Document APIv2 KVM hcall spec for Hostwide counters
kvm powerpc/book3s-apiv2: Add support for Hostwide GSB elements
kvm powerpc/book3s-apiv2: Add kunit tests for Hostwide GSB elements
kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU
powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters
kvm powerpc/book3s-hv-pmu: Add perf-events for Hostwide counters
Documentation/arch/powerpc/kvm-nested.rst | 40 +-
arch/powerpc/include/asm/guest-state-buffer.h | 35 +-
arch/powerpc/include/asm/hvcall.h | 13 +-
arch/powerpc/include/asm/kvm_book3s.h | 12 +
arch/powerpc/kvm/Makefile | 6 +
arch/powerpc/kvm/book3s_hv.c | 7 +
arch/powerpc/kvm/book3s_hv_nestedv2.c | 6 +
arch/powerpc/kvm/book3s_hv_pmu.c | 412 ++++++++++++++++++
arch/powerpc/kvm/guest-state-buffer.c | 39 ++
arch/powerpc/kvm/test-guest-state-buffer.c | 210 +++++++++
10 files changed, 757 insertions(+), 23 deletions(-)
create mode 100644 arch/powerpc/kvm/book3s_hv_pmu.c
--
2.47.1
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
@ 2024-12-22 14:02 ` Vaibhav Jain
2024-12-30 7:03 ` Gautam Menghani
2024-12-22 14:02 ` [PATCH 2/6] kvm powerpc/book3s-apiv2: Add support for Hostwide GSB elements Vaibhav Jain
` (4 subsequent siblings)
5 siblings, 1 reply; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc, linux-doc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
Update kvm-nested APIv2 documentation to include five new
Guest-State-Elements to fetch the hostwide counters. These counters are
per L1-Lpar and indicate the amount of Heap/Page-table memory allocated,
available and Page-table memory reclaimed for all L2-Guests active
instances
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
Documentation/arch/powerpc/kvm-nested.rst | 40 ++++++++++++++++-------
1 file changed, 29 insertions(+), 11 deletions(-)
diff --git a/Documentation/arch/powerpc/kvm-nested.rst b/Documentation/arch/powerpc/kvm-nested.rst
index 5defd13cc6c1..c506192f3f98 100644
--- a/Documentation/arch/powerpc/kvm-nested.rst
+++ b/Documentation/arch/powerpc/kvm-nested.rst
@@ -208,13 +208,9 @@ associated values for each ID in the GSB::
flags:
Bit 0: getGuestWideState: Request state of the Guest instead
of an individual VCPU.
- Bit 1: takeOwnershipOfVcpuState Indicate the L1 is taking
- over ownership of the VCPU state and that the L0 can free
- the storage holding the state. The VCPU state will need to
- be returned to the Hypervisor via H_GUEST_SET_STATE prior
- to H_GUEST_RUN_VCPU being called for this VCPU. The data
- returned in the dataBuffer is in a Hypervisor internal
- format.
+ Bit 1: getHostWideState: Request stats of the Host. This causes
+ the guestId and vcpuId parameters to be ignored and attempting
+ to get the VCPU/Guest state will cause an error.
Bits 2-63: Reserved
guestId: ID obtained from H_GUEST_CREATE
vcpuId: ID of the vCPU pass to H_GUEST_CREATE_VCPU
@@ -402,13 +398,14 @@ GSB element:
The ID in the GSB element specifies what is to be set. This includes
archtected state like GPRs, VSRs, SPRs, plus also some meta data about
-the partition like the timebase offset and partition scoped page
+the partition and like the timebase offset and partition scoped page
table information.
+--------+-------+----+--------+----------------------------------+
-| ID | Size | RW | Thread | Details |
-| | Bytes | | Guest | |
-| | | | Scope | |
+| ID | Size | RW |(H)ost | Details |
+| | Bytes | |(G)uest | |
+| | | |(T)hread| |
+| | | |Scope | |
+========+=======+====+========+==================================+
| 0x0000 | | RW | TG | NOP element |
+--------+-------+----+--------+----------------------------------+
@@ -434,6 +431,27 @@ table information.
| | | | |- 0x8 Table size. |
+--------+-------+----+--------+----------------------------------+
| 0x0007-| | | | Reserved |
+| 0x07FF | | | | |
++--------+-------+----+--------+----------------------------------+
+| 0x0800 | 0x08 | R | H | Current usage in bytes of the |
+| | | | | L0's Guest Management Space |
++--------+-------+----+--------+----------------------------------+
+| 0x0801 | 0x08 | R | H | Max bytes available in the |
+| | | | | L0's Guest Management Space |
++--------+-------+----+--------+----------------------------------+
+| 0x0802 | 0x08 | R | H | Current usage in bytes of the |
+| | | | | L0's Guest Page Table Management |
+| | | | | Space |
++--------+-------+----+--------+----------------------------------+
+| 0x0803 | 0x08 | R | H | Max bytes available in the L0's |
+| | | | | Guest Page Table Management |
+| | | | | Space |
++--------+-------+----+--------+----------------------------------+
+| 0x0804 | 0x08 | R | H | Amount of reclaimed L0 Guest's |
+| | | | | Page Table Management Space due |
+| | | | | to overcommit |
++--------+-------+----+--------+----------------------------------+
+| 0x0805-| | | | Reserved |
| 0x0BFF | | | | |
+--------+-------+----+--------+----------------------------------+
| 0x0C00 | 0x10 | RW | T |Run vCPU Input Buffer: |
--
2.47.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/6] kvm powerpc/book3s-apiv2: Add support for Hostwide GSB elements
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
2024-12-22 14:02 ` [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters Vaibhav Jain
@ 2024-12-22 14:02 ` Vaibhav Jain
2024-12-22 14:02 ` [PATCH 3/6] kvm powerpc/book3s-apiv2: Add kunit tests " Vaibhav Jain
` (3 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
Add support for adding and parsing Hostwide elements to the
Guest-state-buffer data structure used in apiv2. These elements are used to
share meta-information pertaining to entire L1-Lpar and this
meta-information is maintained by L0-PowerVM hypervisor. Example of this
include the amount of the page-table memory currently used by L0-PowerVM
for hosting the Shadow-Pagetable of all active L2-Guests. More of the are
documented in kernel-documentation at [1]. The Hostwide GSB elements are
currently only support with H_GUEST_SET_STATE hcall with a special flag
namely 'KVMPPC_GS_FLAGS_HOST_WIDE'.
The patch introduces new defs for the 5 new Hostwide GSB elements including
their GSIDs as well as introduces a new class of GSB elements namely
'KVMPPC_GS_CLASS_HOSTWIDE' to indicate to GSB construction/parsing
infrastructure in 'kvm/guest-state-buffer.c'. Also
gs_msg_ops_vcpu_get_size(), kvmppc_gsid_type() and
kvmppc_gse_{flatten,unflatten}_iden() are updated to appropriately indicate
the needed size for these Hostwide GSB elements as well as how to
flatten/unflatten their GSIDs so that they can be marked as available in
GSB bitmap.
[1] Documention/arch/powerpc/kvm-nested.rst
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/include/asm/guest-state-buffer.h | 35 ++++++++++++++---
arch/powerpc/include/asm/hvcall.h | 13 ++++---
arch/powerpc/kvm/book3s_hv_nestedv2.c | 6 +++
arch/powerpc/kvm/guest-state-buffer.c | 39 +++++++++++++++++++
4 files changed, 81 insertions(+), 12 deletions(-)
diff --git a/arch/powerpc/include/asm/guest-state-buffer.h b/arch/powerpc/include/asm/guest-state-buffer.h
index d107abe1468f..acd61eb36d59 100644
--- a/arch/powerpc/include/asm/guest-state-buffer.h
+++ b/arch/powerpc/include/asm/guest-state-buffer.h
@@ -28,6 +28,21 @@
/* Process Table Info */
#define KVMPPC_GSID_PROCESS_TABLE 0x0006
+/* Guest Management Heap Size */
+#define KVMPPC_GSID_L0_GUEST_HEAP 0x0800
+
+/* Guest Management Heap Max Size */
+#define KVMPPC_GSID_L0_GUEST_HEAP_MAX 0x0801
+
+/* Guest Pagetable Size */
+#define KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE 0x0802
+
+/* Guest Pagetable Max Size */
+#define KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX 0x0803
+
+/* Guest Pagetable Reclaim in bytes */
+#define KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM 0x0804
+
/* H_GUEST_RUN_VCPU input buffer Info */
#define KVMPPC_GSID_RUN_INPUT 0x0C00
/* H_GUEST_RUN_VCPU output buffer Info */
@@ -106,6 +121,11 @@
#define KVMPPC_GSE_GUESTWIDE_COUNT \
(KVMPPC_GSE_GUESTWIDE_END - KVMPPC_GSE_GUESTWIDE_START + 1)
+#define KVMPPC_GSE_HOSTWIDE_START KVMPPC_GSID_L0_GUEST_HEAP
+#define KVMPPC_GSE_HOSTWIDE_END KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM
+#define KVMPPC_GSE_HOSTWIDE_COUNT \
+ (KVMPPC_GSE_HOSTWIDE_END - KVMPPC_GSE_HOSTWIDE_START + 1)
+
#define KVMPPC_GSE_META_START KVMPPC_GSID_RUN_INPUT
#define KVMPPC_GSE_META_END KVMPPC_GSID_VPA
#define KVMPPC_GSE_META_COUNT (KVMPPC_GSE_META_END - KVMPPC_GSE_META_START + 1)
@@ -130,7 +150,8 @@
(KVMPPC_GSE_INTR_REGS_END - KVMPPC_GSE_INTR_REGS_START + 1)
#define KVMPPC_GSE_IDEN_COUNT \
- (KVMPPC_GSE_GUESTWIDE_COUNT + KVMPPC_GSE_META_COUNT + \
+ (KVMPPC_GSE_HOSTWIDE_COUNT + \
+ KVMPPC_GSE_GUESTWIDE_COUNT + KVMPPC_GSE_META_COUNT + \
KVMPPC_GSE_DW_REGS_COUNT + KVMPPC_GSE_W_REGS_COUNT + \
KVMPPC_GSE_VSRS_COUNT + KVMPPC_GSE_INTR_REGS_COUNT)
@@ -139,10 +160,11 @@
*/
enum {
KVMPPC_GS_CLASS_GUESTWIDE = 0x01,
- KVMPPC_GS_CLASS_META = 0x02,
- KVMPPC_GS_CLASS_DWORD_REG = 0x04,
- KVMPPC_GS_CLASS_WORD_REG = 0x08,
- KVMPPC_GS_CLASS_VECTOR = 0x10,
+ KVMPPC_GS_CLASS_HOSTWIDE = 0x02,
+ KVMPPC_GS_CLASS_META = 0x04,
+ KVMPPC_GS_CLASS_DWORD_REG = 0x08,
+ KVMPPC_GS_CLASS_WORD_REG = 0x10,
+ KVMPPC_GS_CLASS_VECTOR = 0x18,
KVMPPC_GS_CLASS_INTR = 0x20,
};
@@ -164,6 +186,7 @@ enum {
*/
enum {
KVMPPC_GS_FLAGS_WIDE = 0x01,
+ KVMPPC_GS_FLAGS_HOST_WIDE = 0x02,
};
/**
@@ -287,7 +310,7 @@ struct kvmppc_gs_msg_ops {
* struct kvmppc_gs_msg - a guest state message
* @bitmap: the guest state ids that should be included
* @ops: modify message behavior for reading and writing to buffers
- * @flags: guest wide or thread wide
+ * @flags: host wide, guest wide or thread wide
* @data: location where buffer data will be written to or from.
*
* A guest state message is allows flexibility in sending in receiving data
diff --git a/arch/powerpc/include/asm/hvcall.h b/arch/powerpc/include/asm/hvcall.h
index 65d1f291393d..1c12713538a4 100644
--- a/arch/powerpc/include/asm/hvcall.h
+++ b/arch/powerpc/include/asm/hvcall.h
@@ -489,14 +489,15 @@
#define H_RPTI_PAGE_ALL (-1UL)
/* Flags for H_GUEST_{S,G}_STATE */
-#define H_GUEST_FLAGS_WIDE (1UL<<(63-0))
+#define H_GUEST_FLAGS_WIDE (1UL << (63 - 0))
+#define H_GUEST_FLAGS_HOST_WIDE (1UL << (63 - 1))
/* Flag values used for H_{S,G}SET_GUEST_CAPABILITIES */
-#define H_GUEST_CAP_COPY_MEM (1UL<<(63-0))
-#define H_GUEST_CAP_POWER9 (1UL<<(63-1))
-#define H_GUEST_CAP_POWER10 (1UL<<(63-2))
-#define H_GUEST_CAP_POWER11 (1UL<<(63-3))
-#define H_GUEST_CAP_BITMAP2 (1UL<<(63-63))
+#define H_GUEST_CAP_COPY_MEM (1UL << (63 - 0))
+#define H_GUEST_CAP_POWER9 (1UL << (63 - 1))
+#define H_GUEST_CAP_POWER10 (1UL << (63 - 2))
+#define H_GUEST_CAP_POWER11 (1UL << (63 - 3))
+#define H_GUEST_CAP_BITMAP2 (1UL << (63 - 63))
#ifndef __ASSEMBLY__
#include <linux/types.h>
diff --git a/arch/powerpc/kvm/book3s_hv_nestedv2.c b/arch/powerpc/kvm/book3s_hv_nestedv2.c
index e5c7ce1fb761..87691cf86cae 100644
--- a/arch/powerpc/kvm/book3s_hv_nestedv2.c
+++ b/arch/powerpc/kvm/book3s_hv_nestedv2.c
@@ -123,6 +123,12 @@ static size_t gs_msg_ops_vcpu_get_size(struct kvmppc_gs_msg *gsm)
case KVMPPC_GSID_PROCESS_TABLE:
case KVMPPC_GSID_RUN_INPUT:
case KVMPPC_GSID_RUN_OUTPUT:
+ /* Host wide counters */
+ case KVMPPC_GSID_L0_GUEST_HEAP:
+ case KVMPPC_GSID_L0_GUEST_HEAP_MAX:
+ case KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE:
+ case KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX:
+ case KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM:
break;
default:
size += kvmppc_gse_total_size(kvmppc_gsid_size(iden));
diff --git a/arch/powerpc/kvm/guest-state-buffer.c b/arch/powerpc/kvm/guest-state-buffer.c
index b80dbc58621f..871cf60ddeb6 100644
--- a/arch/powerpc/kvm/guest-state-buffer.c
+++ b/arch/powerpc/kvm/guest-state-buffer.c
@@ -92,6 +92,10 @@ static int kvmppc_gsid_class(u16 iden)
(iden <= KVMPPC_GSE_GUESTWIDE_END))
return KVMPPC_GS_CLASS_GUESTWIDE;
+ if ((iden >= KVMPPC_GSE_HOSTWIDE_START) &&
+ (iden <= KVMPPC_GSE_HOSTWIDE_END))
+ return KVMPPC_GS_CLASS_HOSTWIDE;
+
if ((iden >= KVMPPC_GSE_META_START) && (iden <= KVMPPC_GSE_META_END))
return KVMPPC_GS_CLASS_META;
@@ -118,6 +122,21 @@ static int kvmppc_gsid_type(u16 iden)
int type = -1;
switch (kvmppc_gsid_class(iden)) {
+ case KVMPPC_GS_CLASS_HOSTWIDE:
+ switch (iden) {
+ case KVMPPC_GSID_L0_GUEST_HEAP:
+ fallthrough;
+ case KVMPPC_GSID_L0_GUEST_HEAP_MAX:
+ fallthrough;
+ case KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE:
+ fallthrough;
+ case KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX:
+ fallthrough;
+ case KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM:
+ type = KVMPPC_GSE_BE64;
+ break;
+ }
+ break;
case KVMPPC_GS_CLASS_GUESTWIDE:
switch (iden) {
case KVMPPC_GSID_HOST_STATE_SIZE:
@@ -187,6 +206,9 @@ unsigned long kvmppc_gsid_flags(u16 iden)
case KVMPPC_GS_CLASS_GUESTWIDE:
flags = KVMPPC_GS_FLAGS_WIDE;
break;
+ case KVMPPC_GS_CLASS_HOSTWIDE:
+ flags = KVMPPC_GS_FLAGS_HOST_WIDE;
+ break;
case KVMPPC_GS_CLASS_META:
case KVMPPC_GS_CLASS_DWORD_REG:
case KVMPPC_GS_CLASS_WORD_REG:
@@ -310,6 +332,13 @@ static inline int kvmppc_gse_flatten_iden(u16 iden)
bit += KVMPPC_GSE_GUESTWIDE_COUNT;
+ if (class == KVMPPC_GS_CLASS_HOSTWIDE) {
+ bit += iden - KVMPPC_GSE_HOSTWIDE_START;
+ return bit;
+ }
+
+ bit += KVMPPC_GSE_HOSTWIDE_COUNT;
+
if (class == KVMPPC_GS_CLASS_META) {
bit += iden - KVMPPC_GSE_META_START;
return bit;
@@ -356,6 +385,12 @@ static inline u16 kvmppc_gse_unflatten_iden(int bit)
}
bit -= KVMPPC_GSE_GUESTWIDE_COUNT;
+ if (bit < KVMPPC_GSE_HOSTWIDE_COUNT) {
+ iden = KVMPPC_GSE_HOSTWIDE_START + bit;
+ return iden;
+ }
+ bit -= KVMPPC_GSE_HOSTWIDE_COUNT;
+
if (bit < KVMPPC_GSE_META_COUNT) {
iden = KVMPPC_GSE_META_START + bit;
return iden;
@@ -588,6 +623,8 @@ int kvmppc_gsb_send(struct kvmppc_gs_buff *gsb, unsigned long flags)
if (flags & KVMPPC_GS_FLAGS_WIDE)
hflags |= H_GUEST_FLAGS_WIDE;
+ if (flags & KVMPPC_GS_FLAGS_HOST_WIDE)
+ hflags |= H_GUEST_FLAGS_HOST_WIDE;
rc = plpar_guest_set_state(hflags, gsb->guest_id, gsb->vcpu_id,
__pa(gsb->hdr), gsb->capacity, &i);
@@ -613,6 +650,8 @@ int kvmppc_gsb_recv(struct kvmppc_gs_buff *gsb, unsigned long flags)
if (flags & KVMPPC_GS_FLAGS_WIDE)
hflags |= H_GUEST_FLAGS_WIDE;
+ if (flags & KVMPPC_GS_FLAGS_HOST_WIDE)
+ hflags |= H_GUEST_FLAGS_HOST_WIDE;
rc = plpar_guest_get_state(hflags, gsb->guest_id, gsb->vcpu_id,
__pa(gsb->hdr), gsb->capacity, &i);
--
2.47.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 3/6] kvm powerpc/book3s-apiv2: Add kunit tests for Hostwide GSB elements
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
2024-12-22 14:02 ` [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters Vaibhav Jain
2024-12-22 14:02 ` [PATCH 2/6] kvm powerpc/book3s-apiv2: Add support for Hostwide GSB elements Vaibhav Jain
@ 2024-12-22 14:02 ` Vaibhav Jain
2024-12-22 14:02 ` [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU Vaibhav Jain
` (2 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
Update 'test-guest-state-buffer.c' to add two new KUNIT test cases for
validating correctness of changes to Guest-state-buffer management
infrastructure for adding support for Hostwide GSB elements.
The newly introduced test test_gs_hostwide_msg() checks if the Hostwide
elements can be set and parsed from a Guest-state-buffer. The second kunit
test test_gs_hostwide_counters() checks if the Hostwide GSB elements can be
send to the L0-PowerVM hypervisor via the H_GUEST_SET_STATE hcall and
ensures that the returned guest-state-buffer has all the 5 Hostwide stat
counters present.
Below is the KATP test report with the newly added KUNIT tests:
KTAP version 1
# Subtest: guest_state_buffer_test
# module: test_guest_state_buffer
1..7
ok 1 test_creating_buffer
ok 2 test_adding_element
ok 3 test_gs_bitmap
ok 4 test_gs_parsing
ok 5 test_gs_msg
ok 6 test_gs_hostwide_msg
# test_gs_hostwide_counters: Guest Heap Size=0 bytes
# test_gs_hostwide_counters: Guest Heap Size Max=10995367936 bytes
# test_gs_hostwide_counters: Guest Page-table Size=2178304 bytes
# test_gs_hostwide_counters: Guest Page-table Size Max=2147483648 bytes
# test_gs_hostwide_counters: Guest Page-table Reclaim Size=0 bytes
ok 7 test_gs_hostwide_counters
# guest_state_buffer_test: pass:7 fail:0 skip:0 total:7
# Totals: pass:7 fail:0 skip:0 total:7
ok 1 guest_state_buffer_test
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/kvm/test-guest-state-buffer.c | 210 +++++++++++++++++++++
1 file changed, 210 insertions(+)
diff --git a/arch/powerpc/kvm/test-guest-state-buffer.c b/arch/powerpc/kvm/test-guest-state-buffer.c
index bfd225329a18..99a3d4b12843 100644
--- a/arch/powerpc/kvm/test-guest-state-buffer.c
+++ b/arch/powerpc/kvm/test-guest-state-buffer.c
@@ -141,6 +141,16 @@ static void test_gs_bitmap(struct kunit *test)
i++;
}
+ for (u16 iden = KVMPPC_GSID_L0_GUEST_HEAP;
+ iden <= KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM; iden++) {
+ kvmppc_gsbm_set(&gsbm, iden);
+ kvmppc_gsbm_set(&gsbm1, iden);
+ KUNIT_EXPECT_TRUE(test, kvmppc_gsbm_test(&gsbm, iden));
+ kvmppc_gsbm_clear(&gsbm, iden);
+ KUNIT_EXPECT_FALSE(test, kvmppc_gsbm_test(&gsbm, iden));
+ i++;
+ }
+
for (u16 iden = KVMPPC_GSID_RUN_INPUT; iden <= KVMPPC_GSID_VPA;
iden++) {
kvmppc_gsbm_set(&gsbm, iden);
@@ -309,12 +319,212 @@ static void test_gs_msg(struct kunit *test)
kvmppc_gsm_free(gsm);
}
+/* Test data struct for hostwide/L0 counters */
+struct kvmppc_gs_msg_test_hostwide_data {
+ u64 guest_heap;
+ u64 guest_heap_max;
+ u64 guest_pgtable_size;
+ u64 guest_pgtable_size_max;
+ u64 guest_pgtable_reclaim;
+};
+
+static size_t test_hostwide_get_size(struct kvmppc_gs_msg *gsm)
+
+{
+ size_t size = 0;
+ u16 ids[] = {
+ KVMPPC_GSID_L0_GUEST_HEAP,
+ KVMPPC_GSID_L0_GUEST_HEAP_MAX,
+ KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
+ KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
+ KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM
+ };
+
+ for (int i = 0; i < ARRAY_SIZE(ids); i++)
+ size += kvmppc_gse_total_size(kvmppc_gsid_size(ids[i]));
+ return size;
+}
+
+static int test_hostwide_fill_info(struct kvmppc_gs_buff *gsb,
+ struct kvmppc_gs_msg *gsm)
+{
+ struct kvmppc_gs_msg_test_hostwide_data *data = gsm->data;
+
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP,
+ data->guest_heap);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP_MAX))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP_MAX,
+ data->guest_heap_max);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
+ data->guest_pgtable_size);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
+ data->guest_pgtable_size_max);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM,
+ data->guest_pgtable_reclaim);
+
+ return 0;
+}
+
+static int test_hostwide_refresh_info(struct kvmppc_gs_msg *gsm,
+ struct kvmppc_gs_buff *gsb)
+{
+ struct kvmppc_gs_parser gsp = { 0 };
+ struct kvmppc_gs_msg_test_hostwide_data *data = gsm->data;
+ struct kvmppc_gs_elem *gse;
+ int rc;
+
+ rc = kvmppc_gse_parse(&gsp, gsb);
+ if (rc < 0)
+ return rc;
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP);
+ if (gse)
+ data->guest_heap = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
+ if (gse)
+ data->guest_heap_max = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
+ if (gse)
+ data->guest_pgtable_size = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
+ if (gse)
+ data->guest_pgtable_size_max = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
+ if (gse)
+ data->guest_pgtable_reclaim = kvmppc_gse_get_u64(gse);
+
+ return 0;
+}
+
+static struct kvmppc_gs_msg_ops gs_msg_test_hostwide_ops = {
+ .get_size = test_hostwide_get_size,
+ .fill_info = test_hostwide_fill_info,
+ .refresh_info = test_hostwide_refresh_info,
+};
+
+static void test_gs_hostwide_msg(struct kunit *test)
+{
+ struct kvmppc_gs_msg_test_hostwide_data test_data = {
+ .guest_heap = 0xdeadbeef,
+ .guest_heap_max = ~0ULL,
+ .guest_pgtable_size = 0xff,
+ .guest_pgtable_size_max = 0xffffff,
+ .guest_pgtable_reclaim = 0xdeadbeef,
+ };
+ struct kvmppc_gs_msg *gsm;
+ struct kvmppc_gs_buff *gsb;
+
+ gsm = kvmppc_gsm_new(&gs_msg_test_hostwide_ops, &test_data, GSM_SEND,
+ GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsm);
+
+ gsb = kvmppc_gsb_new(kvmppc_gsm_size(gsm), 0, 0, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb);
+
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_HEAP);
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
+
+ kvmppc_gsm_fill_info(gsm, gsb);
+
+ memset(&test_data, 0, sizeof(test_data));
+
+ kvmppc_gsm_refresh_info(gsm, gsb);
+ KUNIT_EXPECT_EQ(test, test_data.guest_heap, 0xdeadbeef);
+ KUNIT_EXPECT_EQ(test, test_data.guest_heap_max, ~0ULL);
+ KUNIT_EXPECT_EQ(test, test_data.guest_pgtable_size, 0xff);
+ KUNIT_EXPECT_EQ(test, test_data.guest_pgtable_size_max, 0xffffff);
+ KUNIT_EXPECT_EQ(test, test_data.guest_pgtable_reclaim, 0xdeadbeef);
+
+ kvmppc_gsm_free(gsm);
+}
+
+/* Test if the H_GUEST_GET_STATE for hostwide counters works */
+static void test_gs_hostwide_counters(struct kunit *test)
+{
+ struct kvmppc_gs_msg_test_hostwide_data test_data;
+ struct kvmppc_gs_parser gsp = { 0 };
+
+ struct kvmppc_gs_msg *gsm;
+ struct kvmppc_gs_buff *gsb;
+ struct kvmppc_gs_elem *gse;
+ int rc;
+
+ gsm = kvmppc_gsm_new(&gs_msg_test_hostwide_ops, &test_data, GSM_SEND,
+ GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsm);
+
+ gsb = kvmppc_gsb_new(kvmppc_gsm_size(gsm), 0, 0, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, gsb);
+
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_HEAP);
+
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
+
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
+
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
+
+ kvmppc_gsm_include(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
+
+ kvmppc_gsm_fill_info(gsm, gsb);
+
+ /* With HOST_WIDE flags guestid and vcpuid will be ignored */
+ rc = kvmppc_gsb_recv(gsb, KVMPPC_GS_FLAGS_HOST_WIDE);
+ KUNIT_ASSERT_EQ(test, rc, 0);
+
+ /* Parse the guest state buffer is successful */
+ rc = kvmppc_gse_parse(&gsp, gsb);
+ KUNIT_ASSERT_EQ(test, rc, 0);
+
+ /* Parse the GSB and get the counters */
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP);
+ KUNIT_ASSERT_NOT_NULL_MSG(test, gse, "L0 Heap counter missing");
+ kunit_info(test, "Guest Heap Size=%llu bytes",
+ kvmppc_gse_get_u64(gse));
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
+ KUNIT_ASSERT_NOT_NULL_MSG(test, gse, "L0 Heap counter max missing");
+ kunit_info(test, "Guest Heap Size Max=%llu bytes",
+ kvmppc_gse_get_u64(gse));
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
+ KUNIT_ASSERT_NOT_NULL_MSG(test, gse, "L0 page-table size missing");
+ kunit_info(test, "Guest Page-table Size=%llu bytes",
+ kvmppc_gse_get_u64(gse));
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
+ KUNIT_ASSERT_NOT_NULL_MSG(test, gse, "L0 page-table size-max missing");
+ kunit_info(test, "Guest Page-table Size Max=%llu bytes",
+ kvmppc_gse_get_u64(gse));
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
+ KUNIT_ASSERT_NOT_NULL_MSG(test, gse, "L0 page-table reclaim size missing");
+ kunit_info(test, "Guest Page-table Reclaim Size=%llu bytes",
+ kvmppc_gse_get_u64(gse));
+
+ kvmppc_gsm_free(gsm);
+ kvmppc_gsb_free(gsb);
+}
+
static struct kunit_case guest_state_buffer_testcases[] = {
KUNIT_CASE(test_creating_buffer),
KUNIT_CASE(test_adding_element),
KUNIT_CASE(test_gs_bitmap),
KUNIT_CASE(test_gs_parsing),
KUNIT_CASE(test_gs_msg),
+ KUNIT_CASE(test_gs_hostwide_msg),
+ KUNIT_CASE(test_gs_hostwide_counters),
{}
};
--
2.47.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
` (2 preceding siblings ...)
2024-12-22 14:02 ` [PATCH 3/6] kvm powerpc/book3s-apiv2: Add kunit tests " Vaibhav Jain
@ 2024-12-22 14:02 ` Vaibhav Jain
2024-12-30 9:28 ` Gautam Menghani
2024-12-22 14:02 ` [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters Vaibhav Jain
2024-12-22 14:02 ` [PATCH 6/6] kvm powerpc/book3s-hv-pmu: Add perf-events for Hostwide counters Vaibhav Jain
5 siblings, 1 reply; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
Introduce a new PMU named 'kvm-hv' to report Book3s kvm-hv specific
performance counters. This will expose KVM-HV specific performance
attributes to user-space via kernel's PMU infrastructure and would enable
users to monitor active kvm-hv based guests.
The patch creates necessary scaffolding to for the new PMU callbacks and
introduces two new exports kvmppc_{,un}register_pmu() that are called from
kvm-hv init and exit function to perform initialize and cleanup for the
'kvm-hv' PMU. The patch doesn't introduce any perf-events yet, which will
be introduced in later patches
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/include/asm/kvm_book3s.h | 12 +++
arch/powerpc/kvm/Makefile | 6 ++
arch/powerpc/kvm/book3s_hv.c | 7 ++
arch/powerpc/kvm/book3s_hv_pmu.c | 133 ++++++++++++++++++++++++++
4 files changed, 158 insertions(+)
create mode 100644 arch/powerpc/kvm/book3s_hv_pmu.c
diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h
index e1ff291ba891..cf91a1493159 100644
--- a/arch/powerpc/include/asm/kvm_book3s.h
+++ b/arch/powerpc/include/asm/kvm_book3s.h
@@ -334,6 +334,9 @@ static inline bool kvmhv_is_nestedv1(void)
return !static_branch_likely(&__kvmhv_is_nestedv2);
}
+int kvmppc_register_pmu(void);
+void kvmppc_unregister_pmu(void);
+
#else
static inline bool kvmhv_is_nestedv2(void)
@@ -346,6 +349,15 @@ static inline bool kvmhv_is_nestedv1(void)
return false;
}
+static int kvmppc_register_pmu(void)
+{
+ return 0;
+}
+
+static void kvmppc_unregister_pmu(void)
+{
+}
+
#endif
int __kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu, struct pt_regs *regs);
diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
index 4bd9d1230869..094c3916d9d0 100644
--- a/arch/powerpc/kvm/Makefile
+++ b/arch/powerpc/kvm/Makefile
@@ -92,6 +92,12 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
$(kvm-book3s_64-builtin-tm-objs-y) \
$(kvm-book3s_64-builtin-xics-objs-y)
+# enable kvm_hv perf events
+ifdef CONFIG_HAVE_PERF_EVENTS
+kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
+ book3s_hv_pmu.o
+endif
+
obj-$(CONFIG_GUEST_STATE_BUFFER_TEST) += test-guest-state-buffer.o
endif
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 25429905ae90..83bcce2fb557 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -6662,6 +6662,12 @@ static int kvmppc_book3s_init_hv(void)
return r;
}
+ r = kvmppc_register_pmu();
+ if (r) {
+ pr_err("KVM-HV: Unable to register PMUs %d\n", r);
+ goto err;
+ }
+
kvm_ops_hv.owner = THIS_MODULE;
kvmppc_hv_ops = &kvm_ops_hv;
@@ -6676,6 +6682,7 @@ static int kvmppc_book3s_init_hv(void)
static void kvmppc_book3s_exit_hv(void)
{
+ kvmppc_unregister_pmu();
kvmppc_uvmem_free();
kvmppc_free_host_rm_ops();
if (kvmppc_radix_possible())
diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
new file mode 100644
index 000000000000..e72542d5e750
--- /dev/null
+++ b/arch/powerpc/kvm/book3s_hv_pmu.c
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Description: PMUs specific to running nested KVM-HV guests
+ * on Book3S processors (specifically POWER9 and later).
+ */
+
+#define pr_fmt(fmt) "kvmppc-pmu: " fmt
+
+#include "asm-generic/local64.h"
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/ratelimit.h>
+#include <linux/kvm_host.h>
+#include <linux/gfp_types.h>
+#include <linux/pgtable.h>
+#include <linux/perf_event.h>
+#include <linux/spinlock_types.h>
+#include <linux/spinlock.h>
+
+#include <asm/types.h>
+#include <asm/kvm_ppc.h>
+#include <asm/kvm_book3s.h>
+#include <asm/mmu.h>
+#include <asm/pgalloc.h>
+#include <asm/pte-walk.h>
+#include <asm/reg.h>
+#include <asm/plpar_wrappers.h>
+#include <asm/firmware.h>
+
+enum kvmppc_pmu_eventid {
+ KVMPPC_EVENT_MAX,
+};
+
+static struct attribute *kvmppc_pmu_events_attr[] = {
+ NULL,
+};
+
+static const struct attribute_group kvmppc_pmu_events_group = {
+ .name = "events",
+ .attrs = kvmppc_pmu_events_attr,
+};
+
+PMU_FORMAT_ATTR(event, "config:0");
+static struct attribute *kvmppc_pmu_format_attr[] = {
+ &format_attr_event.attr,
+ NULL,
+};
+
+static struct attribute_group kvmppc_pmu_format_group = {
+ .name = "format",
+ .attrs = kvmppc_pmu_format_attr,
+};
+
+static const struct attribute_group *kvmppc_pmu_attr_groups[] = {
+ &kvmppc_pmu_events_group,
+ &kvmppc_pmu_format_group,
+ NULL,
+};
+
+static int kvmppc_pmu_event_init(struct perf_event *event)
+{
+ unsigned int config = event->attr.config;
+
+ pr_debug("%s: Event(%p) id=%llu cpu=%x on_cpu=%x config=%u",
+ __func__, event, event->id, event->cpu,
+ event->oncpu, config);
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ if (config >= KVMPPC_EVENT_MAX)
+ return -EINVAL;
+
+ local64_set(&event->hw.prev_count, 0);
+ local64_set(&event->count, 0);
+
+ return 0;
+}
+
+static void kvmppc_pmu_del(struct perf_event *event, int flags)
+{
+}
+
+static int kvmppc_pmu_add(struct perf_event *event, int flags)
+{
+ return 0;
+}
+
+static void kvmppc_pmu_read(struct perf_event *event)
+{
+}
+
+/* L1 wide counters PMU */
+static struct pmu kvmppc_pmu = {
+ .task_ctx_nr = perf_sw_context,
+ .name = "kvm-hv",
+ .event_init = kvmppc_pmu_event_init,
+ .add = kvmppc_pmu_add,
+ .del = kvmppc_pmu_del,
+ .read = kvmppc_pmu_read,
+ .attr_groups = kvmppc_pmu_attr_groups,
+ .type = -1,
+};
+
+int kvmppc_register_pmu(void)
+{
+ int rc = -EOPNOTSUPP;
+
+ /* only support events for nestedv2 right now */
+ if (kvmhv_is_nestedv2()) {
+ /* Setup done now register the PMU */
+ pr_info("Registering kvm-hv pmu");
+
+ /* Register only if we arent already registyered */
+ rc = (kvmppc_pmu.type == -1) ?
+ perf_pmu_register(&kvmppc_pmu, kvmppc_pmu.name,
+ -1) : 0;
+ }
+
+ return rc;
+}
+EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
+
+void kvmppc_unregister_pmu(void)
+{
+ if (kvmhv_is_nestedv2()) {
+ if (kvmppc_pmu.type != -1)
+ perf_pmu_unregister(&kvmppc_pmu);
+
+ pr_info("kvmhv_pmu unregistered.\n");
+ }
+}
+EXPORT_SYMBOL_GPL(kvmppc_unregister_pmu);
--
2.47.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
` (3 preceding siblings ...)
2024-12-22 14:02 ` [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU Vaibhav Jain
@ 2024-12-22 14:02 ` Vaibhav Jain
2024-12-30 7:31 ` Gautam Menghani
2024-12-22 14:02 ` [PATCH 6/6] kvm powerpc/book3s-hv-pmu: Add perf-events for Hostwide counters Vaibhav Jain
5 siblings, 1 reply; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
Implement and setup necessary structures to send a prepolulated
Guest-State-Buffer(GSB) requesting hostwide counters to L0-PowerVM and have
the returned GSB holding the values of these counters parsed. This is done
via existing GSB implementation and with the newly added support of
Hostwide elements in GSB.
The request to L0-PowerVM to return Hostwide counters is done using a
pre-allocated GSB named 'gsb_l0_stats'. To be able to populate this GSB
with the needed Guest-State-Elements (GSIDs) a instance of 'struct
kvmppc_gs_msg' named 'gsm_l0_stats' is introduced. The 'gsm_l0_stats' is
tied to an instance of 'struct kvmppc_gs_msg_ops' named 'gsb_ops_l0_stats'
which holds various callbacks to be compute the size ( hostwide_get_size()
), populate the GSB ( hostwide_fill_info() ) and
refresh ( hostwide_refresh_info() ) the contents of
'l0_stats' that holds the Hostwide counters returned from L0-PowerVM.
To protect these structures from simultaneous access a spinlock
'lock_l0_stats' has been introduced. The allocation and initialization of
the above structures is done in newly introduced kvmppc_init_hostwide() and
similarly the cleanup is performed in newly introduced
kvmppc_cleanup_hostwide().
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/kvm/book3s_hv_pmu.c | 189 +++++++++++++++++++++++++++++++
1 file changed, 189 insertions(+)
diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
index e72542d5e750..f7fd5190ecf7 100644
--- a/arch/powerpc/kvm/book3s_hv_pmu.c
+++ b/arch/powerpc/kvm/book3s_hv_pmu.c
@@ -27,10 +27,31 @@
#include <asm/plpar_wrappers.h>
#include <asm/firmware.h>
+#include "asm/guest-state-buffer.h"
+
enum kvmppc_pmu_eventid {
KVMPPC_EVENT_MAX,
};
+#define KVMPPC_PMU_EVENT_ATTR(_name, _id) \
+ PMU_EVENT_ATTR_ID(_name, power_events_sysfs_show, _id)
+
+/* Holds the hostwide stats */
+static struct kvmppc_hostwide_stats {
+ u64 guest_heap;
+ u64 guest_heap_max;
+ u64 guest_pgtable_size;
+ u64 guest_pgtable_size_max;
+ u64 guest_pgtable_reclaim;
+} l0_stats;
+
+/* Protect access to l0_stats */
+static DEFINE_SPINLOCK(lock_l0_stats);
+
+/* GSB related structs needed to talk to L0 */
+static struct kvmppc_gs_msg *gsm_l0_stats;
+static struct kvmppc_gs_buff *gsb_l0_stats;
+
static struct attribute *kvmppc_pmu_events_attr[] = {
NULL,
};
@@ -90,6 +111,167 @@ static void kvmppc_pmu_read(struct perf_event *event)
{
}
+/* Return the size of the needed guest state buffer */
+static size_t hostwide_get_size(struct kvmppc_gs_msg *gsm)
+
+{
+ size_t size = 0;
+ const u16 ids[] = {
+ KVMPPC_GSID_L0_GUEST_HEAP,
+ KVMPPC_GSID_L0_GUEST_HEAP_MAX,
+ KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
+ KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
+ KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM
+ };
+
+ for (int i = 0; i < ARRAY_SIZE(ids); i++)
+ size += kvmppc_gse_total_size(kvmppc_gsid_size(ids[i]));
+ return size;
+}
+
+/* Populate the request guest state buffer */
+static int hostwide_fill_info(struct kvmppc_gs_buff *gsb,
+ struct kvmppc_gs_msg *gsm)
+{
+ struct kvmppc_hostwide_stats *stats = gsm->data;
+
+ /*
+ * It doesn't matter what values are put into request buffer as
+ * they are going to be overwritten anyways. But for the sake of
+ * testcode and symmetry contents of existing stats are put
+ * populated into the request guest state buffer.
+ */
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP,
+ stats->guest_heap);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP_MAX))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP_MAX,
+ stats->guest_heap_max);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
+ stats->guest_pgtable_size);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
+ stats->guest_pgtable_size_max);
+ if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM))
+ kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM,
+ stats->guest_pgtable_reclaim);
+
+ return 0;
+}
+
+/* Parse and update the host wide stats from returned gsb */
+static int hostwide_refresh_info(struct kvmppc_gs_msg *gsm,
+ struct kvmppc_gs_buff *gsb)
+{
+ struct kvmppc_gs_parser gsp = { 0 };
+ struct kvmppc_hostwide_stats *stats = gsm->data;
+ struct kvmppc_gs_elem *gse;
+ int rc;
+
+ rc = kvmppc_gse_parse(&gsp, gsb);
+ if (rc < 0)
+ return rc;
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP);
+ if (gse)
+ stats->guest_heap = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
+ if (gse)
+ stats->guest_heap_max = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
+ if (gse)
+ stats->guest_pgtable_size = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
+ if (gse)
+ stats->guest_pgtable_size_max = kvmppc_gse_get_u64(gse);
+
+ gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
+ if (gse)
+ stats->guest_pgtable_reclaim = kvmppc_gse_get_u64(gse);
+
+ return 0;
+}
+
+/* gsb-message ops for setting up/parsing */
+static struct kvmppc_gs_msg_ops gsb_ops_l0_stats = {
+ .get_size = hostwide_get_size,
+ .fill_info = hostwide_fill_info,
+ .refresh_info = hostwide_refresh_info,
+};
+
+static int kvmppc_init_hostwide(void)
+{
+ int rc = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&lock_l0_stats, flags);
+
+ /* already registered ? */
+ if (gsm_l0_stats) {
+ rc = 0;
+ goto out;
+ }
+
+ /* setup the Guest state message/buffer to talk to L0 */
+ gsm_l0_stats = kvmppc_gsm_new(&gsb_ops_l0_stats, &l0_stats,
+ GSM_SEND, GFP_KERNEL);
+ if (!gsm_l0_stats) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ /* Populate the Idents */
+ kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_HEAP);
+ kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
+ kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
+ kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
+ kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
+
+ /* allocate GSB. Guest/Vcpu Id is ignored */
+ gsb_l0_stats = kvmppc_gsb_new(kvmppc_gsm_size(gsm_l0_stats), 0, 0,
+ GFP_KERNEL);
+ if (!gsb_l0_stats) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ /* ask the ops to fill in the info */
+ rc = kvmppc_gsm_fill_info(gsm_l0_stats, gsb_l0_stats);
+ if (rc)
+ goto out;
+out:
+ if (rc) {
+ if (gsm_l0_stats)
+ kvmppc_gsm_free(gsm_l0_stats);
+ if (gsb_l0_stats)
+ kvmppc_gsb_free(gsb_l0_stats);
+ gsm_l0_stats = NULL;
+ gsb_l0_stats = NULL;
+ }
+ spin_unlock_irqrestore(&lock_l0_stats, flags);
+ return rc;
+}
+
+static void kvmppc_cleanup_hostwide(void)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&lock_l0_stats, flags);
+
+ if (gsm_l0_stats)
+ kvmppc_gsm_free(gsm_l0_stats);
+ if (gsb_l0_stats)
+ kvmppc_gsb_free(gsb_l0_stats);
+ gsm_l0_stats = NULL;
+ gsb_l0_stats = NULL;
+
+ spin_unlock_irqrestore(&lock_l0_stats, flags);
+}
+
/* L1 wide counters PMU */
static struct pmu kvmppc_pmu = {
.task_ctx_nr = perf_sw_context,
@@ -108,6 +290,10 @@ int kvmppc_register_pmu(void)
/* only support events for nestedv2 right now */
if (kvmhv_is_nestedv2()) {
+ rc = kvmppc_init_hostwide();
+ if (rc)
+ goto out;
+
/* Setup done now register the PMU */
pr_info("Registering kvm-hv pmu");
@@ -117,6 +303,7 @@ int kvmppc_register_pmu(void)
-1) : 0;
}
+out:
return rc;
}
EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
@@ -124,6 +311,8 @@ EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
void kvmppc_unregister_pmu(void)
{
if (kvmhv_is_nestedv2()) {
+ kvmppc_cleanup_hostwide();
+
if (kvmppc_pmu.type != -1)
perf_pmu_unregister(&kvmppc_pmu);
--
2.47.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 6/6] kvm powerpc/book3s-hv-pmu: Add perf-events for Hostwide counters
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
` (4 preceding siblings ...)
2024-12-22 14:02 ` [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters Vaibhav Jain
@ 2024-12-22 14:02 ` Vaibhav Jain
5 siblings, 0 replies; 13+ messages in thread
From: Vaibhav Jain @ 2024-12-22 14:02 UTC (permalink / raw)
To: linuxppc-dev, kvm, kvm-ppc
Cc: Vaibhav Jain, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, gautam, kconsul,
amachhiw
Update 'book3s_hv_pmu.c' to add five new perf-events mapped to the five
Hostwide counters. Since these newly introduced perf events are at system
wide scope and can be read from any L1-Lpar CPU, 'kvmppv_pmu's scope and
capabilities are updated appropriately.
Also introduce two new helpers. First is kvmppc_update_l0_stats() that uses
the infrastructure introduced in previous patches to issues the
H_GUEST_GET_STATE hcall L0-PowerVM to fetch guest-state-buffer holding the
latest values of these counters which is then parsed and 'l0_stats'
variable updated.
Second helper is kvmppc_pmu_event_update() which is called from
'kvmppv_pmu' callbacks and uses kvmppc_update_l0_stats() to update
'l0_stats' and the update the 'struct perf_event's event-counter.
Some minor updates to kvmppc_pmu_{add, del, read}() to remove some debug
scaffolding code.
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/kvm/book3s_hv_pmu.c | 92 +++++++++++++++++++++++++++++++-
1 file changed, 91 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
index f7fd5190ecf7..99bd1ae2dddf 100644
--- a/arch/powerpc/kvm/book3s_hv_pmu.c
+++ b/arch/powerpc/kvm/book3s_hv_pmu.c
@@ -30,6 +30,11 @@
#include "asm/guest-state-buffer.h"
enum kvmppc_pmu_eventid {
+ KVMPPC_EVENT_HOST_HEAP,
+ KVMPPC_EVENT_HOST_HEAP_MAX,
+ KVMPPC_EVENT_HOST_PGTABLE,
+ KVMPPC_EVENT_HOST_PGTABLE_MAX,
+ KVMPPC_EVENT_HOST_PGTABLE_RECLAIM,
KVMPPC_EVENT_MAX,
};
@@ -51,8 +56,14 @@ static DEFINE_SPINLOCK(lock_l0_stats);
/* GSB related structs needed to talk to L0 */
static struct kvmppc_gs_msg *gsm_l0_stats;
static struct kvmppc_gs_buff *gsb_l0_stats;
+static struct kvmppc_gs_parser gsp_l0_stats;
static struct attribute *kvmppc_pmu_events_attr[] = {
+ KVMPPC_PMU_EVENT_ATTR(host_heap, KVMPPC_EVENT_HOST_HEAP),
+ KVMPPC_PMU_EVENT_ATTR(host_heap_max, KVMPPC_EVENT_HOST_HEAP_MAX),
+ KVMPPC_PMU_EVENT_ATTR(host_pagetable, KVMPPC_EVENT_HOST_PGTABLE),
+ KVMPPC_PMU_EVENT_ATTR(host_pagetable_max, KVMPPC_EVENT_HOST_PGTABLE_MAX),
+ KVMPPC_PMU_EVENT_ATTR(host_pagetable_reclaim, KVMPPC_EVENT_HOST_PGTABLE_RECLAIM),
NULL,
};
@@ -61,7 +72,7 @@ static const struct attribute_group kvmppc_pmu_events_group = {
.attrs = kvmppc_pmu_events_attr,
};
-PMU_FORMAT_ATTR(event, "config:0");
+PMU_FORMAT_ATTR(event, "config:0-5");
static struct attribute *kvmppc_pmu_format_attr[] = {
&format_attr_event.attr,
NULL,
@@ -78,6 +89,79 @@ static const struct attribute_group *kvmppc_pmu_attr_groups[] = {
NULL,
};
+/*
+ * Issue the hcall to get the L0-host stats.
+ * Should be called with l0-stat lock held
+ */
+static int kvmppc_update_l0_stats(void)
+{
+ int rc;
+
+ /* With HOST_WIDE flags guestid and vcpuid will be ignored */
+ rc = kvmppc_gsb_recv(gsb_l0_stats, KVMPPC_GS_FLAGS_HOST_WIDE);
+ if (rc)
+ goto out;
+
+ /* Parse the guest state buffer is successful */
+ rc = kvmppc_gse_parse(&gsp_l0_stats, gsb_l0_stats);
+ if (rc)
+ goto out;
+
+ /* Update the l0 returned stats*/
+ memset(&l0_stats, 0, sizeof(l0_stats));
+ rc = kvmppc_gsm_refresh_info(gsm_l0_stats, gsb_l0_stats);
+
+out:
+ return rc;
+}
+
+/* Update the value of the given perf_event */
+static int kvmppc_pmu_event_update(struct perf_event *event)
+{
+ int rc;
+ u64 curr_val, prev_val;
+ unsigned long flags;
+ unsigned int config = event->attr.config;
+
+ /* Ensure no one else is modifying the l0_stats */
+ spin_lock_irqsave(&lock_l0_stats, flags);
+
+ rc = kvmppc_update_l0_stats();
+ if (!rc) {
+ switch (config) {
+ case KVMPPC_EVENT_HOST_HEAP:
+ curr_val = l0_stats.guest_heap;
+ break;
+ case KVMPPC_EVENT_HOST_HEAP_MAX:
+ curr_val = l0_stats.guest_heap_max;
+ break;
+ case KVMPPC_EVENT_HOST_PGTABLE:
+ curr_val = l0_stats.guest_pgtable_size;
+ break;
+ case KVMPPC_EVENT_HOST_PGTABLE_MAX:
+ curr_val = l0_stats.guest_pgtable_size_max;
+ break;
+ case KVMPPC_EVENT_HOST_PGTABLE_RECLAIM:
+ curr_val = l0_stats.guest_pgtable_reclaim;
+ break;
+ default:
+ rc = -ENOENT;
+ break;
+ }
+ }
+
+ spin_unlock_irqrestore(&lock_l0_stats, flags);
+
+ /* If no error than update the perf event */
+ if (!rc) {
+ prev_val = local64_xchg(&event->hw.prev_count, curr_val);
+ if (curr_val > prev_val)
+ local64_add(curr_val - prev_val, &event->count);
+ }
+
+ return rc;
+}
+
static int kvmppc_pmu_event_init(struct perf_event *event)
{
unsigned int config = event->attr.config;
@@ -100,15 +184,19 @@ static int kvmppc_pmu_event_init(struct perf_event *event)
static void kvmppc_pmu_del(struct perf_event *event, int flags)
{
+ /* Do nothing */
}
static int kvmppc_pmu_add(struct perf_event *event, int flags)
{
+ if (flags & PERF_EF_START)
+ return kvmppc_pmu_event_update(event);
return 0;
}
static void kvmppc_pmu_read(struct perf_event *event)
{
+ kvmppc_pmu_event_update(event);
}
/* Return the size of the needed guest state buffer */
@@ -282,6 +370,8 @@ static struct pmu kvmppc_pmu = {
.read = kvmppc_pmu_read,
.attr_groups = kvmppc_pmu_attr_groups,
.type = -1,
+ .scope = PERF_PMU_SCOPE_SYS_WIDE,
+ .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT,
};
int kvmppc_register_pmu(void)
--
2.47.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters
2024-12-22 14:02 ` [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters Vaibhav Jain
@ 2024-12-30 7:03 ` Gautam Menghani
2025-01-06 11:00 ` Vaibhav Jain
0 siblings, 1 reply; 13+ messages in thread
From: Gautam Menghani @ 2024-12-30 7:03 UTC (permalink / raw)
To: Vaibhav Jain
Cc: linuxppc-dev, kvm, kvm-ppc, linux-doc, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Vaidyanathan Srinivasan, sbhat,
kconsul, amachhiw
On Sun, Dec 22, 2024 at 07:32:29PM +0530, Vaibhav Jain wrote:
> Update kvm-nested APIv2 documentation to include five new
> Guest-State-Elements to fetch the hostwide counters. These counters are
> per L1-Lpar and indicate the amount of Heap/Page-table memory allocated,
> available and Page-table memory reclaimed for all L2-Guests active
> instances
>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
> Documentation/arch/powerpc/kvm-nested.rst | 40 ++++++++++++++++-------
> 1 file changed, 29 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/arch/powerpc/kvm-nested.rst b/Documentation/arch/powerpc/kvm-nested.rst
> index 5defd13cc6c1..c506192f3f98 100644
> --- a/Documentation/arch/powerpc/kvm-nested.rst
> +++ b/Documentation/arch/powerpc/kvm-nested.rst
> @@ -208,13 +208,9 @@ associated values for each ID in the GSB::
> flags:
> Bit 0: getGuestWideState: Request state of the Guest instead
> of an individual VCPU.
> - Bit 1: takeOwnershipOfVcpuState Indicate the L1 is taking
> - over ownership of the VCPU state and that the L0 can free
> - the storage holding the state. The VCPU state will need to
> - be returned to the Hypervisor via H_GUEST_SET_STATE prior
> - to H_GUEST_RUN_VCPU being called for this VCPU. The data
> - returned in the dataBuffer is in a Hypervisor internal
> - format.
> + Bit 1: getHostWideState: Request stats of the Host. This causes
> + the guestId and vcpuId parameters to be ignored and attempting
> + to get the VCPU/Guest state will cause an error.
s/Request stats/Request state
> Bits 2-63: Reserved
> guestId: ID obtained from H_GUEST_CREATE
> vcpuId: ID of the vCPU pass to H_GUEST_CREATE_VCPU
> @@ -402,13 +398,14 @@ GSB element:
>
> The ID in the GSB element specifies what is to be set. This includes
> archtected state like GPRs, VSRs, SPRs, plus also some meta data about
> -the partition like the timebase offset and partition scoped page
> +the partition and like the timebase offset and partition scoped page
> table information.
The statement that is already there looks correct IMO.
>
> +--------+-------+----+--------+----------------------------------+
> -| ID | Size | RW | Thread | Details |
> -| | Bytes | | Guest | |
> -| | | | Scope | |
> +| ID | Size | RW |(H)ost | Details |
> +| | Bytes | |(G)uest | |
> +| | | |(T)hread| |
> +| | | |Scope | |
> +========+=======+====+========+==================================+
> | 0x0000 | | RW | TG | NOP element |
> +--------+-------+----+--------+----------------------------------+
> @@ -434,6 +431,27 @@ table information.
> | | | | |- 0x8 Table size. |
> +--------+-------+----+--------+----------------------------------+
> | 0x0007-| | | | Reserved |
> +| 0x07FF | | | | |
> ++--------+-------+----+--------+----------------------------------+
> +| 0x0800 | 0x08 | R | H | Current usage in bytes of the |
> +| | | | | L0's Guest Management Space |
> ++--------+-------+----+--------+----------------------------------+
> +| 0x0801 | 0x08 | R | H | Max bytes available in the |
> +| | | | | L0's Guest Management Space |
> ++--------+-------+----+--------+----------------------------------+
> +| 0x0802 | 0x08 | R | H | Current usage in bytes of the |
> +| | | | | L0's Guest Page Table Management |
> +| | | | | Space |
> ++--------+-------+----+--------+----------------------------------+
> +| 0x0803 | 0x08 | R | H | Max bytes available in the L0's |
> +| | | | | Guest Page Table Management |
> +| | | | | Space |
> ++--------+-------+----+--------+----------------------------------+
> +| 0x0804 | 0x08 | R | H | Amount of reclaimed L0 Guest's |
> +| | | | | Page Table Management Space due |
> +| | | | | to overcommit |
I think it would be more clear to specify "... Management space for L1
..." in the details of all above entries.
> ++--------+-------+----+--------+----------------------------------+
> +| 0x0805-| | | | Reserved |
> | 0x0BFF | | | | |
> +--------+-------+----+--------+----------------------------------+
> | 0x0C00 | 0x10 | RW | T |Run vCPU Input Buffer: |
> --
Also, the row 2 of this table mentions the 'takeOwnershipOfVcpuState' flag
which is no longer supported. You can remove that as well.
> 2.47.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters
2024-12-22 14:02 ` [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters Vaibhav Jain
@ 2024-12-30 7:31 ` Gautam Menghani
2025-01-06 11:24 ` Vaibhav Jain
0 siblings, 1 reply; 13+ messages in thread
From: Gautam Menghani @ 2024-12-30 7:31 UTC (permalink / raw)
To: Vaibhav Jain
Cc: linuxppc-dev, kvm, kvm-ppc, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, kconsul,
amachhiw
On Sun, Dec 22, 2024 at 07:32:33PM +0530, Vaibhav Jain wrote:
> Implement and setup necessary structures to send a prepolulated
> Guest-State-Buffer(GSB) requesting hostwide counters to L0-PowerVM and have
> the returned GSB holding the values of these counters parsed. This is done
> via existing GSB implementation and with the newly added support of
> Hostwide elements in GSB.
>
> The request to L0-PowerVM to return Hostwide counters is done using a
> pre-allocated GSB named 'gsb_l0_stats'. To be able to populate this GSB
> with the needed Guest-State-Elements (GSIDs) a instance of 'struct
> kvmppc_gs_msg' named 'gsm_l0_stats' is introduced. The 'gsm_l0_stats' is
> tied to an instance of 'struct kvmppc_gs_msg_ops' named 'gsb_ops_l0_stats'
> which holds various callbacks to be compute the size ( hostwide_get_size()
> ), populate the GSB ( hostwide_fill_info() ) and
> refresh ( hostwide_refresh_info() ) the contents of
> 'l0_stats' that holds the Hostwide counters returned from L0-PowerVM.
>
> To protect these structures from simultaneous access a spinlock
> 'lock_l0_stats' has been introduced. The allocation and initialization of
> the above structures is done in newly introduced kvmppc_init_hostwide() and
> similarly the cleanup is performed in newly introduced
> kvmppc_cleanup_hostwide().
>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
> arch/powerpc/kvm/book3s_hv_pmu.c | 189 +++++++++++++++++++++++++++++++
> 1 file changed, 189 insertions(+)
>
> diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
> index e72542d5e750..f7fd5190ecf7 100644
> --- a/arch/powerpc/kvm/book3s_hv_pmu.c
> +++ b/arch/powerpc/kvm/book3s_hv_pmu.c
> @@ -27,10 +27,31 @@
> #include <asm/plpar_wrappers.h>
> #include <asm/firmware.h>
>
> +#include "asm/guest-state-buffer.h"
> +
> enum kvmppc_pmu_eventid {
> KVMPPC_EVENT_MAX,
> };
>
> +#define KVMPPC_PMU_EVENT_ATTR(_name, _id) \
> + PMU_EVENT_ATTR_ID(_name, power_events_sysfs_show, _id)
> +
> +/* Holds the hostwide stats */
> +static struct kvmppc_hostwide_stats {
> + u64 guest_heap;
> + u64 guest_heap_max;
> + u64 guest_pgtable_size;
> + u64 guest_pgtable_size_max;
> + u64 guest_pgtable_reclaim;
> +} l0_stats;
> +
> +/* Protect access to l0_stats */
> +static DEFINE_SPINLOCK(lock_l0_stats);
> +
> +/* GSB related structs needed to talk to L0 */
> +static struct kvmppc_gs_msg *gsm_l0_stats;
> +static struct kvmppc_gs_buff *gsb_l0_stats;
> +
> static struct attribute *kvmppc_pmu_events_attr[] = {
> NULL,
> };
> @@ -90,6 +111,167 @@ static void kvmppc_pmu_read(struct perf_event *event)
> {
> }
>
> +/* Return the size of the needed guest state buffer */
> +static size_t hostwide_get_size(struct kvmppc_gs_msg *gsm)
> +
> +{
> + size_t size = 0;
> + const u16 ids[] = {
> + KVMPPC_GSID_L0_GUEST_HEAP,
> + KVMPPC_GSID_L0_GUEST_HEAP_MAX,
> + KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
> + KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
> + KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM
> + };
> +
> + for (int i = 0; i < ARRAY_SIZE(ids); i++)
> + size += kvmppc_gse_total_size(kvmppc_gsid_size(ids[i]));
> + return size;
> +}
> +
> +/* Populate the request guest state buffer */
> +static int hostwide_fill_info(struct kvmppc_gs_buff *gsb,
> + struct kvmppc_gs_msg *gsm)
> +{
> + struct kvmppc_hostwide_stats *stats = gsm->data;
> +
> + /*
> + * It doesn't matter what values are put into request buffer as
> + * they are going to be overwritten anyways. But for the sake of
> + * testcode and symmetry contents of existing stats are put
> + * populated into the request guest state buffer.
> + */
> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP))
> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP,
> + stats->guest_heap);
> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP_MAX))
> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP_MAX,
> + stats->guest_heap_max);
> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE))
> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
> + stats->guest_pgtable_size);
> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX))
> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
> + stats->guest_pgtable_size_max);
> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM))
> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM,
> + stats->guest_pgtable_reclaim);
> +
> + return 0;
> +}
kvmppc_gse_put_u64() can return an error. I think we can handle it just
like gs_msg_ops_vcpu_fill_info()
> +
> +/* Parse and update the host wide stats from returned gsb */
> +static int hostwide_refresh_info(struct kvmppc_gs_msg *gsm,
> + struct kvmppc_gs_buff *gsb)
> +{
> + struct kvmppc_gs_parser gsp = { 0 };
> + struct kvmppc_hostwide_stats *stats = gsm->data;
> + struct kvmppc_gs_elem *gse;
> + int rc;
> +
> + rc = kvmppc_gse_parse(&gsp, gsb);
> + if (rc < 0)
> + return rc;
> +
> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP);
> + if (gse)
> + stats->guest_heap = kvmppc_gse_get_u64(gse);
> +
> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
> + if (gse)
> + stats->guest_heap_max = kvmppc_gse_get_u64(gse);
> +
> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
> + if (gse)
> + stats->guest_pgtable_size = kvmppc_gse_get_u64(gse);
> +
> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
> + if (gse)
> + stats->guest_pgtable_size_max = kvmppc_gse_get_u64(gse);
> +
> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
> + if (gse)
> + stats->guest_pgtable_reclaim = kvmppc_gse_get_u64(gse);
> +
> + return 0;
> +}
> +
> +/* gsb-message ops for setting up/parsing */
> +static struct kvmppc_gs_msg_ops gsb_ops_l0_stats = {
> + .get_size = hostwide_get_size,
> + .fill_info = hostwide_fill_info,
> + .refresh_info = hostwide_refresh_info,
> +};
> +
> +static int kvmppc_init_hostwide(void)
> +{
> + int rc = 0;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&lock_l0_stats, flags);
> +
> + /* already registered ? */
> + if (gsm_l0_stats) {
> + rc = 0;
> + goto out;
> + }
> +
> + /* setup the Guest state message/buffer to talk to L0 */
> + gsm_l0_stats = kvmppc_gsm_new(&gsb_ops_l0_stats, &l0_stats,
> + GSM_SEND, GFP_KERNEL);
> + if (!gsm_l0_stats) {
> + rc = -ENOMEM;
> + goto out;
> + }
> +
> + /* Populate the Idents */
> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_HEAP);
> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
> +
> + /* allocate GSB. Guest/Vcpu Id is ignored */
> + gsb_l0_stats = kvmppc_gsb_new(kvmppc_gsm_size(gsm_l0_stats), 0, 0,
> + GFP_KERNEL);
> + if (!gsb_l0_stats) {
> + rc = -ENOMEM;
> + goto out;
> + }
> +
> + /* ask the ops to fill in the info */
> + rc = kvmppc_gsm_fill_info(gsm_l0_stats, gsb_l0_stats);
> + if (rc)
> + goto out;
> +out:
> + if (rc) {
> + if (gsm_l0_stats)
> + kvmppc_gsm_free(gsm_l0_stats);
> + if (gsb_l0_stats)
> + kvmppc_gsb_free(gsb_l0_stats);
> + gsm_l0_stats = NULL;
> + gsb_l0_stats = NULL;
> + }
> + spin_unlock_irqrestore(&lock_l0_stats, flags);
> + return rc;
> +}
The error handling can probably be simplified to avoid multiple ifs:
<snip>
/* allocate GSB. Guest/Vcpu Id is ignored */
gsb_l0_stats = kvmppc_gsb_new(kvmppc_gsm_size(gsm_l0_stats), 0, 0,
GFP_KERNEL);
if (!gsb_l0_stats) {
rc = -ENOMEM;
goto err_gsm;
}
/* ask the ops to fill in the info */
rc = kvmppc_gsm_fill_info(gsm_l0_stats, gsb_l0_stats);
if (!rc)
goto out;
err_gsb:
kvmppc_gsb_free(gsb_l0_stats);
gsb_l0_stats = NULL;
err_gsm:
kvmppc_gsm_free(gsm_l0_stats);
gsm_l0_stats = NULL;
out:
spin_unlock_irqrestore(&lock_l0_stats, flags);
return rc;
}
> +
> +static void kvmppc_cleanup_hostwide(void)
> +{
> + unsigned long flags;
> +
> + spin_lock_irqsave(&lock_l0_stats, flags);
> +
> + if (gsm_l0_stats)
> + kvmppc_gsm_free(gsm_l0_stats);
> + if (gsb_l0_stats)
> + kvmppc_gsb_free(gsb_l0_stats);
> + gsm_l0_stats = NULL;
> + gsb_l0_stats = NULL;
> +
> + spin_unlock_irqrestore(&lock_l0_stats, flags);
> +}
> +
> /* L1 wide counters PMU */
> static struct pmu kvmppc_pmu = {
> .task_ctx_nr = perf_sw_context,
> @@ -108,6 +290,10 @@ int kvmppc_register_pmu(void)
>
> /* only support events for nestedv2 right now */
> if (kvmhv_is_nestedv2()) {
> + rc = kvmppc_init_hostwide();
> + if (rc)
> + goto out;
> +
> /* Setup done now register the PMU */
> pr_info("Registering kvm-hv pmu");
>
> @@ -117,6 +303,7 @@ int kvmppc_register_pmu(void)
> -1) : 0;
> }
>
> +out:
> return rc;
> }
> EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
> @@ -124,6 +311,8 @@ EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
> void kvmppc_unregister_pmu(void)
> {
> if (kvmhv_is_nestedv2()) {
> + kvmppc_cleanup_hostwide();
> +
> if (kvmppc_pmu.type != -1)
> perf_pmu_unregister(&kvmppc_pmu);
>
> --
> 2.47.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU
2024-12-22 14:02 ` [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU Vaibhav Jain
@ 2024-12-30 9:28 ` Gautam Menghani
2025-01-06 11:01 ` Vaibhav Jain
0 siblings, 1 reply; 13+ messages in thread
From: Gautam Menghani @ 2024-12-30 9:28 UTC (permalink / raw)
To: Vaibhav Jain
Cc: linuxppc-dev, kvm, kvm-ppc, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, kconsul,
amachhiw
On Sun, Dec 22, 2024 at 07:32:32PM +0530, Vaibhav Jain wrote:
> Introduce a new PMU named 'kvm-hv' to report Book3s kvm-hv specific
> performance counters. This will expose KVM-HV specific performance
> attributes to user-space via kernel's PMU infrastructure and would enable
> users to monitor active kvm-hv based guests.
>
> The patch creates necessary scaffolding to for the new PMU callbacks and
> introduces two new exports kvmppc_{,un}register_pmu() that are called from
> kvm-hv init and exit function to perform initialize and cleanup for the
> 'kvm-hv' PMU. The patch doesn't introduce any perf-events yet, which will
> be introduced in later patches
>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
> arch/powerpc/include/asm/kvm_book3s.h | 12 +++
> arch/powerpc/kvm/Makefile | 6 ++
> arch/powerpc/kvm/book3s_hv.c | 7 ++
> arch/powerpc/kvm/book3s_hv_pmu.c | 133 ++++++++++++++++++++++++++
> 4 files changed, 158 insertions(+)
> create mode 100644 arch/powerpc/kvm/book3s_hv_pmu.c
>
> diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h
> index e1ff291ba891..cf91a1493159 100644
> --- a/arch/powerpc/include/asm/kvm_book3s.h
> +++ b/arch/powerpc/include/asm/kvm_book3s.h
> @@ -334,6 +334,9 @@ static inline bool kvmhv_is_nestedv1(void)
> return !static_branch_likely(&__kvmhv_is_nestedv2);
> }
>
> +int kvmppc_register_pmu(void);
> +void kvmppc_unregister_pmu(void);
> +
> #else
>
> static inline bool kvmhv_is_nestedv2(void)
> @@ -346,6 +349,15 @@ static inline bool kvmhv_is_nestedv1(void)
> return false;
> }
>
> +static int kvmppc_register_pmu(void)
> +{
> + return 0;
> +}
> +
> +static void kvmppc_unregister_pmu(void)
> +{
> +}
> +
> #endif
>
> int __kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu, struct pt_regs *regs);
> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
> index 4bd9d1230869..094c3916d9d0 100644
> --- a/arch/powerpc/kvm/Makefile
> +++ b/arch/powerpc/kvm/Makefile
> @@ -92,6 +92,12 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
> $(kvm-book3s_64-builtin-tm-objs-y) \
> $(kvm-book3s_64-builtin-xics-objs-y)
>
> +# enable kvm_hv perf events
> +ifdef CONFIG_HAVE_PERF_EVENTS
> +kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
> + book3s_hv_pmu.o
> +endif
> +
> obj-$(CONFIG_GUEST_STATE_BUFFER_TEST) += test-guest-state-buffer.o
> endif
>
> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
> index 25429905ae90..83bcce2fb557 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -6662,6 +6662,12 @@ static int kvmppc_book3s_init_hv(void)
> return r;
> }
>
> + r = kvmppc_register_pmu();
> + if (r) {
> + pr_err("KVM-HV: Unable to register PMUs %d\n", r);
> + goto err;
> + }
> +
> kvm_ops_hv.owner = THIS_MODULE;
> kvmppc_hv_ops = &kvm_ops_hv;
>
> @@ -6676,6 +6682,7 @@ static int kvmppc_book3s_init_hv(void)
>
> static void kvmppc_book3s_exit_hv(void)
> {
> + kvmppc_unregister_pmu();
> kvmppc_uvmem_free();
> kvmppc_free_host_rm_ops();
> if (kvmppc_radix_possible())
> diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
> new file mode 100644
> index 000000000000..e72542d5e750
> --- /dev/null
> +++ b/arch/powerpc/kvm/book3s_hv_pmu.c
> @@ -0,0 +1,133 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Description: PMUs specific to running nested KVM-HV guests
> + * on Book3S processors (specifically POWER9 and later).
> + */
> +
> +#define pr_fmt(fmt) "kvmppc-pmu: " fmt
> +
> +#include "asm-generic/local64.h"
> +#include <linux/kernel.h>
> +#include <linux/errno.h>
> +#include <linux/ratelimit.h>
> +#include <linux/kvm_host.h>
> +#include <linux/gfp_types.h>
> +#include <linux/pgtable.h>
> +#include <linux/perf_event.h>
> +#include <linux/spinlock_types.h>
> +#include <linux/spinlock.h>
> +
> +#include <asm/types.h>
> +#include <asm/kvm_ppc.h>
> +#include <asm/kvm_book3s.h>
> +#include <asm/mmu.h>
> +#include <asm/pgalloc.h>
> +#include <asm/pte-walk.h>
> +#include <asm/reg.h>
> +#include <asm/plpar_wrappers.h>
> +#include <asm/firmware.h>
> +
> +enum kvmppc_pmu_eventid {
> + KVMPPC_EVENT_MAX,
> +};
> +
> +static struct attribute *kvmppc_pmu_events_attr[] = {
> + NULL,
> +};
> +
> +static const struct attribute_group kvmppc_pmu_events_group = {
> + .name = "events",
> + .attrs = kvmppc_pmu_events_attr,
> +};
> +
> +PMU_FORMAT_ATTR(event, "config:0");
> +static struct attribute *kvmppc_pmu_format_attr[] = {
> + &format_attr_event.attr,
> + NULL,
> +};
> +
> +static struct attribute_group kvmppc_pmu_format_group = {
> + .name = "format",
> + .attrs = kvmppc_pmu_format_attr,
> +};
> +
> +static const struct attribute_group *kvmppc_pmu_attr_groups[] = {
> + &kvmppc_pmu_events_group,
> + &kvmppc_pmu_format_group,
> + NULL,
> +};
> +
> +static int kvmppc_pmu_event_init(struct perf_event *event)
> +{
> + unsigned int config = event->attr.config;
> +
> + pr_debug("%s: Event(%p) id=%llu cpu=%x on_cpu=%x config=%u",
> + __func__, event, event->id, event->cpu,
> + event->oncpu, config);
> +
> + if (event->attr.type != event->pmu->type)
> + return -ENOENT;
> +
> + if (config >= KVMPPC_EVENT_MAX)
> + return -EINVAL;
> +
> + local64_set(&event->hw.prev_count, 0);
> + local64_set(&event->count, 0);
> +
> + return 0;
> +}
> +
> +static void kvmppc_pmu_del(struct perf_event *event, int flags)
> +{
> +}
> +
> +static int kvmppc_pmu_add(struct perf_event *event, int flags)
> +{
> + return 0;
> +}
> +
> +static void kvmppc_pmu_read(struct perf_event *event)
> +{
> +}
> +
> +/* L1 wide counters PMU */
> +static struct pmu kvmppc_pmu = {
> + .task_ctx_nr = perf_sw_context,
> + .name = "kvm-hv",
> + .event_init = kvmppc_pmu_event_init,
> + .add = kvmppc_pmu_add,
> + .del = kvmppc_pmu_del,
> + .read = kvmppc_pmu_read,
> + .attr_groups = kvmppc_pmu_attr_groups,
> + .type = -1,
> +};
> +
> +int kvmppc_register_pmu(void)
> +{
> + int rc = -EOPNOTSUPP;
> +
> + /* only support events for nestedv2 right now */
> + if (kvmhv_is_nestedv2()) {
> + /* Setup done now register the PMU */
> + pr_info("Registering kvm-hv pmu");
> +
> + /* Register only if we arent already registyered */
> + rc = (kvmppc_pmu.type == -1) ?
> + perf_pmu_register(&kvmppc_pmu, kvmppc_pmu.name,
> + -1) : 0;
> + }
> +
> + return rc;
> +}
> +EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
This logic breaks on bare metal. kvmppc_register_pmu() returns
-EOPNOTSUPP because of which kvm_hv cannot be loaded.
> +
> +void kvmppc_unregister_pmu(void)
> +{
> + if (kvmhv_is_nestedv2()) {
> + if (kvmppc_pmu.type != -1)
> + perf_pmu_unregister(&kvmppc_pmu);
> +
> + pr_info("kvmhv_pmu unregistered.\n");
> + }
> +}
> +EXPORT_SYMBOL_GPL(kvmppc_unregister_pmu);
> --
> 2.47.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters
2024-12-30 7:03 ` Gautam Menghani
@ 2025-01-06 11:00 ` Vaibhav Jain
0 siblings, 0 replies; 13+ messages in thread
From: Vaibhav Jain @ 2025-01-06 11:00 UTC (permalink / raw)
To: Gautam Menghani
Cc: linuxppc-dev, kvm, kvm-ppc, linux-doc, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Vaidyanathan Srinivasan, sbhat,
kconsul, amachhiw
Hi Gautam,
Thanks for reviewing this patch. My responses to your review comments
inline below:
Gautam Menghani <gautam@linux.ibm.com> writes:
> On Sun, Dec 22, 2024 at 07:32:29PM +0530, Vaibhav Jain wrote:
>> Update kvm-nested APIv2 documentation to include five new
>> Guest-State-Elements to fetch the hostwide counters. These counters are
>> per L1-Lpar and indicate the amount of Heap/Page-table memory allocated,
>> available and Page-table memory reclaimed for all L2-Guests active
>> instances
>>
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
>> ---
>> Documentation/arch/powerpc/kvm-nested.rst | 40 ++++++++++++++++-------
>> 1 file changed, 29 insertions(+), 11 deletions(-)
>>
>> diff --git a/Documentation/arch/powerpc/kvm-nested.rst b/Documentation/arch/powerpc/kvm-nested.rst
>> index 5defd13cc6c1..c506192f3f98 100644
>> --- a/Documentation/arch/powerpc/kvm-nested.rst
>> +++ b/Documentation/arch/powerpc/kvm-nested.rst
>> @@ -208,13 +208,9 @@ associated values for each ID in the GSB::
>> flags:
>> Bit 0: getGuestWideState: Request state of the Guest instead
>> of an individual VCPU.
>> - Bit 1: takeOwnershipOfVcpuState Indicate the L1 is taking
>> - over ownership of the VCPU state and that the L0 can free
>> - the storage holding the state. The VCPU state will need to
>> - be returned to the Hypervisor via H_GUEST_SET_STATE prior
>> - to H_GUEST_RUN_VCPU being called for this VCPU. The data
>> - returned in the dataBuffer is in a Hypervisor internal
>> - format.
>> + Bit 1: getHostWideState: Request stats of the Host. This causes
>> + the guestId and vcpuId parameters to be ignored and attempting
>> + to get the VCPU/Guest state will cause an error.
>
> s/Request stats/Request state
>
Its is 'Request stats' are this flag currently return Hostwide stat
counters.
>> Bits 2-63: Reserved
>> guestId: ID obtained from H_GUEST_CREATE
>> vcpuId: ID of the vCPU pass to H_GUEST_CREATE_VCPU
>> @@ -402,13 +398,14 @@ GSB element:
>>
>> The ID in the GSB element specifies what is to be set. This includes
>> archtected state like GPRs, VSRs, SPRs, plus also some meta data about
>> -the partition like the timebase offset and partition scoped page
>> +the partition and like the timebase offset and partition scoped page
>> table information.
>
> The statement that is already there looks correct IMO.
>
Right. I will update it to reflect L1-lpar stats in v2.
>>
>> +--------+-------+----+--------+----------------------------------+
>> -| ID | Size | RW | Thread | Details |
>> -| | Bytes | | Guest | |
>> -| | | | Scope | |
>> +| ID | Size | RW |(H)ost | Details |
>> +| | Bytes | |(G)uest | |
>> +| | | |(T)hread| |
>> +| | | |Scope | |
>> +========+=======+====+========+==================================+
>> | 0x0000 | | RW | TG | NOP element |
>> +--------+-------+----+--------+----------------------------------+
>> @@ -434,6 +431,27 @@ table information.
>> | | | | |- 0x8 Table size. |
>> +--------+-------+----+--------+----------------------------------+
>> | 0x0007-| | | | Reserved |
>> +| 0x07FF | | | | |
>> ++--------+-------+----+--------+----------------------------------+
>> +| 0x0800 | 0x08 | R | H | Current usage in bytes of the |
>> +| | | | | L0's Guest Management Space |
>> ++--------+-------+----+--------+----------------------------------+
>> +| 0x0801 | 0x08 | R | H | Max bytes available in the |
>> +| | | | | L0's Guest Management Space |
>> ++--------+-------+----+--------+----------------------------------+
>> +| 0x0802 | 0x08 | R | H | Current usage in bytes of the |
>> +| | | | | L0's Guest Page Table Management |
>> +| | | | | Space |
>> ++--------+-------+----+--------+----------------------------------+
>> +| 0x0803 | 0x08 | R | H | Max bytes available in the L0's |
>> +| | | | | Guest Page Table Management |
>> +| | | | | Space |
>> ++--------+-------+----+--------+----------------------------------+
>> +| 0x0804 | 0x08 | R | H | Amount of reclaimed L0 Guest's |
>> +| | | | | Page Table Management Space due |
>> +| | | | | to overcommit |
>
> I think it would be more clear to specify "... Management space for L1
> ..." in the details of all above entries.
>
Agree, Updated that in v2.
>> ++--------+-------+----+--------+----------------------------------+
>> +| 0x0805-| | | | Reserved |
>> | 0x0BFF | | | | |
>> +--------+-------+----+--------+----------------------------------+
>> | 0x0C00 | 0x10 | RW | T |Run vCPU Input Buffer: |
>> --
>
> Also, the row 2 of this table mentions the 'takeOwnershipOfVcpuState' flag
> which is no longer supported. You can remove that as well.
>
Updating this document for 'takeOwnershipOfVcpuState' will be done as a
separate patch and not part of this patch series
>> 2.47.1
>>
>
--
Cheers
~ Vaibhav
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU
2024-12-30 9:28 ` Gautam Menghani
@ 2025-01-06 11:01 ` Vaibhav Jain
0 siblings, 0 replies; 13+ messages in thread
From: Vaibhav Jain @ 2025-01-06 11:01 UTC (permalink / raw)
To: Gautam Menghani
Cc: linuxppc-dev, kvm, kvm-ppc, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, kconsul,
amachhiw
Hi Gautam,
Thanks for reviewing this patch. My responses to your review comments
inline below:
Gautam Menghani <gautam@linux.ibm.com> writes:
> On Sun, Dec 22, 2024 at 07:32:32PM +0530, Vaibhav Jain wrote:
>> Introduce a new PMU named 'kvm-hv' to report Book3s kvm-hv specific
>> performance counters. This will expose KVM-HV specific performance
>> attributes to user-space via kernel's PMU infrastructure and would enable
>> users to monitor active kvm-hv based guests.
>>
>> The patch creates necessary scaffolding to for the new PMU callbacks and
>> introduces two new exports kvmppc_{,un}register_pmu() that are called from
>> kvm-hv init and exit function to perform initialize and cleanup for the
>> 'kvm-hv' PMU. The patch doesn't introduce any perf-events yet, which will
>> be introduced in later patches
>>
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
>> ---
>> arch/powerpc/include/asm/kvm_book3s.h | 12 +++
>> arch/powerpc/kvm/Makefile | 6 ++
>> arch/powerpc/kvm/book3s_hv.c | 7 ++
>> arch/powerpc/kvm/book3s_hv_pmu.c | 133 ++++++++++++++++++++++++++
>> 4 files changed, 158 insertions(+)
>> create mode 100644 arch/powerpc/kvm/book3s_hv_pmu.c
>>
>> diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h
>> index e1ff291ba891..cf91a1493159 100644
>> --- a/arch/powerpc/include/asm/kvm_book3s.h
>> +++ b/arch/powerpc/include/asm/kvm_book3s.h
>> @@ -334,6 +334,9 @@ static inline bool kvmhv_is_nestedv1(void)
>> return !static_branch_likely(&__kvmhv_is_nestedv2);
>> }
>>
>> +int kvmppc_register_pmu(void);
>> +void kvmppc_unregister_pmu(void);
>> +
>> #else
>>
>> static inline bool kvmhv_is_nestedv2(void)
>> @@ -346,6 +349,15 @@ static inline bool kvmhv_is_nestedv1(void)
>> return false;
>> }
>>
>> +static int kvmppc_register_pmu(void)
>> +{
>> + return 0;
>> +}
>> +
>> +static void kvmppc_unregister_pmu(void)
>> +{
>> +}
>> +
>> #endif
>>
>> int __kvmhv_nestedv2_reload_ptregs(struct kvm_vcpu *vcpu, struct pt_regs *regs);
>> diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile
>> index 4bd9d1230869..094c3916d9d0 100644
>> --- a/arch/powerpc/kvm/Makefile
>> +++ b/arch/powerpc/kvm/Makefile
>> @@ -92,6 +92,12 @@ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
>> $(kvm-book3s_64-builtin-tm-objs-y) \
>> $(kvm-book3s_64-builtin-xics-objs-y)
>>
>> +# enable kvm_hv perf events
>> +ifdef CONFIG_HAVE_PERF_EVENTS
>> +kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \
>> + book3s_hv_pmu.o
>> +endif
>> +
>> obj-$(CONFIG_GUEST_STATE_BUFFER_TEST) += test-guest-state-buffer.o
>> endif
>>
>> diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
>> index 25429905ae90..83bcce2fb557 100644
>> --- a/arch/powerpc/kvm/book3s_hv.c
>> +++ b/arch/powerpc/kvm/book3s_hv.c
>> @@ -6662,6 +6662,12 @@ static int kvmppc_book3s_init_hv(void)
>> return r;
>> }
>>
>> + r = kvmppc_register_pmu();
>> + if (r) {
>> + pr_err("KVM-HV: Unable to register PMUs %d\n", r);
>> + goto err;
>> + }
>> +
>> kvm_ops_hv.owner = THIS_MODULE;
>> kvmppc_hv_ops = &kvm_ops_hv;
>>
>> @@ -6676,6 +6682,7 @@ static int kvmppc_book3s_init_hv(void)
>>
>> static void kvmppc_book3s_exit_hv(void)
>> {
>> + kvmppc_unregister_pmu();
>> kvmppc_uvmem_free();
>> kvmppc_free_host_rm_ops();
>> if (kvmppc_radix_possible())
>> diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
>> new file mode 100644
>> index 000000000000..e72542d5e750
>> --- /dev/null
>> +++ b/arch/powerpc/kvm/book3s_hv_pmu.c
>> @@ -0,0 +1,133 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Description: PMUs specific to running nested KVM-HV guests
>> + * on Book3S processors (specifically POWER9 and later).
>> + */
>> +
>> +#define pr_fmt(fmt) "kvmppc-pmu: " fmt
>> +
>> +#include "asm-generic/local64.h"
>> +#include <linux/kernel.h>
>> +#include <linux/errno.h>
>> +#include <linux/ratelimit.h>
>> +#include <linux/kvm_host.h>
>> +#include <linux/gfp_types.h>
>> +#include <linux/pgtable.h>
>> +#include <linux/perf_event.h>
>> +#include <linux/spinlock_types.h>
>> +#include <linux/spinlock.h>
>> +
>> +#include <asm/types.h>
>> +#include <asm/kvm_ppc.h>
>> +#include <asm/kvm_book3s.h>
>> +#include <asm/mmu.h>
>> +#include <asm/pgalloc.h>
>> +#include <asm/pte-walk.h>
>> +#include <asm/reg.h>
>> +#include <asm/plpar_wrappers.h>
>> +#include <asm/firmware.h>
>> +
>> +enum kvmppc_pmu_eventid {
>> + KVMPPC_EVENT_MAX,
>> +};
>> +
>> +static struct attribute *kvmppc_pmu_events_attr[] = {
>> + NULL,
>> +};
>> +
>> +static const struct attribute_group kvmppc_pmu_events_group = {
>> + .name = "events",
>> + .attrs = kvmppc_pmu_events_attr,
>> +};
>> +
>> +PMU_FORMAT_ATTR(event, "config:0");
>> +static struct attribute *kvmppc_pmu_format_attr[] = {
>> + &format_attr_event.attr,
>> + NULL,
>> +};
>> +
>> +static struct attribute_group kvmppc_pmu_format_group = {
>> + .name = "format",
>> + .attrs = kvmppc_pmu_format_attr,
>> +};
>> +
>> +static const struct attribute_group *kvmppc_pmu_attr_groups[] = {
>> + &kvmppc_pmu_events_group,
>> + &kvmppc_pmu_format_group,
>> + NULL,
>> +};
>> +
>> +static int kvmppc_pmu_event_init(struct perf_event *event)
>> +{
>> + unsigned int config = event->attr.config;
>> +
>> + pr_debug("%s: Event(%p) id=%llu cpu=%x on_cpu=%x config=%u",
>> + __func__, event, event->id, event->cpu,
>> + event->oncpu, config);
>> +
>> + if (event->attr.type != event->pmu->type)
>> + return -ENOENT;
>> +
>> + if (config >= KVMPPC_EVENT_MAX)
>> + return -EINVAL;
>> +
>> + local64_set(&event->hw.prev_count, 0);
>> + local64_set(&event->count, 0);
>> +
>> + return 0;
>> +}
>> +
>> +static void kvmppc_pmu_del(struct perf_event *event, int flags)
>> +{
>> +}
>> +
>> +static int kvmppc_pmu_add(struct perf_event *event, int flags)
>> +{
>> + return 0;
>> +}
>> +
>> +static void kvmppc_pmu_read(struct perf_event *event)
>> +{
>> +}
>> +
>> +/* L1 wide counters PMU */
>> +static struct pmu kvmppc_pmu = {
>> + .task_ctx_nr = perf_sw_context,
>> + .name = "kvm-hv",
>> + .event_init = kvmppc_pmu_event_init,
>> + .add = kvmppc_pmu_add,
>> + .del = kvmppc_pmu_del,
>> + .read = kvmppc_pmu_read,
>> + .attr_groups = kvmppc_pmu_attr_groups,
>> + .type = -1,
>> +};
>> +
>> +int kvmppc_register_pmu(void)
>> +{
>> + int rc = -EOPNOTSUPP;
>> +
>> + /* only support events for nestedv2 right now */
>> + if (kvmhv_is_nestedv2()) {
>> + /* Setup done now register the PMU */
>> + pr_info("Registering kvm-hv pmu");
>> +
>> + /* Register only if we arent already registyered */
>> + rc = (kvmppc_pmu.type == -1) ?
>> + perf_pmu_register(&kvmppc_pmu, kvmppc_pmu.name,
>> + -1) : 0;
>> + }
>> +
>> + return rc;
>> +}
>> +EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
>
> This logic breaks on bare metal. kvmppc_register_pmu() returns
> -EOPNOTSUPP because of which kvm_hv cannot be loaded.
>
Good catch. Fixing this in v2
>> +
>> +void kvmppc_unregister_pmu(void)
>> +{
>> + if (kvmhv_is_nestedv2()) {
>> + if (kvmppc_pmu.type != -1)
>> + perf_pmu_unregister(&kvmppc_pmu);
>> +
>> + pr_info("kvmhv_pmu unregistered.\n");
>> + }
>> +}
>> +EXPORT_SYMBOL_GPL(kvmppc_unregister_pmu);
>> --
>> 2.47.1
>>
--
Cheers
~ Vaibhav
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters
2024-12-30 7:31 ` Gautam Menghani
@ 2025-01-06 11:24 ` Vaibhav Jain
0 siblings, 0 replies; 13+ messages in thread
From: Vaibhav Jain @ 2025-01-06 11:24 UTC (permalink / raw)
To: Gautam Menghani
Cc: linuxppc-dev, kvm, kvm-ppc, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Vaidyanathan Srinivasan, sbhat, kconsul,
amachhiw
Hi Gautam,
Thanks for reviewing this patch. My responses to your review comments
inline below:
Gautam Menghani <gautam@linux.ibm.com> writes:
> On Sun, Dec 22, 2024 at 07:32:33PM +0530, Vaibhav Jain wrote:
>> Implement and setup necessary structures to send a prepolulated
>> Guest-State-Buffer(GSB) requesting hostwide counters to L0-PowerVM and have
>> the returned GSB holding the values of these counters parsed. This is done
>> via existing GSB implementation and with the newly added support of
>> Hostwide elements in GSB.
>>
>> The request to L0-PowerVM to return Hostwide counters is done using a
>> pre-allocated GSB named 'gsb_l0_stats'. To be able to populate this GSB
>> with the needed Guest-State-Elements (GSIDs) a instance of 'struct
>> kvmppc_gs_msg' named 'gsm_l0_stats' is introduced. The 'gsm_l0_stats' is
>> tied to an instance of 'struct kvmppc_gs_msg_ops' named 'gsb_ops_l0_stats'
>> which holds various callbacks to be compute the size ( hostwide_get_size()
>> ), populate the GSB ( hostwide_fill_info() ) and
>> refresh ( hostwide_refresh_info() ) the contents of
>> 'l0_stats' that holds the Hostwide counters returned from L0-PowerVM.
>>
>> To protect these structures from simultaneous access a spinlock
>> 'lock_l0_stats' has been introduced. The allocation and initialization of
>> the above structures is done in newly introduced kvmppc_init_hostwide() and
>> similarly the cleanup is performed in newly introduced
>> kvmppc_cleanup_hostwide().
>>
>> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
>> ---
>> arch/powerpc/kvm/book3s_hv_pmu.c | 189 +++++++++++++++++++++++++++++++
>> 1 file changed, 189 insertions(+)
>>
>> diff --git a/arch/powerpc/kvm/book3s_hv_pmu.c b/arch/powerpc/kvm/book3s_hv_pmu.c
>> index e72542d5e750..f7fd5190ecf7 100644
>> --- a/arch/powerpc/kvm/book3s_hv_pmu.c
>> +++ b/arch/powerpc/kvm/book3s_hv_pmu.c
>> @@ -27,10 +27,31 @@
>> #include <asm/plpar_wrappers.h>
>> #include <asm/firmware.h>
>>
>> +#include "asm/guest-state-buffer.h"
>> +
>> enum kvmppc_pmu_eventid {
>> KVMPPC_EVENT_MAX,
>> };
>>
>> +#define KVMPPC_PMU_EVENT_ATTR(_name, _id) \
>> + PMU_EVENT_ATTR_ID(_name, power_events_sysfs_show, _id)
>> +
>> +/* Holds the hostwide stats */
>> +static struct kvmppc_hostwide_stats {
>> + u64 guest_heap;
>> + u64 guest_heap_max;
>> + u64 guest_pgtable_size;
>> + u64 guest_pgtable_size_max;
>> + u64 guest_pgtable_reclaim;
>> +} l0_stats;
>> +
>> +/* Protect access to l0_stats */
>> +static DEFINE_SPINLOCK(lock_l0_stats);
>> +
>> +/* GSB related structs needed to talk to L0 */
>> +static struct kvmppc_gs_msg *gsm_l0_stats;
>> +static struct kvmppc_gs_buff *gsb_l0_stats;
>> +
>> static struct attribute *kvmppc_pmu_events_attr[] = {
>> NULL,
>> };
>> @@ -90,6 +111,167 @@ static void kvmppc_pmu_read(struct perf_event *event)
>> {
>> }
>>
>> +/* Return the size of the needed guest state buffer */
>> +static size_t hostwide_get_size(struct kvmppc_gs_msg *gsm)
>> +
>> +{
>> + size_t size = 0;
>> + const u16 ids[] = {
>> + KVMPPC_GSID_L0_GUEST_HEAP,
>> + KVMPPC_GSID_L0_GUEST_HEAP_MAX,
>> + KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
>> + KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
>> + KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM
>> + };
>> +
>> + for (int i = 0; i < ARRAY_SIZE(ids); i++)
>> + size += kvmppc_gse_total_size(kvmppc_gsid_size(ids[i]));
>> + return size;
>> +}
>> +
>> +/* Populate the request guest state buffer */
>> +static int hostwide_fill_info(struct kvmppc_gs_buff *gsb,
>> + struct kvmppc_gs_msg *gsm)
>> +{
>> + struct kvmppc_hostwide_stats *stats = gsm->data;
>> +
>> + /*
>> + * It doesn't matter what values are put into request buffer as
>> + * they are going to be overwritten anyways. But for the sake of
>> + * testcode and symmetry contents of existing stats are put
>> + * populated into the request guest state buffer.
>> + */
>> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP))
>> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP,
>> + stats->guest_heap);
>> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_HEAP_MAX))
>> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_HEAP_MAX,
>> + stats->guest_heap_max);
>> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE))
>> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE,
>> + stats->guest_pgtable_size);
>> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX))
>> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX,
>> + stats->guest_pgtable_size_max);
>> + if (kvmppc_gsm_includes(gsm, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM))
>> + kvmppc_gse_put_u64(gsb, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM,
>> + stats->guest_pgtable_reclaim);
>> +
>> + return 0;
>> +}
>
> kvmppc_gse_put_u64() can return an error. I think we can handle it just
> like gs_msg_ops_vcpu_fill_info()
>
Good suggestion. Will incorporate that in v2.
>> +
>> +/* Parse and update the host wide stats from returned gsb */
>> +static int hostwide_refresh_info(struct kvmppc_gs_msg *gsm,
>> + struct kvmppc_gs_buff *gsb)
>> +{
>> + struct kvmppc_gs_parser gsp = { 0 };
>> + struct kvmppc_hostwide_stats *stats = gsm->data;
>> + struct kvmppc_gs_elem *gse;
>> + int rc;
>> +
>> + rc = kvmppc_gse_parse(&gsp, gsb);
>> + if (rc < 0)
>> + return rc;
>> +
>> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP);
>> + if (gse)
>> + stats->guest_heap = kvmppc_gse_get_u64(gse);
>> +
>> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
>> + if (gse)
>> + stats->guest_heap_max = kvmppc_gse_get_u64(gse);
>> +
>> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
>> + if (gse)
>> + stats->guest_pgtable_size = kvmppc_gse_get_u64(gse);
>> +
>> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
>> + if (gse)
>> + stats->guest_pgtable_size_max = kvmppc_gse_get_u64(gse);
>> +
>> + gse = kvmppc_gsp_lookup(&gsp, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
>> + if (gse)
>> + stats->guest_pgtable_reclaim = kvmppc_gse_get_u64(gse);
>> +
>> + return 0;
>> +}
>> +
>> +/* gsb-message ops for setting up/parsing */
>> +static struct kvmppc_gs_msg_ops gsb_ops_l0_stats = {
>> + .get_size = hostwide_get_size,
>> + .fill_info = hostwide_fill_info,
>> + .refresh_info = hostwide_refresh_info,
>> +};
>> +
>> +static int kvmppc_init_hostwide(void)
>> +{
>> + int rc = 0;
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&lock_l0_stats, flags);
>> +
>> + /* already registered ? */
>> + if (gsm_l0_stats) {
>> + rc = 0;
>> + goto out;
>> + }
>> +
>> + /* setup the Guest state message/buffer to talk to L0 */
>> + gsm_l0_stats = kvmppc_gsm_new(&gsb_ops_l0_stats, &l0_stats,
>> + GSM_SEND, GFP_KERNEL);
>> + if (!gsm_l0_stats) {
>> + rc = -ENOMEM;
>> + goto out;
>> + }
>> +
>> + /* Populate the Idents */
>> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_HEAP);
>> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_HEAP_MAX);
>> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE);
>> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_SIZE_MAX);
>> + kvmppc_gsm_include(gsm_l0_stats, KVMPPC_GSID_L0_GUEST_PGTABLE_RECLAIM);
>> +
>> + /* allocate GSB. Guest/Vcpu Id is ignored */
>> + gsb_l0_stats = kvmppc_gsb_new(kvmppc_gsm_size(gsm_l0_stats), 0, 0,
>> + GFP_KERNEL);
>> + if (!gsb_l0_stats) {
>> + rc = -ENOMEM;
>> + goto out;
>> + }
>> +
>> + /* ask the ops to fill in the info */
>> + rc = kvmppc_gsm_fill_info(gsm_l0_stats, gsb_l0_stats);
>> + if (rc)
>> + goto out;
>> +out:
>> + if (rc) {
>> + if (gsm_l0_stats)
>> + kvmppc_gsm_free(gsm_l0_stats);
>> + if (gsb_l0_stats)
>> + kvmppc_gsb_free(gsb_l0_stats);
>> + gsm_l0_stats = NULL;
>> + gsb_l0_stats = NULL;
>> + }
>> + spin_unlock_irqrestore(&lock_l0_stats, flags);
>> + return rc;
>> +}
>
> The error handling can probably be simplified to avoid multiple ifs:
>
> <snip>
>
> /* allocate GSB. Guest/Vcpu Id is ignored */
> gsb_l0_stats = kvmppc_gsb_new(kvmppc_gsm_size(gsm_l0_stats), 0, 0,
> GFP_KERNEL);
> if (!gsb_l0_stats) {
> rc = -ENOMEM;
> goto err_gsm;
> }
>
> /* ask the ops to fill in the info */
> rc = kvmppc_gsm_fill_info(gsm_l0_stats, gsb_l0_stats);
> if (!rc)
> goto out;
>
> err_gsb:
> kvmppc_gsb_free(gsb_l0_stats);
> gsb_l0_stats = NULL;
>
> err_gsm:
> kvmppc_gsm_free(gsm_l0_stats);
> gsm_l0_stats = NULL;
>
> out:
> spin_unlock_irqrestore(&lock_l0_stats, flags);
> return rc;
> }
>
Thats subjective opinion and I tend to prefer less number of goto jump
labels in the function hence the function is implemented the way it is.
>> +
>> +static void kvmppc_cleanup_hostwide(void)
>> +{
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&lock_l0_stats, flags);
>> +
>> + if (gsm_l0_stats)
>> + kvmppc_gsm_free(gsm_l0_stats);
>> + if (gsb_l0_stats)
>> + kvmppc_gsb_free(gsb_l0_stats);
>> + gsm_l0_stats = NULL;
>> + gsb_l0_stats = NULL;
>> +
>> + spin_unlock_irqrestore(&lock_l0_stats, flags);
>> +}
>> +
>> /* L1 wide counters PMU */
>> static struct pmu kvmppc_pmu = {
>> .task_ctx_nr = perf_sw_context,
>> @@ -108,6 +290,10 @@ int kvmppc_register_pmu(void)
>>
>> /* only support events for nestedv2 right now */
>> if (kvmhv_is_nestedv2()) {
>> + rc = kvmppc_init_hostwide();
>> + if (rc)
>> + goto out;
>> +
>> /* Setup done now register the PMU */
>> pr_info("Registering kvm-hv pmu");
>>
>> @@ -117,6 +303,7 @@ int kvmppc_register_pmu(void)
>> -1) : 0;
>> }
>>
>> +out:
>> return rc;
>> }
>> EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
>> @@ -124,6 +311,8 @@ EXPORT_SYMBOL_GPL(kvmppc_register_pmu);
>> void kvmppc_unregister_pmu(void)
>> {
>> if (kvmhv_is_nestedv2()) {
>> + kvmppc_cleanup_hostwide();
>> +
>> if (kvmppc_pmu.type != -1)
>> perf_pmu_unregister(&kvmppc_pmu);
>>
>> --
>> 2.47.1
>>
--
Cheers
~ Vaibhav
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2025-01-06 11:24 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-12-22 14:02 [PATCH 0/6] kvm powerpc/book3s-hv: Expose Hostwide counters as perf-events Vaibhav Jain
2024-12-22 14:02 ` [PATCH 1/6] [DOC] powerpc: Document APIv2 KVM hcall spec for Hostwide counters Vaibhav Jain
2024-12-30 7:03 ` Gautam Menghani
2025-01-06 11:00 ` Vaibhav Jain
2024-12-22 14:02 ` [PATCH 2/6] kvm powerpc/book3s-apiv2: Add support for Hostwide GSB elements Vaibhav Jain
2024-12-22 14:02 ` [PATCH 3/6] kvm powerpc/book3s-apiv2: Add kunit tests " Vaibhav Jain
2024-12-22 14:02 ` [PATCH 4/6] kvm powerpc/book3s-apiv2: Introduce kvm-hv specific PMU Vaibhav Jain
2024-12-30 9:28 ` Gautam Menghani
2025-01-06 11:01 ` Vaibhav Jain
2024-12-22 14:02 ` [PATCH 5/6] powerpc/book3s-hv-pmu: Implement GSB message-ops for hostwide counters Vaibhav Jain
2024-12-30 7:31 ` Gautam Menghani
2025-01-06 11:24 ` Vaibhav Jain
2024-12-22 14:02 ` [PATCH 6/6] kvm powerpc/book3s-hv-pmu: Add perf-events for Hostwide counters Vaibhav Jain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).