* [PATCH 1/4] arm/arm64: KVM: PSCI: use MPIDR to identify a target CPU
2013-10-18 17:19 [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Marc Zyngier
@ 2013-10-18 17:19 ` Marc Zyngier
2013-10-18 17:19 ` [PATCH 2/4] ARM: KVM: Fix MPIDR computing to support virtual clusters Marc Zyngier
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2013-10-18 17:19 UTC (permalink / raw)
To: linux-arm-kernel
The KVM PSCI code blindly assumes that vcpu_id and MPIDR are
the same thing. This is true when vcpus are organized as a flat
topology, but is wrong when trying to emulate any other topology
(such as A15 clusters).
Change the KVM PSCI CPU_ON code to look at the MPIDR instead
of the vcpu_id to pick a target CPU.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/include/asm/kvm_emulate.h | 5 +++++
arch/arm/kvm/psci.c | 17 +++++++++++++----
arch/arm64/include/asm/kvm_emulate.h | 5 +++++
3 files changed, 23 insertions(+), 4 deletions(-)
diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
index a464e8d..708e4d8 100644
--- a/arch/arm/include/asm/kvm_emulate.h
+++ b/arch/arm/include/asm/kvm_emulate.h
@@ -157,4 +157,9 @@ static inline u32 kvm_vcpu_hvc_get_imm(struct kvm_vcpu *vcpu)
return kvm_vcpu_get_hsr(vcpu) & HSR_HVC_IMM_MASK;
}
+static inline unsigned long kvm_vcpu_get_mpidr(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.cp15[c0_MPIDR];
+}
+
#endif /* __ARM_KVM_EMULATE_H__ */
diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c
index 86a693a..3112631 100644
--- a/arch/arm/kvm/psci.c
+++ b/arch/arm/kvm/psci.c
@@ -18,6 +18,7 @@
#include <linux/kvm_host.h>
#include <linux/wait.h>
+#include <asm/cputype.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_psci.h>
@@ -34,22 +35,30 @@ static void kvm_psci_vcpu_off(struct kvm_vcpu *vcpu)
static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)
{
struct kvm *kvm = source_vcpu->kvm;
- struct kvm_vcpu *vcpu;
+ struct kvm_vcpu *vcpu = NULL, *tmp;
wait_queue_head_t *wq;
unsigned long cpu_id;
+ unsigned long mpidr;
phys_addr_t target_pc;
+ int i;
cpu_id = *vcpu_reg(source_vcpu, 1);
if (vcpu_mode_is_32bit(source_vcpu))
cpu_id &= ~((u32) 0);
- if (cpu_id >= atomic_read(&kvm->online_vcpus))
+ kvm_for_each_vcpu(i, tmp, kvm) {
+ mpidr = kvm_vcpu_get_mpidr(tmp);
+ if ((mpidr & MPIDR_HWID_BITMASK) == (cpu_id & MPIDR_HWID_BITMASK)) {
+ vcpu = tmp;
+ break;
+ }
+ }
+
+ if (!vcpu)
return KVM_PSCI_RET_INVAL;
target_pc = *vcpu_reg(source_vcpu, 2);
- vcpu = kvm_get_vcpu(kvm, cpu_id);
-
wq = kvm_arch_vcpu_wq(vcpu);
if (!waitqueue_active(wq))
return KVM_PSCI_RET_INVAL;
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index eec0738..6df93cd 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -177,4 +177,9 @@ static inline u8 kvm_vcpu_trap_get_fault(const struct kvm_vcpu *vcpu)
return kvm_vcpu_get_hsr(vcpu) & ESR_EL2_FSC_TYPE;
}
+static inline unsigned long kvm_vcpu_get_mpidr(struct kvm_vcpu *vcpu)
+{
+ return vcpu_sys_reg(vcpu, MPIDR_EL1);
+}
+
#endif /* __ARM64_KVM_EMULATE_H__ */
--
1.8.2.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/4] ARM: KVM: Fix MPIDR computing to support virtual clusters
2013-10-18 17:19 [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Marc Zyngier
2013-10-18 17:19 ` [PATCH 1/4] arm/arm64: KVM: PSCI: use MPIDR to identify a target CPU Marc Zyngier
@ 2013-10-18 17:19 ` Marc Zyngier
2013-10-18 17:19 ` [PATCH 3/4] ARM: KVM: fix L2CTLR to be per-cluster Marc Zyngier
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2013-10-18 17:19 UTC (permalink / raw)
To: linux-arm-kernel
In order to be able to support more than 4 A7 or A15 CPUs,
we need to fix the MPIDR computing to reflect the fact that
both A15 and A7 can only exist in clusters of at most 4 CPUs.
Fix the MPIDR computing to allow virtual clusters to be exposed
to the guest.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/kvm/coproc.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index a629f2c..631e6bd 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -74,11 +74,13 @@ int kvm_handle_cp14_access(struct kvm_vcpu *vcpu, struct kvm_run *run)
static void reset_mpidr(struct kvm_vcpu *vcpu, const struct coproc_reg *r)
{
/*
- * Compute guest MPIDR. No need to mess around with different clusters
- * but we read the 'U' bit from the underlying hardware directly.
+ * Compute guest MPIDR. We build a virtual cluster out of the
+ * vcpu_id, but we read the 'U' bit from the underlying
+ * hardware directly.
*/
- vcpu->arch.cp15[c0_MPIDR] = (read_cpuid_mpidr() & MPIDR_SMP_BITMASK)
- | vcpu->vcpu_id;
+ vcpu->arch.cp15[c0_MPIDR] = ((read_cpuid_mpidr() & MPIDR_SMP_BITMASK) |
+ ((vcpu->vcpu_id >> 2) << MPIDR_LEVEL_BITS) |
+ (vcpu->vcpu_id & 3));
}
/* TRM entries A7:4.3.31 A15:4.3.28 - RO WI */
--
1.8.2.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/4] ARM: KVM: fix L2CTLR to be per-cluster
2013-10-18 17:19 [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Marc Zyngier
2013-10-18 17:19 ` [PATCH 1/4] arm/arm64: KVM: PSCI: use MPIDR to identify a target CPU Marc Zyngier
2013-10-18 17:19 ` [PATCH 2/4] ARM: KVM: Fix MPIDR computing to support virtual clusters Marc Zyngier
@ 2013-10-18 17:19 ` Marc Zyngier
2013-10-18 17:19 ` [PATCH 4/4] ARM: KVM: drop limitation to 4 CPU VMs Marc Zyngier
2013-10-22 14:10 ` [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Christoffer Dall
4 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2013-10-18 17:19 UTC (permalink / raw)
To: linux-arm-kernel
The L2CTLR register contains the number of CPUs in this cluster.
Make sure the register content is actually relevant to the vcpu
that is being configured by computing the number of cores that are
part of its cluster.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/kvm/coproc.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm/kvm/coproc.c b/arch/arm/kvm/coproc.c
index 631e6bd..78c0885 100644
--- a/arch/arm/kvm/coproc.c
+++ b/arch/arm/kvm/coproc.c
@@ -124,6 +124,10 @@ static void reset_l2ctlr(struct kvm_vcpu *vcpu, const struct coproc_reg *r)
asm volatile("mrc p15, 1, %0, c9, c0, 2\n" : "=r" (l2ctlr));
l2ctlr &= ~(3 << 24);
ncores = atomic_read(&vcpu->kvm->online_vcpus) - 1;
+ /* How many cores in the current cluster and the next ones */
+ ncores -= (vcpu->vcpu_id & ~3);
+ /* Cap it to the maximum number of cores in a single cluster */
+ ncores = min(ncores, 3U);
l2ctlr |= (ncores & 3) << 24;
vcpu->arch.cp15[c9_L2CTLR] = l2ctlr;
--
1.8.2.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 4/4] ARM: KVM: drop limitation to 4 CPU VMs
2013-10-18 17:19 [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Marc Zyngier
` (2 preceding siblings ...)
2013-10-18 17:19 ` [PATCH 3/4] ARM: KVM: fix L2CTLR to be per-cluster Marc Zyngier
@ 2013-10-18 17:19 ` Marc Zyngier
2013-10-22 14:10 ` [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Christoffer Dall
4 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2013-10-18 17:19 UTC (permalink / raw)
To: linux-arm-kernel
Now that the KVM/arm code knows about affinity, remove the hard
limit of 4 vcpus per VM.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/kvm/reset.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
index d9bbd83..2c5add0 100644
--- a/arch/arm/kvm/reset.c
+++ b/arch/arm/kvm/reset.c
@@ -33,8 +33,6 @@
* Cortex-A15 and Cortex-A7 Reset Values
*/
-static const int cortexa_max_cpu_idx = 3;
-
static struct kvm_regs cortexa_regs_reset = {
.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
};
@@ -64,8 +62,6 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
switch (vcpu->arch.target) {
case KVM_ARM_TARGET_CORTEX_A7:
case KVM_ARM_TARGET_CORTEX_A15:
- if (vcpu->vcpu_id > cortexa_max_cpu_idx)
- return -EINVAL;
cpu_reset = &cortexa_regs_reset;
vcpu->arch.midr = read_cpuid_id();
cpu_vtimer_irq = &cortexa_vtimer_irq;
--
1.8.2.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs
2013-10-18 17:19 [PATCH 0/4] ARM: KVM: allow VMs with more than 4 CPUs Marc Zyngier
` (3 preceding siblings ...)
2013-10-18 17:19 ` [PATCH 4/4] ARM: KVM: drop limitation to 4 CPU VMs Marc Zyngier
@ 2013-10-22 14:10 ` Christoffer Dall
4 siblings, 0 replies; 6+ messages in thread
From: Christoffer Dall @ 2013-10-22 14:10 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 18, 2013 at 06:19:02PM +0100, Marc Zyngier wrote:
> Currently, KVM/ARM is limited to 4 A7 or A15 per VM. This is due to
> the fact that KVM doesn't know about clusters, and puts all CPUs at
> affinity level 0.
>
> This patch series removes this limitation by allowing additionnal CPUs
> to be part of a separate cluster (affinity level 1).
>
> This also requires a patch to kvmtool so the generated DT matches the
> expectations of the guest (posted separately).
>
> Tested on TC2 and arm64 model.
>
> Patches based on Christoffer's next tree as of today.
>
> Marc Zyngier (4):
> arm/arm64: KVM: PSCI: use MPIDR to identify a target CPU
> ARM: KVM: Fix MPIDR computing to support virtual clusters
> ARM: KVM: fix L2CTLR to be per-cluster
> ARM: KVM: drop limitation to 4 CPU VMs
>
> arch/arm/include/asm/kvm_emulate.h | 5 +++++
> arch/arm/kvm/coproc.c | 14 ++++++++++----
> arch/arm/kvm/psci.c | 17 +++++++++++++----
> arch/arm/kvm/reset.c | 4 ----
> arch/arm64/include/asm/kvm_emulate.h | 5 +++++
> 5 files changed, 33 insertions(+), 12 deletions(-)
>
> --
> 1.8.2.3
>
>
ack on the series,
I'll apply these to my tree.
--
Christoffer
^ permalink raw reply [flat|nested] 6+ messages in thread