From: andre.przywara@arm.com (Andre Przywara)
To: linux-arm-kernel@lists.infradead.org
Subject: [PATCH v5 01/19] arm/arm64: KVM: rework MPIDR assignment and add accessors
Date: Mon, 08 Dec 2014 15:26:09 +0000 [thread overview]
Message-ID: <5485C311.7040605@arm.com> (raw)
In-Reply-To: <20141208150654.GA17904@leverpostej>
Hi Mark,
On 08/12/14 15:06, Mark Rutland wrote:
> Hi Andre,
>
> On Mon, Dec 08, 2014 at 12:37:36PM +0000, Andre Przywara wrote:
>> The virtual MPIDR registers (containing topology information) for the
>> guest are currently mapped linearily to the vcpu_id. Improve this
>> mapping for arm64 by using three levels to not artificially limit the
>> number of vCPUs.
>
> For AArch64, there are four levels in the MPIDR_EL1, Aff{3,2,1,0}, and
> all four are covered by MPIDR_HWID_BITMASK.
>
> The code below only seems to handle Aff{2,1,0}. Is the vCPU limited at
> creation time such that Aff3 is always zero?
Yes. The GICv3 has a bit to optionally drop Aff3 support from it's own
CPU addressing scheme, which we use to avoid "real" 64-bit register
accesses [1].
So by definition we don't use Aff3 for the VCPUs, which limits our
number of VCPUs to 2**20 (8+8+4), which isn't really a limit right now.
Other parts of the code only allow 255 VCPUs at the moment.
Cheers,
Andre.
[1]
http://www.linux-arm.org/git?p=linux-ap.git;a=blob;f=virt/kvm/arm/vgic-v3-emul.c;h=18af188e8c64da5ed21291f8b7da02cd7aed07c4;hb=8bcfd318e0edabbf9aafa8512af0b56507495661#l261
>> To help this, change and rename the kvm_vcpu_get_mpidr() function to
>> mask off the non-affinity bits in the MPIDR register.
>> Also add an accessor to later allow easier access to a vCPU with a
>> given MPIDR. Use this new accessor in the PSCI emulation.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
>> ---
>> Changelog v4...v5:
>> (add Reviewed-by:)
>>
>> Changelog v3...v4:
>> - rename kvm_vcpu_get_mpidr() to kvm_vcpu_get_mpidr_aff()
>> - simplify kvm_mpidr_to_vcpu()
>> - fixup comment
>>
>> arch/arm/include/asm/kvm_emulate.h | 5 +++--
>> arch/arm/include/asm/kvm_host.h | 2 ++
>> arch/arm/kvm/arm.c | 13 +++++++++++++
>> arch/arm/kvm/psci.c | 17 +++++------------
>> arch/arm64/include/asm/kvm_emulate.h | 5 +++--
>> arch/arm64/include/asm/kvm_host.h | 2 ++
>> arch/arm64/kvm/sys_regs.c | 13 +++++++++++--
>> 7 files changed, 39 insertions(+), 18 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_emulate.h b/arch/arm/include/asm/kvm_emulate.h
>> index b9db269..3ae88ac 100644
>> --- a/arch/arm/include/asm/kvm_emulate.h
>> +++ b/arch/arm/include/asm/kvm_emulate.h
>> @@ -23,6 +23,7 @@
>> #include <asm/kvm_asm.h>
>> #include <asm/kvm_mmio.h>
>> #include <asm/kvm_arm.h>
>> +#include <asm/cputype.h>
>>
>> unsigned long *vcpu_reg(struct kvm_vcpu *vcpu, u8 reg_num);
>> unsigned long *vcpu_spsr(struct kvm_vcpu *vcpu);
>> @@ -162,9 +163,9 @@ static inline u32 kvm_vcpu_hvc_get_imm(struct kvm_vcpu *vcpu)
>> return kvm_vcpu_get_hsr(vcpu) & HSR_HVC_IMM_MASK;
>> }
>>
>> -static inline unsigned long kvm_vcpu_get_mpidr(struct kvm_vcpu *vcpu)
>> +static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
>> {
>> - return vcpu->arch.cp15[c0_MPIDR];
>> + return vcpu->arch.cp15[c0_MPIDR] & MPIDR_HWID_BITMASK;
>> }
>>
>> static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
>> index 53036e2..b443dfe 100644
>> --- a/arch/arm/include/asm/kvm_host.h
>> +++ b/arch/arm/include/asm/kvm_host.h
>> @@ -236,6 +236,8 @@ static inline void vgic_arch_setup(const struct vgic_params *vgic)
>> int kvm_perf_init(void);
>> int kvm_perf_teardown(void);
>>
>> +struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr);
>> +
>> static inline void kvm_arch_hardware_disable(void) {}
>> static inline void kvm_arch_hardware_unsetup(void) {}
>> static inline void kvm_arch_sync_events(struct kvm *kvm) {}
>> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
>> index 9e193c8..c2a5c69 100644
>> --- a/arch/arm/kvm/arm.c
>> +++ b/arch/arm/kvm/arm.c
>> @@ -977,6 +977,19 @@ static void check_kvm_target_cpu(void *ret)
>> *(int *)ret = kvm_target_cpu();
>> }
>>
>> +struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
>> +{
>> + struct kvm_vcpu *vcpu;
>> + int i;
>> +
>> + mpidr &= MPIDR_HWID_BITMASK;
>> + kvm_for_each_vcpu(i, vcpu, kvm) {
>> + if (mpidr == kvm_vcpu_get_mpidr_aff(vcpu))
>> + return vcpu;
>> + }
>> + return NULL;
>> +}
>> +
>> /**
>> * Initialize Hyp-mode and memory mappings on all CPUs.
>> */
>> diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c
>> index 09cf377..84121b2 100644
>> --- a/arch/arm/kvm/psci.c
>> +++ b/arch/arm/kvm/psci.c
>> @@ -21,6 +21,7 @@
>> #include <asm/cputype.h>
>> #include <asm/kvm_emulate.h>
>> #include <asm/kvm_psci.h>
>> +#include <asm/kvm_host.h>
>>
>> /*
>> * This is an implementation of the Power State Coordination Interface
>> @@ -65,25 +66,17 @@ static void kvm_psci_vcpu_off(struct kvm_vcpu *vcpu)
>> static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu)
>> {
>> struct kvm *kvm = source_vcpu->kvm;
>> - struct kvm_vcpu *vcpu = NULL, *tmp;
>> + struct kvm_vcpu *vcpu = NULL;
>> wait_queue_head_t *wq;
>> unsigned long cpu_id;
>> unsigned long context_id;
>> - unsigned long mpidr;
>> phys_addr_t target_pc;
>> - int i;
>>
>> - cpu_id = *vcpu_reg(source_vcpu, 1);
>> + cpu_id = *vcpu_reg(source_vcpu, 1) & MPIDR_HWID_BITMASK;
>> if (vcpu_mode_is_32bit(source_vcpu))
>> cpu_id &= ~((u32) 0);
>>
>> - kvm_for_each_vcpu(i, tmp, kvm) {
>> - mpidr = kvm_vcpu_get_mpidr(tmp);
>> - if ((mpidr & MPIDR_HWID_BITMASK) == (cpu_id & MPIDR_HWID_BITMASK)) {
>> - vcpu = tmp;
>> - break;
>> - }
>> - }
>> + vcpu = kvm_mpidr_to_vcpu(kvm, cpu_id);
>>
>> /*
>> * Make sure the caller requested a valid CPU and that the CPU is
>> @@ -154,7 +147,7 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu)
>> * then ON else OFF
>> */
>> kvm_for_each_vcpu(i, tmp, kvm) {
>> - mpidr = kvm_vcpu_get_mpidr(tmp);
>> + mpidr = kvm_vcpu_get_mpidr_aff(tmp);
>> if (((mpidr & target_affinity_mask) == target_affinity) &&
>> !tmp->arch.pause) {
>> return PSCI_0_2_AFFINITY_LEVEL_ON;
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>> index 5674a55..d4daaa5 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -27,6 +27,7 @@
>> #include <asm/kvm_arm.h>
>> #include <asm/kvm_mmio.h>
>> #include <asm/ptrace.h>
>> +#include <asm/cputype.h>
>>
>> unsigned long *vcpu_reg32(const struct kvm_vcpu *vcpu, u8 reg_num);
>> unsigned long *vcpu_spsr32(const struct kvm_vcpu *vcpu);
>> @@ -182,9 +183,9 @@ static inline u8 kvm_vcpu_trap_get_fault_type(const struct kvm_vcpu *vcpu)
>> return kvm_vcpu_get_hsr(vcpu) & ESR_EL2_FSC_TYPE;
>> }
>>
>> -static inline unsigned long kvm_vcpu_get_mpidr(struct kvm_vcpu *vcpu)
>> +static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)
>> {
>> - return vcpu_sys_reg(vcpu, MPIDR_EL1);
>> + return vcpu_sys_reg(vcpu, MPIDR_EL1) & MPIDR_HWID_BITMASK;
>> }
>>
>> static inline void kvm_vcpu_set_be(struct kvm_vcpu *vcpu)
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>> index 2012c4b..286bb61 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -207,6 +207,8 @@ int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> int kvm_perf_init(void);
>> int kvm_perf_teardown(void);
>>
>> +struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr);
>> +
>> static inline void __cpu_init_hyp_mode(phys_addr_t boot_pgd_ptr,
>> phys_addr_t pgd_ptr,
>> unsigned long hyp_stack_ptr,
>> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
>> index 3d7c2df..136e679 100644
>> --- a/arch/arm64/kvm/sys_regs.c
>> +++ b/arch/arm64/kvm/sys_regs.c
>> @@ -252,10 +252,19 @@ static void reset_amair_el1(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
>>
>> static void reset_mpidr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)
>> {
>> + u64 mpidr;
>> +
>> /*
>> - * Simply map the vcpu_id into the Aff0 field of the MPIDR.
>> + * Map the vcpu_id into the first three affinity level fields of
>> + * the MPIDR. We limit the number of VCPUs in level 0 due to a
>> + * limitation to 16 CPUs in that level in the ICC_SGIxR registers
>> + * of the GICv3 to be able to address each CPU directly when
>> + * sending IPIs.
>> */
>> - vcpu_sys_reg(vcpu, MPIDR_EL1) = (1UL << 31) | (vcpu->vcpu_id & 0xff);
>> + mpidr = (vcpu->vcpu_id & 0x0f) << MPIDR_LEVEL_SHIFT(0);
>> + mpidr |= ((vcpu->vcpu_id >> 4) & 0xff) << MPIDR_LEVEL_SHIFT(1);
>> + mpidr |= ((vcpu->vcpu_id >> 12) & 0xff) << MPIDR_LEVEL_SHIFT(2);
>> + vcpu_sys_reg(vcpu, MPIDR_EL1) = (1ULL << 31) | mpidr;
>> }
>>
>> /* Silly macro to expand the DBG{BCR,BVR,WVR,WCR}n_EL1 registers in one go */
>> --
>> 1.7.9.5
>>
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>
next prev parent reply other threads:[~2014-12-08 15:26 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-12-08 12:37 [PATCH v5 00/19] KVM GICv3 emulation Andre Przywara
2014-12-08 12:37 ` [PATCH v5 01/19] arm/arm64: KVM: rework MPIDR assignment and add accessors Andre Przywara
2014-12-08 15:06 ` Mark Rutland
2014-12-08 15:26 ` Andre Przywara [this message]
2014-12-18 9:00 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 02/19] arm/arm64: KVM: pass down user space provided GIC type into vGIC code Andre Przywara
2014-12-18 9:03 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 03/19] arm/arm64: KVM: refactor vgic_handle_mmio() function Andre Przywara
2014-12-18 9:06 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 04/19] arm/arm64: KVM: wrap 64 bit MMIO accesses with two 32 bit ones Andre Przywara
2014-12-18 9:09 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 05/19] arm/arm64: KVM: introduce per-VM ops Andre Przywara
2014-12-13 13:29 ` Christoffer Dall
2014-12-23 11:43 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 06/19] arm/arm64: KVM: move kvm_register_device_ops() into vGIC probing Andre Przywara
2014-12-23 11:57 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 07/19] arm/arm64: KVM: dont rely on a valid GICH base address Andre Przywara
2014-12-23 11:58 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 08/19] arm/arm64: KVM: make the maximum number of vCPUs a per-VM value Andre Przywara
2014-12-13 13:31 ` Christoffer Dall
2014-12-23 12:07 ` Marc Zyngier
2015-01-05 17:34 ` Andre Przywara
2015-01-06 13:29 ` Christoffer Dall
2015-01-06 14:03 ` Andre Przywara
2015-01-06 13:58 ` Peter Maydell
2014-12-08 12:37 ` [PATCH v5 09/19] arm/arm64: KVM: make the value of ICC_SRE_EL1 a per-VM variable Andre Przywara
2014-12-23 12:14 ` Marc Zyngier
2014-12-08 12:37 ` [PATCH v5 10/19] arm/arm64: KVM: refactor MMIO accessors Andre Przywara
2014-12-08 12:37 ` [PATCH v5 11/19] arm/arm64: KVM: refactor/wrap vgic_set/get_attr() Andre Przywara
2014-12-08 12:37 ` [PATCH v5 12/19] arm/arm64: KVM: add vgic.h header file Andre Przywara
2014-12-08 12:37 ` [PATCH v5 13/19] arm/arm64: KVM: split GICv2 specific emulation code from vgic.c Andre Przywara
2014-12-08 12:37 ` [PATCH v5 14/19] arm/arm64: KVM: add opaque private pointer to MMIO data Andre Przywara
2014-12-08 12:37 ` [PATCH v5 15/19] arm/arm64: KVM: add virtual GICv3 distributor emulation Andre Przywara
2014-12-13 13:37 ` Christoffer Dall
2014-12-15 11:32 ` Andre Przywara
2014-12-08 12:37 ` [PATCH v5 16/19] arm64: GICv3: introduce symbolic names for GICv3 ICC_SGI1R_EL1 fields Andre Przywara
2014-12-08 12:37 ` [PATCH v5 17/19] arm64: KVM: add SGI generation register emulation Andre Przywara
2014-12-08 12:37 ` [PATCH v5 18/19] arm/arm64: KVM: enable kernel side of GICv3 emulation Andre Przywara
2014-12-13 13:42 ` Christoffer Dall
2015-01-05 17:58 ` Andre Przywara
2014-12-08 12:37 ` [PATCH v5 19/19] arm/arm64: KVM: allow userland to request a virtual GICv3 Andre Przywara
2014-12-13 13:45 ` Christoffer Dall
2014-12-15 11:50 ` Andre Przywara
2014-12-15 13:02 ` Christoffer Dall
2014-12-13 13:53 ` [PATCH v5 00/19] KVM GICv3 emulation Christoffer Dall
2014-12-15 14:57 ` Andre Przywara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5485C311.7040605@arm.com \
--to=andre.przywara@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).