* [RFC PATCH v1 0/4] arm/arm64: vgic-new: Implement API for vGICv3 live migration
@ 2016-07-20 13:02 vijay.kilari at gmail.com
2016-07-20 13:02 ` [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support vijay.kilari at gmail.com
` (3 more replies)
0 siblings, 4 replies; 10+ messages in thread
From: vijay.kilari at gmail.com @ 2016-07-20 13:02 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
This patchset adds API for saving and restoring
of VGICv3 registers to support live migration with new vgic feature.
This API definition is as per draft version of VGICv3 specification
https://lists.cs.columbia.edu/pipermail/kvmarm/2016-May/020355.html
To test live migration with QEMU, the QEMU patches will
be posted soon.
The patch 3 & 4 are picked from the Pavel's previous implementation.
http://www.spinics.net/lists/kvm/msg122040.html
Vijaya Kumar K (4):
arm/arm64: vgic-new: Introduce 64-bit reg access support
arm/arm64: vgic-new: Add distributor and redistributor access
arm/arm64: vgic-new: Introduce find_reg_by_id()
arm/arm64: vgic-new: Implement VGICv3 CPU interface access
arch/arm64/include/uapi/asm/kvm.h | 12 ++
arch/arm64/kvm/Makefile | 1 +
arch/arm64/kvm/sys_regs.c | 22 ++--
arch/arm64/kvm/sys_regs.h | 4 +
include/linux/irqchip/arm-gic-v3.h | 4 +
virt/kvm/arm/vgic/vgic-kvm-device.c | 120 +++++++++++++++++--
virt/kvm/arm/vgic/vgic-mmio-v2.c | 4 +-
virt/kvm/arm/vgic/vgic-mmio-v3.c | 111 +++++++++++++++++
virt/kvm/arm/vgic/vgic-mmio.c | 2 +-
virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 225 +++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic.h | 14 +++
11 files changed, 497 insertions(+), 22 deletions(-)
create mode 100644 virt/kvm/arm/vgic/vgic-sys-reg-v3.c
--
1.7.9.5
^ permalink raw reply [flat|nested] 10+ messages in thread* [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support 2016-07-20 13:02 [RFC PATCH v1 0/4] arm/arm64: vgic-new: Implement API for vGICv3 live migration vijay.kilari at gmail.com @ 2016-07-20 13:02 ` vijay.kilari at gmail.com 2016-07-20 14:02 ` Marc Zyngier 2016-07-20 13:02 ` [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access vijay.kilari at gmail.com ` (2 subsequent siblings) 3 siblings, 1 reply; 10+ messages in thread From: vijay.kilari at gmail.com @ 2016-07-20 13:02 UTC (permalink / raw) To: linux-arm-kernel From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> vgic_attr_regs_access() handles only 32-bit register value. Introduce union ureg to handle both 32 and 64 bit register size. Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> --- virt/kvm/arm/vgic/vgic-kvm-device.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c index cc843fe..cace996 100644 --- a/virt/kvm/arm/vgic/vgic-kvm-device.c +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c @@ -19,6 +19,11 @@ #include <asm/kvm_mmu.h> #include "vgic.h" +union ureg { + u32 reg32; + u64 reg64; +}; + /* common helpers */ static int vgic_check_ioaddr(struct kvm *kvm, phys_addr_t *ioaddr, @@ -255,7 +260,7 @@ void kvm_register_vgic_device(unsigned long type) */ static int vgic_attr_regs_access(struct kvm_device *dev, struct kvm_device_attr *attr, - u32 *reg, bool is_write) + union ureg *reg, bool is_write) { gpa_t addr; int cpuid, ret, c; @@ -293,10 +298,10 @@ static int vgic_attr_regs_access(struct kvm_device *dev, switch (attr->group) { case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: - ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, reg); + ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, ®->reg32); break; case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: - ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, reg); + ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, ®->reg32); break; default: ret = -EINVAL; @@ -328,9 +333,9 @@ static int vgic_v2_set_attr(struct kvm_device *dev, case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: { u32 __user *uaddr = (u32 __user *)(long)attr->addr; - u32 reg; + union ureg reg; - if (get_user(reg, uaddr)) + if (get_user(reg.reg32, uaddr)) return -EFAULT; return vgic_attr_regs_access(dev, attr, ®, true); @@ -353,12 +358,12 @@ static int vgic_v2_get_attr(struct kvm_device *dev, case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: { u32 __user *uaddr = (u32 __user *)(long)attr->addr; - u32 reg = 0; + union ureg reg; ret = vgic_attr_regs_access(dev, attr, ®, false); if (ret) return ret; - return put_user(reg, uaddr); + return put_user(reg.reg32, uaddr); } } -- 1.7.9.5 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support 2016-07-20 13:02 ` [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support vijay.kilari at gmail.com @ 2016-07-20 14:02 ` Marc Zyngier 0 siblings, 0 replies; 10+ messages in thread From: Marc Zyngier @ 2016-07-20 14:02 UTC (permalink / raw) To: linux-arm-kernel On 20/07/16 14:02, vijay.kilari at gmail.com wrote: > From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > > vgic_attr_regs_access() handles only 32-bit register > value. Introduce union ureg to handle both 32 and 64 bit > register size. > > Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > --- > virt/kvm/arm/vgic/vgic-kvm-device.c | 19 ++++++++++++------- > 1 file changed, 12 insertions(+), 7 deletions(-) > > diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c > index cc843fe..cace996 100644 > --- a/virt/kvm/arm/vgic/vgic-kvm-device.c > +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c > @@ -19,6 +19,11 @@ > #include <asm/kvm_mmu.h> > #include "vgic.h" > > +union ureg { > + u32 reg32; > + u64 reg64; > +}; > + Please don't. That's pointlessly ugly, and creates type confusion. I want to see explicit types, all the time. > /* common helpers */ > > static int vgic_check_ioaddr(struct kvm *kvm, phys_addr_t *ioaddr, > @@ -255,7 +260,7 @@ void kvm_register_vgic_device(unsigned long type) > */ > static int vgic_attr_regs_access(struct kvm_device *dev, > struct kvm_device_attr *attr, > - u32 *reg, bool is_write) > + union ureg *reg, bool is_write) Just pass a u64 pointer here... > { > gpa_t addr; > int cpuid, ret, c; > @@ -293,10 +298,10 @@ static int vgic_attr_regs_access(struct kvm_device *dev, u32 tmp32; if (is_write) tmp32 = *reg; > > switch (attr->group) { > case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: > - ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, reg); > + ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, ®->reg32); use tmp32 here. > break; > case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > - ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, reg); > + ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, ®->reg32); > break; > default: > ret = -EINVAL; and in the epilogue: if (!is_write) *reg = tmp32; > @@ -328,9 +333,9 @@ static int vgic_v2_set_attr(struct kvm_device *dev, > case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: { > u32 __user *uaddr = (u32 __user *)(long)attr->addr; > - u32 reg; > + union ureg reg; > > - if (get_user(reg, uaddr)) > + if (get_user(reg.reg32, uaddr)) > return -EFAULT; > > return vgic_attr_regs_access(dev, attr, ®, true); > @@ -353,12 +358,12 @@ static int vgic_v2_get_attr(struct kvm_device *dev, > case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: { > u32 __user *uaddr = (u32 __user *)(long)attr->addr; > - u32 reg = 0; > + union ureg reg; > > ret = vgic_attr_regs_access(dev, attr, ®, false); > if (ret) > return ret; > - return put_user(reg, uaddr); > + return put_user(reg.reg32, uaddr); > } > } > > Same thing everywhere. Thanks, M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access 2016-07-20 13:02 [RFC PATCH v1 0/4] arm/arm64: vgic-new: Implement API for vGICv3 live migration vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support vijay.kilari at gmail.com @ 2016-07-20 13:02 ` vijay.kilari at gmail.com 2016-08-02 14:43 ` Christoffer Dall 2016-07-20 13:02 ` [RFC PATCH v1 3/4] arm/arm64: vgic-new: Introduce find_reg_by_id() vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 4/4] arm/arm64: vgic-new: Implement VGICv3 CPU interface access vijay.kilari at gmail.com 3 siblings, 1 reply; 10+ messages in thread From: vijay.kilari at gmail.com @ 2016-07-20 13:02 UTC (permalink / raw) To: linux-arm-kernel From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> VGICv3 Distributor and Redistributor registers are accessed using KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls. These registers are accessed as 32-bit and cpu mpidr value passed along with register offset is used to identify the cpu for redistributor registers access. The draft version of VGIC v3 specification is define here https://lists.cs.columbia.edu/pipermail/kvmarm/2016-May/020355.html Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> --- arch/arm64/include/uapi/asm/kvm.h | 3 + virt/kvm/arm/vgic/vgic-kvm-device.c | 72 ++++++++++++++++++++++-- virt/kvm/arm/vgic/vgic-mmio-v3.c | 105 +++++++++++++++++++++++++++++++++++ virt/kvm/arm/vgic/vgic-mmio.c | 2 +- virt/kvm/arm/vgic/vgic.h | 8 +++ 5 files changed, 183 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index f209ea1..a6b996e 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -199,10 +199,13 @@ struct kvm_arch_memory_slot { #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2 #define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32 #define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) +#define KVM_DEV_ARM_VGIC_V3_CPUID_MASK \ + (0xffffffffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0 #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT) #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3 #define KVM_DEV_ARM_VGIC_GRP_CTRL 4 +#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 #define KVM_DEV_ARM_VGIC_CTRL_INIT 0 /* Device Control API on vcpu fd */ diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c index cace996..996a720 100644 --- a/virt/kvm/arm/vgic/vgic-kvm-device.c +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c @@ -266,10 +266,17 @@ static int vgic_attr_regs_access(struct kvm_device *dev, int cpuid, ret, c; struct kvm_vcpu *vcpu, *tmp_vcpu; int vcpu_lock_idx = -1; + struct vgic_dist *vgic = &dev->kvm->arch.vgic; - cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> - KVM_DEV_ARM_VGIC_CPUID_SHIFT; - vcpu = kvm_get_vcpu(dev->kvm, cpuid); + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) { + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; + vcpu = kvm_get_vcpu(dev->kvm, cpuid); + } else { + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); + } addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; mutex_lock(&dev->kvm->lock); @@ -301,7 +308,19 @@ static int vgic_attr_regs_access(struct kvm_device *dev, ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, ®->reg32); break; case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: - ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, ®->reg32); + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) + ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, + ®->reg32); + else + ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, + ®->reg32); + break; + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) + ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, + ®->reg32); + else + ret = -EINVAL; break; default: ret = -EINVAL; @@ -411,13 +430,51 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = { static int vgic_v3_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr) { - return vgic_set_common_attr(dev, attr); + int ret; + + ret = vgic_set_common_attr(dev, attr); + if (ret != -ENXIO) + return ret; + + switch (attr->group) { + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { + u32 __user *uaddr = (u32 __user *)(long)attr->addr; + union ureg reg; + + if (get_user(reg.reg32, uaddr)) + return -EFAULT; + + return vgic_attr_regs_access(dev, attr, ®, true); + } + } + return -ENXIO; } static int vgic_v3_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr) { - return vgic_get_common_attr(dev, attr); + int ret; + + ret = vgic_get_common_attr(dev, attr); + if (ret != -ENXIO) + return ret; + + switch (attr->group) { + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { + u32 __user *uaddr = (u32 __user *)(long)attr->addr; + union ureg reg; + + ret = vgic_attr_regs_access(dev, attr, ®, false); + if (ret) + return ret; + ret = put_user(reg.reg32, uaddr); + return ret; + } + } + + return -ENXIO; } static int vgic_v3_has_attr(struct kvm_device *dev, @@ -431,6 +488,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev, return 0; } break; + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: + return vgic_v3_has_attr_regs(dev, attr); case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: return 0; case KVM_DEV_ARM_VGIC_GRP_CTRL: diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c index a0c515a..f6a4e97 100644 --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c @@ -18,6 +18,8 @@ #include <kvm/arm_vgic.h> #include <asm/kvm_emulate.h> +#include <asm/kvm_arm.h> +#include <asm/kvm_mmu.h> #include "vgic.h" #include "vgic-mmio.h" @@ -226,6 +228,9 @@ static const struct vgic_register_region vgic_v3_rdbase_registers[] = { REGISTER_DESC_WITH_LENGTH(GICR_TYPER, vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), + REGISTER_DESC_WITH_LENGTH(GICR_WAKER, + vgic_mmio_read_raz, vgic_mmio_write_wi, 8, + VGIC_ACCESS_32bit), REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER, vgic_mmio_read_raz, vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), @@ -348,6 +353,52 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address) return ret; } +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr) +{ + struct kvm_vcpu *vcpu; + int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS; + const struct vgic_register_region *regions; + gpa_t addr; + int nr_regions, i, len, cpuid; + + addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); + + switch (attr->group) { + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: + regions = vgic_v3_dist_registers; + nr_regions = ARRAY_SIZE(vgic_v3_dist_registers); + break; + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{ + struct vgic_io_device *devices; + struct vgic_io_device *rd_dev; + + devices = dev->kvm->arch.vgic.redist_iodevs; + rd_dev = &devices[vcpu->vcpu_id * 2]; + + regions = rd_dev->regions; + nr_regions = rd_dev->nr_regions; + break; + } + default: + return -ENXIO; + } + + for (i = 0; i < nr_regions; i++) { + if (regions[i].bits_per_irq) + len = (regions[i].bits_per_irq * nr_irqs) / 8; + else + len = regions[i].len; + + if (regions[i].reg_offset <= addr && + regions[i].reg_offset + len > addr) + return 0; + } + + return -ENXIO; +} /* * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI * generation register ICC_SGI1R_EL1) with a given VCPU. @@ -453,3 +504,57 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg) vgic_queue_irq_unlock(vcpu->kvm, irq); } } + +/* + * When userland tries to access the VGIC register handlers, we need to + * create a usable struct vgic_io_device to be passed to the handlers and we + * have to set up a buffer similar to what would have happened if a guest MMIO + * access occurred, including doing endian conversions on BE systems. + */ +static int vgic_v3_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, + bool is_write, int offset, u32 *val) +{ + unsigned int len = 4; + u8 buf[4]; + int ret; + + if (is_write) { + vgic_data_host_to_mmio_bus(buf, len, *val); + ret = kvm_io_gic_ops.write(vcpu, &dev->dev, + dev->base_addr + offset, len, buf); + } else { + ret = kvm_io_gic_ops.read(vcpu, &dev->dev, + dev->base_addr + offset, len, buf); + if (!ret) + *val = vgic_data_mmio_bus_to_host(buf, len); + } + + return ret; +} + +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, + int offset, u32 *val) +{ + struct vgic_io_device *dev; + + dev = &vcpu->kvm->arch.vgic.dist_iodev; + return vgic_v3_uaccess(vcpu, dev, is_write, offset, val); +} + +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, + int offset, u32 *val) +{ + struct vgic_io_device *devices; + struct vgic_io_device *rd_dev; + const struct vgic_register_region *region; + + devices = vcpu->kvm->arch.vgic.redist_iodevs; + rd_dev = &devices[(vcpu->vcpu_id * 2) + 1]; + + region = vgic_find_mmio_region(rd_dev->regions, rd_dev->nr_regions, + offset); + if (region == NULL) + rd_dev = &devices[vcpu->vcpu_id * 2]; + + return vgic_v3_uaccess(vcpu, rd_dev, is_write, offset, val); +} diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c index 9f6fab7..f583959 100644 --- a/virt/kvm/arm/vgic/vgic-mmio.c +++ b/virt/kvm/arm/vgic/vgic-mmio.c @@ -363,7 +363,7 @@ static int match_region(const void *key, const void *elt) } /* Find the proper register handler entry given a certain address offset. */ -static const struct vgic_register_region * +const struct vgic_register_region * vgic_find_mmio_region(const struct vgic_register_region *region, int nr_regions, unsigned int offset) { diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h index 7b300ca..8637690 100644 --- a/virt/kvm/arm/vgic/vgic.h +++ b/virt/kvm/arm/vgic/vgic.h @@ -59,6 +59,9 @@ int vgic_v2_map_resources(struct kvm *kvm); int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address, enum vgic_type); +const struct vgic_register_region * + vgic_find_mmio_region(const struct vgic_register_region *region, + int nr_regions, unsigned int offset); #ifdef CONFIG_KVM_ARM_VGIC_V3 void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu); void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); @@ -71,6 +74,11 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu); int vgic_v3_probe(const struct gic_kvm_info *info); int vgic_v3_map_resources(struct kvm *kvm); int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address); +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr); +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, + int offset, u32 *val); +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, + int offset, u32 *val); #else static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu) { -- 1.7.9.5 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access 2016-07-20 13:02 ` [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access vijay.kilari at gmail.com @ 2016-08-02 14:43 ` Christoffer Dall 2016-08-03 8:33 ` Vijay Kilari 0 siblings, 1 reply; 10+ messages in thread From: Christoffer Dall @ 2016-08-02 14:43 UTC (permalink / raw) To: linux-arm-kernel On Wed, Jul 20, 2016 at 06:32:26PM +0530, vijay.kilari at gmail.com wrote: > From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > > VGICv3 Distributor and Redistributor registers are accessed using > KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS > with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls. > These registers are accessed as 32-bit and cpu mpidr > value passed along with register offset is used to identify the > cpu for redistributor registers access. > > The draft version of VGIC v3 specification is define here > https://lists.cs.columbia.edu/pipermail/kvmarm/2016-May/020355.html > > Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > --- > arch/arm64/include/uapi/asm/kvm.h | 3 + > virt/kvm/arm/vgic/vgic-kvm-device.c | 72 ++++++++++++++++++++++-- > virt/kvm/arm/vgic/vgic-mmio-v3.c | 105 +++++++++++++++++++++++++++++++++++ > virt/kvm/arm/vgic/vgic-mmio.c | 2 +- > virt/kvm/arm/vgic/vgic.h | 8 +++ > 5 files changed, 183 insertions(+), 7 deletions(-) > > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h > index f209ea1..a6b996e 100644 > --- a/arch/arm64/include/uapi/asm/kvm.h > +++ b/arch/arm64/include/uapi/asm/kvm.h > @@ -199,10 +199,13 @@ struct kvm_arch_memory_slot { > #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2 > #define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32 > #define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) > +#define KVM_DEV_ARM_VGIC_V3_CPUID_MASK \ > + (0xffffffffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) > #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0 > #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT) > #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3 > #define KVM_DEV_ARM_VGIC_GRP_CTRL 4 > +#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 > #define KVM_DEV_ARM_VGIC_CTRL_INIT 0 > > /* Device Control API on vcpu fd */ > diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c > index cace996..996a720 100644 > --- a/virt/kvm/arm/vgic/vgic-kvm-device.c > +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c > @@ -266,10 +266,17 @@ static int vgic_attr_regs_access(struct kvm_device *dev, > int cpuid, ret, c; > struct kvm_vcpu *vcpu, *tmp_vcpu; > int vcpu_lock_idx = -1; > + struct vgic_dist *vgic = &dev->kvm->arch.vgic; > > - cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> > - KVM_DEV_ARM_VGIC_CPUID_SHIFT; > - vcpu = kvm_get_vcpu(dev->kvm, cpuid); > + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) { > + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> > + KVM_DEV_ARM_VGIC_CPUID_SHIFT; > + vcpu = kvm_get_vcpu(dev->kvm, cpuid); > + } else { > + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> > + KVM_DEV_ARM_VGIC_CPUID_SHIFT; > + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); > + } > addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; > > mutex_lock(&dev->kvm->lock); > @@ -301,7 +308,19 @@ static int vgic_attr_regs_access(struct kvm_device *dev, > ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, ®->reg32); > break; > case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > - ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, ®->reg32); > + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) > + ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, > + ®->reg32); > + else > + ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, > + ®->reg32); > + break; > + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: > + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) > + ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, > + ®->reg32); > + else > + ret = -EINVAL; > break; > default: > ret = -EINVAL; > @@ -411,13 +430,51 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = { > static int vgic_v3_set_attr(struct kvm_device *dev, > struct kvm_device_attr *attr) > { > - return vgic_set_common_attr(dev, attr); > + int ret; > + > + ret = vgic_set_common_attr(dev, attr); > + if (ret != -ENXIO) > + return ret; > + > + switch (attr->group) { > + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { > + u32 __user *uaddr = (u32 __user *)(long)attr->addr; > + union ureg reg; > + > + if (get_user(reg.reg32, uaddr)) > + return -EFAULT; > + > + return vgic_attr_regs_access(dev, attr, ®, true); > + } > + } > + return -ENXIO; > } > > static int vgic_v3_get_attr(struct kvm_device *dev, > struct kvm_device_attr *attr) > { > - return vgic_get_common_attr(dev, attr); > + int ret; > + > + ret = vgic_get_common_attr(dev, attr); > + if (ret != -ENXIO) > + return ret; > + > + switch (attr->group) { > + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { > + u32 __user *uaddr = (u32 __user *)(long)attr->addr; > + union ureg reg; > + > + ret = vgic_attr_regs_access(dev, attr, ®, false); > + if (ret) > + return ret; > + ret = put_user(reg.reg32, uaddr); > + return ret; > + } > + } > + > + return -ENXIO; > } > > static int vgic_v3_has_attr(struct kvm_device *dev, > @@ -431,6 +488,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev, > return 0; > } > break; > + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: > + return vgic_v3_has_attr_regs(dev, attr); > case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: > return 0; > case KVM_DEV_ARM_VGIC_GRP_CTRL: > diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c > index a0c515a..f6a4e97 100644 > --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c > +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c > @@ -18,6 +18,8 @@ > #include <kvm/arm_vgic.h> > > #include <asm/kvm_emulate.h> > +#include <asm/kvm_arm.h> > +#include <asm/kvm_mmu.h> > > #include "vgic.h" > #include "vgic-mmio.h" > @@ -226,6 +228,9 @@ static const struct vgic_register_region vgic_v3_rdbase_registers[] = { > REGISTER_DESC_WITH_LENGTH(GICR_TYPER, > vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8, > VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), > + REGISTER_DESC_WITH_LENGTH(GICR_WAKER, > + vgic_mmio_read_raz, vgic_mmio_write_wi, 8, > + VGIC_ACCESS_32bit), > REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER, > vgic_mmio_read_raz, vgic_mmio_write_wi, 8, > VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), > @@ -348,6 +353,52 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address) > return ret; > } > > +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr) > +{ > + struct kvm_vcpu *vcpu; > + int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS; > + const struct vgic_register_region *regions; > + gpa_t addr; > + int nr_regions, i, len, cpuid; > + > + addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; > + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> > + KVM_DEV_ARM_VGIC_CPUID_SHIFT; > + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); > + > + switch (attr->group) { > + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > + regions = vgic_v3_dist_registers; > + nr_regions = ARRAY_SIZE(vgic_v3_dist_registers); > + break; > + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{ > + struct vgic_io_device *devices; > + struct vgic_io_device *rd_dev; > + > + devices = dev->kvm->arch.vgic.redist_iodevs; > + rd_dev = &devices[vcpu->vcpu_id * 2]; > + > + regions = rd_dev->regions; > + nr_regions = rd_dev->nr_regions; > + break; > + } > + default: > + return -ENXIO; > + } > + > + for (i = 0; i < nr_regions; i++) { > + if (regions[i].bits_per_irq) > + len = (regions[i].bits_per_irq * nr_irqs) / 8; > + else > + len = regions[i].len; > + > + if (regions[i].reg_offset <= addr && > + regions[i].reg_offset + len > addr) > + return 0; > + } > + > + return -ENXIO; > +} > /* > * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI > * generation register ICC_SGI1R_EL1) with a given VCPU. > @@ -453,3 +504,57 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg) > vgic_queue_irq_unlock(vcpu->kvm, irq); > } > } > + > +/* > + * When userland tries to access the VGIC register handlers, we need to > + * create a usable struct vgic_io_device to be passed to the handlers and we > + * have to set up a buffer similar to what would have happened if a guest MMIO > + * access occurred, including doing endian conversions on BE systems. > + */ > +static int vgic_v3_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, > + bool is_write, int offset, u32 *val) > +{ > + unsigned int len = 4; > + u8 buf[4]; > + int ret; > + > + if (is_write) { > + vgic_data_host_to_mmio_bus(buf, len, *val); > + ret = kvm_io_gic_ops.write(vcpu, &dev->dev, > + dev->base_addr + offset, len, buf); > + } else { > + ret = kvm_io_gic_ops.read(vcpu, &dev->dev, > + dev->base_addr + offset, len, buf); > + if (!ret) > + *val = vgic_data_mmio_bus_to_host(buf, len); > + } > + > + return ret; > +} > + > +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > + int offset, u32 *val) > +{ > + struct vgic_io_device *dev; > + > + dev = &vcpu->kvm->arch.vgic.dist_iodev; You're not supposed to do this. You're supposed to create a temporary vgic_io_device for this, just like we do in vgic_v2_dist_uaccess(). This is why we went down the road of all the init sequence problems. Any reason why you didn't just follow the lead from the v2 implementation? Thanks, -Christoffer > + return vgic_v3_uaccess(vcpu, dev, is_write, offset, val); > +} > + > +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > + int offset, u32 *val) > +{ > + struct vgic_io_device *devices; > + struct vgic_io_device *rd_dev; > + const struct vgic_register_region *region; > + > + devices = vcpu->kvm->arch.vgic.redist_iodevs; > + rd_dev = &devices[(vcpu->vcpu_id * 2) + 1]; > + > + region = vgic_find_mmio_region(rd_dev->regions, rd_dev->nr_regions, > + offset); > + if (region == NULL) > + rd_dev = &devices[vcpu->vcpu_id * 2]; > + > + return vgic_v3_uaccess(vcpu, rd_dev, is_write, offset, val); > +} > diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c > index 9f6fab7..f583959 100644 > --- a/virt/kvm/arm/vgic/vgic-mmio.c > +++ b/virt/kvm/arm/vgic/vgic-mmio.c > @@ -363,7 +363,7 @@ static int match_region(const void *key, const void *elt) > } > > /* Find the proper register handler entry given a certain address offset. */ > -static const struct vgic_register_region * > +const struct vgic_register_region * > vgic_find_mmio_region(const struct vgic_register_region *region, int nr_regions, > unsigned int offset) > { > diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h > index 7b300ca..8637690 100644 > --- a/virt/kvm/arm/vgic/vgic.h > +++ b/virt/kvm/arm/vgic/vgic.h > @@ -59,6 +59,9 @@ int vgic_v2_map_resources(struct kvm *kvm); > int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address, > enum vgic_type); > > +const struct vgic_register_region * > + vgic_find_mmio_region(const struct vgic_register_region *region, > + int nr_regions, unsigned int offset); > #ifdef CONFIG_KVM_ARM_VGIC_V3 > void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu); > void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); > @@ -71,6 +74,11 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu); > int vgic_v3_probe(const struct gic_kvm_info *info); > int vgic_v3_map_resources(struct kvm *kvm); > int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address); > +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr); > +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > + int offset, u32 *val); > +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > + int offset, u32 *val); > #else > static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu) > { > -- > 1.7.9.5 > ^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access 2016-08-02 14:43 ` Christoffer Dall @ 2016-08-03 8:33 ` Vijay Kilari 2016-08-03 8:42 ` Christoffer Dall 0 siblings, 1 reply; 10+ messages in thread From: Vijay Kilari @ 2016-08-03 8:33 UTC (permalink / raw) To: linux-arm-kernel On Tue, Aug 2, 2016 at 8:13 PM, Christoffer Dall <christoffer.dall@linaro.org> wrote: > On Wed, Jul 20, 2016 at 06:32:26PM +0530, vijay.kilari at gmail.com wrote: >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> >> >> VGICv3 Distributor and Redistributor registers are accessed using >> KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS >> with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls. >> These registers are accessed as 32-bit and cpu mpidr >> value passed along with register offset is used to identify the >> cpu for redistributor registers access. >> >> The draft version of VGIC v3 specification is define here >> https://lists.cs.columbia.edu/pipermail/kvmarm/2016-May/020355.html >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> >> --- >> arch/arm64/include/uapi/asm/kvm.h | 3 + >> virt/kvm/arm/vgic/vgic-kvm-device.c | 72 ++++++++++++++++++++++-- >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 105 +++++++++++++++++++++++++++++++++++ >> virt/kvm/arm/vgic/vgic-mmio.c | 2 +- >> virt/kvm/arm/vgic/vgic.h | 8 +++ >> 5 files changed, 183 insertions(+), 7 deletions(-) >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h >> index f209ea1..a6b996e 100644 >> --- a/arch/arm64/include/uapi/asm/kvm.h >> +++ b/arch/arm64/include/uapi/asm/kvm.h >> @@ -199,10 +199,13 @@ struct kvm_arch_memory_slot { >> #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2 >> #define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32 >> #define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) >> +#define KVM_DEV_ARM_VGIC_V3_CPUID_MASK \ >> + (0xffffffffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0 >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT) >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3 >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4 >> +#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0 >> >> /* Device Control API on vcpu fd */ >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c >> index cace996..996a720 100644 >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c >> @@ -266,10 +266,17 @@ static int vgic_attr_regs_access(struct kvm_device *dev, >> int cpuid, ret, c; >> struct kvm_vcpu *vcpu, *tmp_vcpu; >> int vcpu_lock_idx = -1; >> + struct vgic_dist *vgic = &dev->kvm->arch.vgic; >> >> - cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> >> - KVM_DEV_ARM_VGIC_CPUID_SHIFT; >> - vcpu = kvm_get_vcpu(dev->kvm, cpuid); >> + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) { >> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; >> + vcpu = kvm_get_vcpu(dev->kvm, cpuid); >> + } else { >> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); >> + } >> addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; >> >> mutex_lock(&dev->kvm->lock); >> @@ -301,7 +308,19 @@ static int vgic_attr_regs_access(struct kvm_device *dev, >> ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, ®->reg32); >> break; >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: >> - ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, ®->reg32); >> + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) >> + ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, >> + ®->reg32); >> + else >> + ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, >> + ®->reg32); >> + break; >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: >> + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) >> + ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, >> + ®->reg32); >> + else >> + ret = -EINVAL; >> break; >> default: >> ret = -EINVAL; >> @@ -411,13 +430,51 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = { >> static int vgic_v3_set_attr(struct kvm_device *dev, >> struct kvm_device_attr *attr) >> { >> - return vgic_set_common_attr(dev, attr); >> + int ret; >> + >> + ret = vgic_set_common_attr(dev, attr); >> + if (ret != -ENXIO) >> + return ret; >> + >> + switch (attr->group) { >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { >> + u32 __user *uaddr = (u32 __user *)(long)attr->addr; >> + union ureg reg; >> + >> + if (get_user(reg.reg32, uaddr)) >> + return -EFAULT; >> + >> + return vgic_attr_regs_access(dev, attr, ®, true); >> + } >> + } >> + return -ENXIO; >> } >> >> static int vgic_v3_get_attr(struct kvm_device *dev, >> struct kvm_device_attr *attr) >> { >> - return vgic_get_common_attr(dev, attr); >> + int ret; >> + >> + ret = vgic_get_common_attr(dev, attr); >> + if (ret != -ENXIO) >> + return ret; >> + >> + switch (attr->group) { >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { >> + u32 __user *uaddr = (u32 __user *)(long)attr->addr; >> + union ureg reg; >> + >> + ret = vgic_attr_regs_access(dev, attr, ®, false); >> + if (ret) >> + return ret; >> + ret = put_user(reg.reg32, uaddr); >> + return ret; >> + } >> + } >> + >> + return -ENXIO; >> } >> >> static int vgic_v3_has_attr(struct kvm_device *dev, >> @@ -431,6 +488,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev, >> return 0; >> } >> break; >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: >> + return vgic_v3_has_attr_regs(dev, attr); >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: >> return 0; >> case KVM_DEV_ARM_VGIC_GRP_CTRL: >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c >> index a0c515a..f6a4e97 100644 >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c >> @@ -18,6 +18,8 @@ >> #include <kvm/arm_vgic.h> >> >> #include <asm/kvm_emulate.h> >> +#include <asm/kvm_arm.h> >> +#include <asm/kvm_mmu.h> >> >> #include "vgic.h" >> #include "vgic-mmio.h" >> @@ -226,6 +228,9 @@ static const struct vgic_register_region vgic_v3_rdbase_registers[] = { >> REGISTER_DESC_WITH_LENGTH(GICR_TYPER, >> vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8, >> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), >> + REGISTER_DESC_WITH_LENGTH(GICR_WAKER, >> + vgic_mmio_read_raz, vgic_mmio_write_wi, 8, >> + VGIC_ACCESS_32bit), >> REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER, >> vgic_mmio_read_raz, vgic_mmio_write_wi, 8, >> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), >> @@ -348,6 +353,52 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address) >> return ret; >> } >> >> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr) >> +{ >> + struct kvm_vcpu *vcpu; >> + int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS; >> + const struct vgic_register_region *regions; >> + gpa_t addr; >> + int nr_regions, i, len, cpuid; >> + >> + addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; >> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); >> + >> + switch (attr->group) { >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: >> + regions = vgic_v3_dist_registers; >> + nr_regions = ARRAY_SIZE(vgic_v3_dist_registers); >> + break; >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{ >> + struct vgic_io_device *devices; >> + struct vgic_io_device *rd_dev; >> + >> + devices = dev->kvm->arch.vgic.redist_iodevs; >> + rd_dev = &devices[vcpu->vcpu_id * 2]; >> + >> + regions = rd_dev->regions; >> + nr_regions = rd_dev->nr_regions; >> + break; >> + } >> + default: >> + return -ENXIO; >> + } >> + >> + for (i = 0; i < nr_regions; i++) { >> + if (regions[i].bits_per_irq) >> + len = (regions[i].bits_per_irq * nr_irqs) / 8; >> + else >> + len = regions[i].len; >> + >> + if (regions[i].reg_offset <= addr && >> + regions[i].reg_offset + len > addr) >> + return 0; >> + } >> + >> + return -ENXIO; >> +} >> /* >> * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI >> * generation register ICC_SGI1R_EL1) with a given VCPU. >> @@ -453,3 +504,57 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg) >> vgic_queue_irq_unlock(vcpu->kvm, irq); >> } >> } >> + >> +/* >> + * When userland tries to access the VGIC register handlers, we need to >> + * create a usable struct vgic_io_device to be passed to the handlers and we >> + * have to set up a buffer similar to what would have happened if a guest MMIO >> + * access occurred, including doing endian conversions on BE systems. >> + */ >> +static int vgic_v3_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, >> + bool is_write, int offset, u32 *val) >> +{ >> + unsigned int len = 4; >> + u8 buf[4]; >> + int ret; >> + >> + if (is_write) { >> + vgic_data_host_to_mmio_bus(buf, len, *val); >> + ret = kvm_io_gic_ops.write(vcpu, &dev->dev, >> + dev->base_addr + offset, len, buf); >> + } else { >> + ret = kvm_io_gic_ops.read(vcpu, &dev->dev, >> + dev->base_addr + offset, len, buf); >> + if (!ret) >> + *val = vgic_data_mmio_bus_to_host(buf, len); >> + } >> + >> + return ret; >> +} >> + >> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, >> + int offset, u32 *val) >> +{ >> + struct vgic_io_device *dev; >> + >> + dev = &vcpu->kvm->arch.vgic.dist_iodev; > > You're not supposed to do this. > > You're supposed to create a temporary vgic_io_device for this, just like > we do in vgic_v2_dist_uaccess(). > > This is why we went down the road of all the init sequence problems. > > Any reason why you didn't just follow the lead from the v2 > implementation? I have not looked at v2 implementation. But you are right. With v2 kind of implementation for v3, the init sequence patch is no more required. I will drop the patch "arm/arm64: vgic-new: Create dist and redist iodevs earlier" Thanks, Vijay > > Thanks, > -Christoffer > >> + return vgic_v3_uaccess(vcpu, dev, is_write, offset, val); >> +} >> + >> +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, >> + int offset, u32 *val) >> +{ >> + struct vgic_io_device *devices; >> + struct vgic_io_device *rd_dev; >> + const struct vgic_register_region *region; >> + >> + devices = vcpu->kvm->arch.vgic.redist_iodevs; >> + rd_dev = &devices[(vcpu->vcpu_id * 2) + 1]; >> + >> + region = vgic_find_mmio_region(rd_dev->regions, rd_dev->nr_regions, >> + offset); >> + if (region == NULL) >> + rd_dev = &devices[vcpu->vcpu_id * 2]; >> + >> + return vgic_v3_uaccess(vcpu, rd_dev, is_write, offset, val); >> +} >> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c >> index 9f6fab7..f583959 100644 >> --- a/virt/kvm/arm/vgic/vgic-mmio.c >> +++ b/virt/kvm/arm/vgic/vgic-mmio.c >> @@ -363,7 +363,7 @@ static int match_region(const void *key, const void *elt) >> } >> >> /* Find the proper register handler entry given a certain address offset. */ >> -static const struct vgic_register_region * >> +const struct vgic_register_region * >> vgic_find_mmio_region(const struct vgic_register_region *region, int nr_regions, >> unsigned int offset) >> { >> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h >> index 7b300ca..8637690 100644 >> --- a/virt/kvm/arm/vgic/vgic.h >> +++ b/virt/kvm/arm/vgic/vgic.h >> @@ -59,6 +59,9 @@ int vgic_v2_map_resources(struct kvm *kvm); >> int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address, >> enum vgic_type); >> >> +const struct vgic_register_region * >> + vgic_find_mmio_region(const struct vgic_register_region *region, >> + int nr_regions, unsigned int offset); >> #ifdef CONFIG_KVM_ARM_VGIC_V3 >> void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu); >> void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu); >> @@ -71,6 +74,11 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu); >> int vgic_v3_probe(const struct gic_kvm_info *info); >> int vgic_v3_map_resources(struct kvm *kvm); >> int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t dist_base_address); >> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr); >> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, >> + int offset, u32 *val); >> +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, >> + int offset, u32 *val); >> #else >> static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu) >> { >> -- >> 1.7.9.5 >> ^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access 2016-08-03 8:33 ` Vijay Kilari @ 2016-08-03 8:42 ` Christoffer Dall 0 siblings, 0 replies; 10+ messages in thread From: Christoffer Dall @ 2016-08-03 8:42 UTC (permalink / raw) To: linux-arm-kernel On Wed, Aug 03, 2016 at 02:03:39PM +0530, Vijay Kilari wrote: > On Tue, Aug 2, 2016 at 8:13 PM, Christoffer Dall > <christoffer.dall@linaro.org> wrote: > > On Wed, Jul 20, 2016 at 06:32:26PM +0530, vijay.kilari at gmail.com wrote: > >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > >> > >> VGICv3 Distributor and Redistributor registers are accessed using > >> KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS > >> with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls. > >> These registers are accessed as 32-bit and cpu mpidr > >> value passed along with register offset is used to identify the > >> cpu for redistributor registers access. > >> > >> The draft version of VGIC v3 specification is define here > >> https://lists.cs.columbia.edu/pipermail/kvmarm/2016-May/020355.html > >> > >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > >> --- > >> arch/arm64/include/uapi/asm/kvm.h | 3 + > >> virt/kvm/arm/vgic/vgic-kvm-device.c | 72 ++++++++++++++++++++++-- > >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 105 +++++++++++++++++++++++++++++++++++ > >> virt/kvm/arm/vgic/vgic-mmio.c | 2 +- > >> virt/kvm/arm/vgic/vgic.h | 8 +++ > >> 5 files changed, 183 insertions(+), 7 deletions(-) > >> > >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h > >> index f209ea1..a6b996e 100644 > >> --- a/arch/arm64/include/uapi/asm/kvm.h > >> +++ b/arch/arm64/include/uapi/asm/kvm.h > >> @@ -199,10 +199,13 @@ struct kvm_arch_memory_slot { > >> #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2 > >> #define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32 > >> #define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) > >> +#define KVM_DEV_ARM_VGIC_V3_CPUID_MASK \ > >> + (0xffffffffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT) > >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0 > >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT) > >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3 > >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4 > >> +#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 > >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0 > >> > >> /* Device Control API on vcpu fd */ > >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c > >> index cace996..996a720 100644 > >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c > >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c > >> @@ -266,10 +266,17 @@ static int vgic_attr_regs_access(struct kvm_device *dev, > >> int cpuid, ret, c; > >> struct kvm_vcpu *vcpu, *tmp_vcpu; > >> int vcpu_lock_idx = -1; > >> + struct vgic_dist *vgic = &dev->kvm->arch.vgic; > >> > >> - cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> > >> - KVM_DEV_ARM_VGIC_CPUID_SHIFT; > >> - vcpu = kvm_get_vcpu(dev->kvm, cpuid); > >> + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) { > >> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >> > >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; > >> + vcpu = kvm_get_vcpu(dev->kvm, cpuid); > >> + } else { > >> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> > >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; > >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); > >> + } > >> addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; > >> > >> mutex_lock(&dev->kvm->lock); > >> @@ -301,7 +308,19 @@ static int vgic_attr_regs_access(struct kvm_device *dev, > >> ret = vgic_v2_cpuif_uaccess(vcpu, is_write, addr, ®->reg32); > >> break; > >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > >> - ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, ®->reg32); > >> + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) > >> + ret = vgic_v2_dist_uaccess(vcpu, is_write, addr, > >> + ®->reg32); > >> + else > >> + ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, > >> + ®->reg32); > >> + break; > >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: > >> + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) > >> + ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, > >> + ®->reg32); > >> + else > >> + ret = -EINVAL; > >> break; > >> default: > >> ret = -EINVAL; > >> @@ -411,13 +430,51 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = { > >> static int vgic_v3_set_attr(struct kvm_device *dev, > >> struct kvm_device_attr *attr) > >> { > >> - return vgic_set_common_attr(dev, attr); > >> + int ret; > >> + > >> + ret = vgic_set_common_attr(dev, attr); > >> + if (ret != -ENXIO) > >> + return ret; > >> + > >> + switch (attr->group) { > >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { > >> + u32 __user *uaddr = (u32 __user *)(long)attr->addr; > >> + union ureg reg; > >> + > >> + if (get_user(reg.reg32, uaddr)) > >> + return -EFAULT; > >> + > >> + return vgic_attr_regs_access(dev, attr, ®, true); > >> + } > >> + } > >> + return -ENXIO; > >> } > >> > >> static int vgic_v3_get_attr(struct kvm_device *dev, > >> struct kvm_device_attr *attr) > >> { > >> - return vgic_get_common_attr(dev, attr); > >> + int ret; > >> + > >> + ret = vgic_get_common_attr(dev, attr); > >> + if (ret != -ENXIO) > >> + return ret; > >> + > >> + switch (attr->group) { > >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: { > >> + u32 __user *uaddr = (u32 __user *)(long)attr->addr; > >> + union ureg reg; > >> + > >> + ret = vgic_attr_regs_access(dev, attr, ®, false); > >> + if (ret) > >> + return ret; > >> + ret = put_user(reg.reg32, uaddr); > >> + return ret; > >> + } > >> + } > >> + > >> + return -ENXIO; > >> } > >> > >> static int vgic_v3_has_attr(struct kvm_device *dev, > >> @@ -431,6 +488,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev, > >> return 0; > >> } > >> break; > >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: > >> + return vgic_v3_has_attr_regs(dev, attr); > >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: > >> return 0; > >> case KVM_DEV_ARM_VGIC_GRP_CTRL: > >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c > >> index a0c515a..f6a4e97 100644 > >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c > >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c > >> @@ -18,6 +18,8 @@ > >> #include <kvm/arm_vgic.h> > >> > >> #include <asm/kvm_emulate.h> > >> +#include <asm/kvm_arm.h> > >> +#include <asm/kvm_mmu.h> > >> > >> #include "vgic.h" > >> #include "vgic-mmio.h" > >> @@ -226,6 +228,9 @@ static const struct vgic_register_region vgic_v3_rdbase_registers[] = { > >> REGISTER_DESC_WITH_LENGTH(GICR_TYPER, > >> vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8, > >> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), > >> + REGISTER_DESC_WITH_LENGTH(GICR_WAKER, > >> + vgic_mmio_read_raz, vgic_mmio_write_wi, 8, > >> + VGIC_ACCESS_32bit), > >> REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER, > >> vgic_mmio_read_raz, vgic_mmio_write_wi, 8, > >> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit), > >> @@ -348,6 +353,52 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address) > >> return ret; > >> } > >> > >> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr) > >> +{ > >> + struct kvm_vcpu *vcpu; > >> + int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS; > >> + const struct vgic_register_region *regions; > >> + gpa_t addr; > >> + int nr_regions, i, len, cpuid; > >> + > >> + addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK; > >> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_V3_CPUID_MASK) >> > >> + KVM_DEV_ARM_VGIC_CPUID_SHIFT; > >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, cpuid); > >> + > >> + switch (attr->group) { > >> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > >> + regions = vgic_v3_dist_registers; > >> + nr_regions = ARRAY_SIZE(vgic_v3_dist_registers); > >> + break; > >> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{ > >> + struct vgic_io_device *devices; > >> + struct vgic_io_device *rd_dev; > >> + > >> + devices = dev->kvm->arch.vgic.redist_iodevs; > >> + rd_dev = &devices[vcpu->vcpu_id * 2]; > >> + > >> + regions = rd_dev->regions; > >> + nr_regions = rd_dev->nr_regions; > >> + break; > >> + } > >> + default: > >> + return -ENXIO; > >> + } > >> + > >> + for (i = 0; i < nr_regions; i++) { > >> + if (regions[i].bits_per_irq) > >> + len = (regions[i].bits_per_irq * nr_irqs) / 8; > >> + else > >> + len = regions[i].len; > >> + > >> + if (regions[i].reg_offset <= addr && > >> + regions[i].reg_offset + len > addr) > >> + return 0; > >> + } > >> + > >> + return -ENXIO; > >> +} > >> /* > >> * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI > >> * generation register ICC_SGI1R_EL1) with a given VCPU. > >> @@ -453,3 +504,57 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg) > >> vgic_queue_irq_unlock(vcpu->kvm, irq); > >> } > >> } > >> + > >> +/* > >> + * When userland tries to access the VGIC register handlers, we need to > >> + * create a usable struct vgic_io_device to be passed to the handlers and we > >> + * have to set up a buffer similar to what would have happened if a guest MMIO > >> + * access occurred, including doing endian conversions on BE systems. > >> + */ > >> +static int vgic_v3_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev, > >> + bool is_write, int offset, u32 *val) > >> +{ > >> + unsigned int len = 4; > >> + u8 buf[4]; > >> + int ret; > >> + > >> + if (is_write) { > >> + vgic_data_host_to_mmio_bus(buf, len, *val); > >> + ret = kvm_io_gic_ops.write(vcpu, &dev->dev, > >> + dev->base_addr + offset, len, buf); > >> + } else { > >> + ret = kvm_io_gic_ops.read(vcpu, &dev->dev, > >> + dev->base_addr + offset, len, buf); > >> + if (!ret) > >> + *val = vgic_data_mmio_bus_to_host(buf, len); > >> + } > >> + > >> + return ret; > >> +} > >> + > >> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > >> + int offset, u32 *val) > >> +{ > >> + struct vgic_io_device *dev; > >> + > >> + dev = &vcpu->kvm->arch.vgic.dist_iodev; > > > > You're not supposed to do this. > > > > You're supposed to create a temporary vgic_io_device for this, just like > > we do in vgic_v2_dist_uaccess(). > > > > This is why we went down the road of all the init sequence problems. > > > > Any reason why you didn't just follow the lead from the v2 > > implementation? > > I have not looked at v2 implementation. No need to reinvent the wheel, we're allowed to be inspired by each other. In fact, symmetry between the v2 and v3 specific parts of the VGIC where it makes sense is a preference. > But you are right. > With v2 kind of implementation for v3, the init sequence patch is no > more required. I will drop the patch "arm/arm64: vgic-new: Create dist > and redist iodevs earlier" > Thanks, looking forward to the next version. -Christoffer ^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC PATCH v1 3/4] arm/arm64: vgic-new: Introduce find_reg_by_id() 2016-07-20 13:02 [RFC PATCH v1 0/4] arm/arm64: vgic-new: Implement API for vGICv3 live migration vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access vijay.kilari at gmail.com @ 2016-07-20 13:02 ` vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 4/4] arm/arm64: vgic-new: Implement VGICv3 CPU interface access vijay.kilari at gmail.com 3 siblings, 0 replies; 10+ messages in thread From: vijay.kilari at gmail.com @ 2016-07-20 13:02 UTC (permalink / raw) To: linux-arm-kernel From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> In order to implement vGICv3 CPU interface access, we will need to perform table lookup of system registers. We would need both index_to_params() and find_reg() exported for that purpose, but instead we export a single function which combines them both. Signed-off-by: Pavel Fedin <p.fedin@samsung.com> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> --- arch/arm64/kvm/sys_regs.c | 22 +++++++++++++++------- arch/arm64/kvm/sys_regs.h | 4 ++++ 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index a57d650..16c33ca 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1802,6 +1802,17 @@ static bool index_to_params(u64 id, struct sys_reg_params *params) } } +const struct sys_reg_desc *find_reg_by_id(u64 id, + struct sys_reg_params *params, + const struct sys_reg_desc table[], + unsigned int num) +{ + if (!index_to_params(id, params)) + return NULL; + + return find_reg(params, table, num); +} + /* Decode an index value, and find the sys_reg_desc entry. */ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu, u64 id) @@ -1929,10 +1940,8 @@ static int get_invariant_sys_reg(u64 id, void __user *uaddr) struct sys_reg_params params; const struct sys_reg_desc *r; - if (!index_to_params(id, ¶ms)) - return -ENOENT; - - r = find_reg(¶ms, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs)); + r = find_reg_by_id(id, ¶ms, invariant_sys_regs, + ARRAY_SIZE(invariant_sys_regs)); if (!r) return -ENOENT; @@ -1946,9 +1955,8 @@ static int set_invariant_sys_reg(u64 id, void __user *uaddr) int err; u64 val = 0; /* Make sure high bits are 0 for 32-bit regs */ - if (!index_to_params(id, ¶ms)) - return -ENOENT; - r = find_reg(¶ms, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs)); + r = find_reg_by_id(id, ¶ms, invariant_sys_regs, + ARRAY_SIZE(invariant_sys_regs)); if (!r) return -ENOENT; diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h index dbbb01c..9c6ffd0 100644 --- a/arch/arm64/kvm/sys_regs.h +++ b/arch/arm64/kvm/sys_regs.h @@ -136,6 +136,10 @@ static inline int cmp_sys_reg(const struct sys_reg_desc *i1, return i1->Op2 - i2->Op2; } +const struct sys_reg_desc *find_reg_by_id(u64 id, + struct sys_reg_params *params, + const struct sys_reg_desc table[], + unsigned int num); #define Op0(_x) .Op0 = _x #define Op1(_x) .Op1 = _x -- 1.7.9.5 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [RFC PATCH v1 4/4] arm/arm64: vgic-new: Implement VGICv3 CPU interface access 2016-07-20 13:02 [RFC PATCH v1 0/4] arm/arm64: vgic-new: Implement API for vGICv3 live migration vijay.kilari at gmail.com ` (2 preceding siblings ...) 2016-07-20 13:02 ` [RFC PATCH v1 3/4] arm/arm64: vgic-new: Introduce find_reg_by_id() vijay.kilari at gmail.com @ 2016-07-20 13:02 ` vijay.kilari at gmail.com 2016-08-02 14:46 ` Christoffer Dall 3 siblings, 1 reply; 10+ messages in thread From: vijay.kilari at gmail.com @ 2016-07-20 13:02 UTC (permalink / raw) To: linux-arm-kernel From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> CPU sysregs access size is always 64 bits. MPIDR value passed along with reg id is used to identify the CPU. Signed-off-by: Pavel Fedin <p.fedin@samsung.com> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> --- arch/arm64/include/uapi/asm/kvm.h | 9 ++ arch/arm64/kvm/Makefile | 1 + include/linux/irqchip/arm-gic-v3.h | 4 + virt/kvm/arm/vgic/vgic-kvm-device.c | 31 +++++ virt/kvm/arm/vgic/vgic-mmio-v2.c | 4 +- virt/kvm/arm/vgic/vgic-mmio-v3.c | 6 + virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 225 +++++++++++++++++++++++++++++++++++ virt/kvm/arm/vgic/vgic.h | 6 + 8 files changed, 284 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index a6b996e..35c4a99 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -206,7 +206,16 @@ struct kvm_arch_memory_slot { #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3 #define KVM_DEV_ARM_VGIC_GRP_CTRL 4 #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6 + #define KVM_DEV_ARM_VGIC_CTRL_INIT 0 +#define KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \ + KVM_REG_ARM64_SYSREG_OP1_MASK | \ + KVM_REG_ARM64_SYSREG_CRN_MASK | \ + KVM_REG_ARM64_SYSREG_CRM_MASK | \ + KVM_REG_ARM64_SYSREG_OP2_MASK) +#define KVM_DEV_ARM_VGIC_SYSREG(op0, op1, crn, crm, op2) \ + __ARM64_SYS_REG(op0, op1, crn, crm, op2) /* Device Control API on vcpu fd */ #define KVM_ARM_VCPU_PMU_V3_CTRL 0 diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index f00b2cd..53400c1 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -29,5 +29,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h index bfbd707..ae84cde 100644 --- a/include/linux/irqchip/arm-gic-v3.h +++ b/include/linux/irqchip/arm-gic-v3.h @@ -288,6 +288,10 @@ #define ICH_VMCR_CTLR_SHIFT 0 #define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT) +#define ICH_VMCR_ENG0_SHIFT 0 +#define ICH_VMCR_ENG0 (1 << ICH_VMCR_ENG0_SHIFT) +#define ICH_VMCR_ENG1_SHIFT 1 +#define ICH_VMCR_ENG1 (1 << ICH_VMCR_ENG1_SHIFT) #define ICH_VMCR_BPR1_SHIFT 18 #define ICH_VMCR_BPR1_MASK (7 << ICH_VMCR_BPR1_SHIFT) #define ICH_VMCR_BPR0_SHIFT 21 diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c index 996a720..78c50cc 100644 --- a/virt/kvm/arm/vgic/vgic-kvm-device.c +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c @@ -266,6 +266,7 @@ static int vgic_attr_regs_access(struct kvm_device *dev, int cpuid, ret, c; struct kvm_vcpu *vcpu, *tmp_vcpu; int vcpu_lock_idx = -1; + u64 regid; struct vgic_dist *vgic = &dev->kvm->arch.vgic; if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) { @@ -322,6 +323,16 @@ static int vgic_attr_regs_access(struct kvm_device *dev, else ret = -EINVAL; break; + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) { + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_MASK) | + KVM_REG_SIZE_U64; + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write, + regid, ®->reg64); + } else { + ret = -EINVAL; + } + break; default: ret = -EINVAL; break; @@ -447,6 +458,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev, return vgic_attr_regs_access(dev, attr, ®, true); } + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: { + u64 __user *uaddr = (u64 __user *)(long)attr->addr; + union ureg reg; + + if (get_user(reg.reg64, uaddr)) + return -EFAULT; + + return vgic_attr_regs_access(dev, attr, ®, true); + } } return -ENXIO; } @@ -472,6 +492,16 @@ static int vgic_v3_get_attr(struct kvm_device *dev, ret = put_user(reg.reg32, uaddr); return ret; } + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: { + u64 __user *uaddr = (u64 __user *)(long)attr->addr; + union ureg reg; + + ret = vgic_attr_regs_access(dev, attr, ®, false); + if (ret) + return ret; + ret = put_user(reg.reg64, uaddr); + return ret; + } } return -ENXIO; @@ -490,6 +520,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev, break; case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: return vgic_v3_has_attr_regs(dev, attr); case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: return 0; diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c index a213936..97d94b2 100644 --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c @@ -204,7 +204,7 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu, } } -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) { if (kvm_vgic_global_state.type == VGIC_V2) vgic_v2_set_vmcr(vcpu, vmcr); @@ -212,7 +212,7 @@ static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) vgic_v3_set_vmcr(vcpu, vmcr); } -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) { if (kvm_vgic_global_state.type == VGIC_V2) vgic_v2_get_vmcr(vcpu, vmcr); diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c index f6a4e97..aa63f39 100644 --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c @@ -23,6 +23,7 @@ #include "vgic.h" #include "vgic-mmio.h" +#include "sys_regs.h" /* extract @num bytes at @offset bytes offset in data */ static unsigned long extract_bytes(unsigned long data, unsigned int offset, @@ -382,6 +383,11 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr) nr_regions = rd_dev->nr_regions; break; } + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: { + u64 reg; + + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, cpuid, ®); + } default: return -ENXIO; } diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c new file mode 100644 index 0000000..d877cd8 --- /dev/null +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c @@ -0,0 +1,225 @@ +#include <linux/irqchip/arm-gic-v3.h> +#include <linux/kvm.h> +#include <linux/kvm_host.h> +#include <kvm/iodev.h> +#include <kvm/arm_vgic.h> +#include <asm/kvm_emulate.h> +#include <asm/kvm_arm.h> +#include <asm/kvm_mmu.h> + +#include "vgic.h" +#include "vgic-mmio.h" +#include "sys_regs.h" + +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_vmcr vmcr; + + vgic_get_vmcr(vcpu, &vmcr); + if (p->is_write) { + vmcr.ctlr = (u32)p->regval; + vgic_set_vmcr(vcpu, &vmcr); + } else { + p->regval = vmcr.ctlr; + } + + return true; +} + +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_vmcr vmcr; + + vgic_get_vmcr(vcpu, &vmcr); + if (p->is_write) { + vmcr.pmr = (u32)p->regval; + vgic_set_vmcr(vcpu, &vmcr); + } else { + p->regval = vmcr.pmr; + } + + return true; +} + +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_vmcr vmcr; + + vgic_get_vmcr(vcpu, &vmcr); + if (p->is_write) { + vmcr.bpr = (u32)p->regval; + vgic_set_vmcr(vcpu, &vmcr); + } else { + p->regval = vmcr.bpr; + } + + return true; +} + +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_vmcr vmcr; + + vgic_get_vmcr(vcpu, &vmcr); + if (p->is_write) { + vmcr.abpr = (u32)p->regval; + vgic_set_vmcr(vcpu, &vmcr); + } else { + p->regval = vmcr.abpr; + } + + return true; +} + +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; + + if (p->is_write) { + vgicv3->vgic_vmcr &= ~ICH_VMCR_ENG0; + vgicv3->vgic_vmcr |= (p->regval << ICH_VMCR_ENG0_SHIFT) & + ICH_VMCR_ENG0; + } else { + p->regval = (vgicv3->vgic_vmcr & ICH_VMCR_ENG0) >> + ICH_VMCR_ENG0_SHIFT; + } + + return true; +} + +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; + + if (p->is_write) { + vgicv3->vgic_vmcr &= ~ICH_VMCR_ENG1; + vgicv3->vgic_vmcr |= (p->regval << ICH_VMCR_ENG1_SHIFT) & + ICH_VMCR_ENG1; + } else { + p->regval = (vgicv3->vgic_vmcr & ICH_VMCR_ENG1) >> + ICH_VMCR_ENG1_SHIFT; + } + + return true; +} + +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; + u8 idx = r->Op2 & 3; + + if (p->is_write) + vgicv3->vgic_ap0r[idx] = p->regval; + else + p->regval = vgicv3->vgic_ap0r[idx]; + + return true; +} + +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; + u8 idx = r->Op2 & 3; + + if (p->is_write) + vgicv3->vgic_ap1r[idx] = p->regval; + else + p->regval = vgicv3->vgic_ap1r[idx]; + + return true; +} + +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = { + /* ICC_PMR_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b0100), CRm(0b0110), Op2(0b000), + access_gic_pmr }, + /* ICC_BPR0_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b011), + access_gic_bpr0 }, + /* ICC_AP0R0_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b100), + access_gic_ap0r }, + /* ICC_AP0R1_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b101), + access_gic_ap0r }, + /* ICC_AP0R2_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b110), + access_gic_ap0r }, + /* ICC_AP0R3_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b111), + access_gic_ap0r }, + /* ICC_AP1R0_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b000), + access_gic_ap1r }, + /* ICC_AP1R1_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b001), + access_gic_ap1r }, + /* ICC_AP1R2_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b010), + access_gic_ap1r }, + /* ICC_AP1R3_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b011), + access_gic_ap1r }, + /* ICC_BPR1_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b011), + access_gic_bpr1 }, + /* ICC_CTLR_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b100), + access_gic_ctlr }, + /* ICC_IGRPEN0_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b110), + access_gic_grpen0 }, + /* ICC_GRPEN1_EL1 */ + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b111), + access_gic_grpen1 }, +}; + +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id, + u64 *reg) +{ + struct sys_reg_params params; + + params.regval = le64_to_cpu(*reg); + params.is_write = is_write; + params.is_aarch32 = false; + params.is_32bit = false; + + return find_reg_by_id(id, ¶ms, gic_v3_icc_reg_descs, + ARRAY_SIZE(gic_v3_icc_reg_descs)) ? + 0 : -ENXIO; +} + +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id, + u64 *reg) +{ + struct sys_reg_params params; + const struct sys_reg_desc *r; + + if (is_write) + params.regval = le64_to_cpu(*reg); + params.is_write = is_write; + params.is_aarch32 = false; + params.is_32bit = false; + + r = find_reg_by_id(id, ¶ms, gic_v3_icc_reg_descs, + ARRAY_SIZE(gic_v3_icc_reg_descs)); + if (!r) + return -ENXIO; + + if (!r->access(vcpu, ¶ms, r)) + return -EINVAL; + + if (!is_write) + *reg = cpu_to_le64(params.regval); + + return 0; +} + diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h index 8637690..23db38d 100644 --- a/virt/kvm/arm/vgic/vgic.h +++ b/virt/kvm/arm/vgic/vgic.h @@ -79,6 +79,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, int offset, u32 *val); int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, int offset, u32 *val); +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, + u64 id, u64 *val); +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id, + u64 *reg); #else static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu) { @@ -132,6 +136,8 @@ static inline int vgic_register_redist_iodevs(struct kvm *kvm, } #endif +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr); +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr); void kvm_register_vgic_device(unsigned long type); int vgic_lazy_init(struct kvm *kvm); int vgic_init(struct kvm *kvm); -- 1.7.9.5 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [RFC PATCH v1 4/4] arm/arm64: vgic-new: Implement VGICv3 CPU interface access 2016-07-20 13:02 ` [RFC PATCH v1 4/4] arm/arm64: vgic-new: Implement VGICv3 CPU interface access vijay.kilari at gmail.com @ 2016-08-02 14:46 ` Christoffer Dall 0 siblings, 0 replies; 10+ messages in thread From: Christoffer Dall @ 2016-08-02 14:46 UTC (permalink / raw) To: linux-arm-kernel On Wed, Jul 20, 2016 at 06:32:28PM +0530, vijay.kilari at gmail.com wrote: > From: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > > CPU sysregs access size is always 64 bits. MPIDR value > passed along with reg id is used to identify the CPU. Please provide a decent commit message along with a 200+ lines patch. Thanks, -Christoffer > > Signed-off-by: Pavel Fedin <p.fedin@samsung.com> > Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com> > --- > arch/arm64/include/uapi/asm/kvm.h | 9 ++ > arch/arm64/kvm/Makefile | 1 + > include/linux/irqchip/arm-gic-v3.h | 4 + > virt/kvm/arm/vgic/vgic-kvm-device.c | 31 +++++ > virt/kvm/arm/vgic/vgic-mmio-v2.c | 4 +- > virt/kvm/arm/vgic/vgic-mmio-v3.c | 6 + > virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 225 +++++++++++++++++++++++++++++++++++ > virt/kvm/arm/vgic/vgic.h | 6 + > 8 files changed, 284 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h > index a6b996e..35c4a99 100644 > --- a/arch/arm64/include/uapi/asm/kvm.h > +++ b/arch/arm64/include/uapi/asm/kvm.h > @@ -206,7 +206,16 @@ struct kvm_arch_memory_slot { > #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3 > #define KVM_DEV_ARM_VGIC_GRP_CTRL 4 > #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5 > +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6 > + > #define KVM_DEV_ARM_VGIC_CTRL_INIT 0 > +#define KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \ > + KVM_REG_ARM64_SYSREG_OP1_MASK | \ > + KVM_REG_ARM64_SYSREG_CRN_MASK | \ > + KVM_REG_ARM64_SYSREG_CRM_MASK | \ > + KVM_REG_ARM64_SYSREG_OP2_MASK) > +#define KVM_DEV_ARM_VGIC_SYSREG(op0, op1, crn, crm, op2) \ > + __ARM64_SYS_REG(op0, op1, crn, crm, op2) > > /* Device Control API on vcpu fd */ > #define KVM_ARM_VCPU_PMU_V3_CTRL 0 > diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile > index f00b2cd..53400c1 100644 > --- a/arch/arm64/kvm/Makefile > +++ b/arch/arm64/kvm/Makefile > @@ -29,5 +29,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio.o > kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v2.o > kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o > kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o > +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o > kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o > kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o > diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h > index bfbd707..ae84cde 100644 > --- a/include/linux/irqchip/arm-gic-v3.h > +++ b/include/linux/irqchip/arm-gic-v3.h > @@ -288,6 +288,10 @@ > > #define ICH_VMCR_CTLR_SHIFT 0 > #define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT) > +#define ICH_VMCR_ENG0_SHIFT 0 > +#define ICH_VMCR_ENG0 (1 << ICH_VMCR_ENG0_SHIFT) > +#define ICH_VMCR_ENG1_SHIFT 1 > +#define ICH_VMCR_ENG1 (1 << ICH_VMCR_ENG1_SHIFT) > #define ICH_VMCR_BPR1_SHIFT 18 > #define ICH_VMCR_BPR1_MASK (7 << ICH_VMCR_BPR1_SHIFT) > #define ICH_VMCR_BPR0_SHIFT 21 > diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c > index 996a720..78c50cc 100644 > --- a/virt/kvm/arm/vgic/vgic-kvm-device.c > +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c > @@ -266,6 +266,7 @@ static int vgic_attr_regs_access(struct kvm_device *dev, > int cpuid, ret, c; > struct kvm_vcpu *vcpu, *tmp_vcpu; > int vcpu_lock_idx = -1; > + u64 regid; > struct vgic_dist *vgic = &dev->kvm->arch.vgic; > > if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V2) { > @@ -322,6 +323,16 @@ static int vgic_attr_regs_access(struct kvm_device *dev, > else > ret = -EINVAL; > break; > + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: > + if (vgic->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) { > + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_MASK) | > + KVM_REG_SIZE_U64; > + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write, > + regid, ®->reg64); > + } else { > + ret = -EINVAL; > + } > + break; > default: > ret = -EINVAL; > break; > @@ -447,6 +458,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev, > > return vgic_attr_regs_access(dev, attr, ®, true); > } > + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: { > + u64 __user *uaddr = (u64 __user *)(long)attr->addr; > + union ureg reg; > + > + if (get_user(reg.reg64, uaddr)) > + return -EFAULT; > + > + return vgic_attr_regs_access(dev, attr, ®, true); > + } > } > return -ENXIO; > } > @@ -472,6 +492,16 @@ static int vgic_v3_get_attr(struct kvm_device *dev, > ret = put_user(reg.reg32, uaddr); > return ret; > } > + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: { > + u64 __user *uaddr = (u64 __user *)(long)attr->addr; > + union ureg reg; > + > + ret = vgic_attr_regs_access(dev, attr, ®, false); > + if (ret) > + return ret; > + ret = put_user(reg.reg64, uaddr); > + return ret; > + } > } > > return -ENXIO; > @@ -490,6 +520,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev, > break; > case KVM_DEV_ARM_VGIC_GRP_DIST_REGS: > case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: > + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: > return vgic_v3_has_attr_regs(dev, attr); > case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: > return 0; > diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c > index a213936..97d94b2 100644 > --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c > +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c > @@ -204,7 +204,7 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu, > } > } > > -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) > +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) > { > if (kvm_vgic_global_state.type == VGIC_V2) > vgic_v2_set_vmcr(vcpu, vmcr); > @@ -212,7 +212,7 @@ static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) > vgic_v3_set_vmcr(vcpu, vmcr); > } > > -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) > +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr) > { > if (kvm_vgic_global_state.type == VGIC_V2) > vgic_v2_get_vmcr(vcpu, vmcr); > diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c > index f6a4e97..aa63f39 100644 > --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c > +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c > @@ -23,6 +23,7 @@ > > #include "vgic.h" > #include "vgic-mmio.h" > +#include "sys_regs.h" > > /* extract @num bytes at @offset bytes offset in data */ > static unsigned long extract_bytes(unsigned long data, unsigned int offset, > @@ -382,6 +383,11 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr) > nr_regions = rd_dev->nr_regions; > break; > } > + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: { > + u64 reg; > + > + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, cpuid, ®); > + } > default: > return -ENXIO; > } > diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c > new file mode 100644 > index 0000000..d877cd8 > --- /dev/null > +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c > @@ -0,0 +1,225 @@ > +#include <linux/irqchip/arm-gic-v3.h> > +#include <linux/kvm.h> > +#include <linux/kvm_host.h> > +#include <kvm/iodev.h> > +#include <kvm/arm_vgic.h> > +#include <asm/kvm_emulate.h> > +#include <asm/kvm_arm.h> > +#include <asm/kvm_mmu.h> > + > +#include "vgic.h" > +#include "vgic-mmio.h" > +#include "sys_regs.h" > + > +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_vmcr vmcr; > + > + vgic_get_vmcr(vcpu, &vmcr); > + if (p->is_write) { > + vmcr.ctlr = (u32)p->regval; > + vgic_set_vmcr(vcpu, &vmcr); > + } else { > + p->regval = vmcr.ctlr; > + } > + > + return true; > +} > + > +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_vmcr vmcr; > + > + vgic_get_vmcr(vcpu, &vmcr); > + if (p->is_write) { > + vmcr.pmr = (u32)p->regval; > + vgic_set_vmcr(vcpu, &vmcr); > + } else { > + p->regval = vmcr.pmr; > + } > + > + return true; > +} > + > +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_vmcr vmcr; > + > + vgic_get_vmcr(vcpu, &vmcr); > + if (p->is_write) { > + vmcr.bpr = (u32)p->regval; > + vgic_set_vmcr(vcpu, &vmcr); > + } else { > + p->regval = vmcr.bpr; > + } > + > + return true; > +} > + > +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_vmcr vmcr; > + > + vgic_get_vmcr(vcpu, &vmcr); > + if (p->is_write) { > + vmcr.abpr = (u32)p->regval; > + vgic_set_vmcr(vcpu, &vmcr); > + } else { > + p->regval = vmcr.abpr; > + } > + > + return true; > +} > + > +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; > + > + if (p->is_write) { > + vgicv3->vgic_vmcr &= ~ICH_VMCR_ENG0; > + vgicv3->vgic_vmcr |= (p->regval << ICH_VMCR_ENG0_SHIFT) & > + ICH_VMCR_ENG0; > + } else { > + p->regval = (vgicv3->vgic_vmcr & ICH_VMCR_ENG0) >> > + ICH_VMCR_ENG0_SHIFT; > + } > + > + return true; > +} > + > +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; > + > + if (p->is_write) { > + vgicv3->vgic_vmcr &= ~ICH_VMCR_ENG1; > + vgicv3->vgic_vmcr |= (p->regval << ICH_VMCR_ENG1_SHIFT) & > + ICH_VMCR_ENG1; > + } else { > + p->regval = (vgicv3->vgic_vmcr & ICH_VMCR_ENG1) >> > + ICH_VMCR_ENG1_SHIFT; > + } > + > + return true; > +} > + > +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; > + u8 idx = r->Op2 & 3; > + > + if (p->is_write) > + vgicv3->vgic_ap0r[idx] = p->regval; > + else > + p->regval = vgicv3->vgic_ap0r[idx]; > + > + return true; > +} > + > +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p, > + const struct sys_reg_desc *r) > +{ > + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3; > + u8 idx = r->Op2 & 3; > + > + if (p->is_write) > + vgicv3->vgic_ap1r[idx] = p->regval; > + else > + p->regval = vgicv3->vgic_ap1r[idx]; > + > + return true; > +} > + > +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = { > + /* ICC_PMR_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b0100), CRm(0b0110), Op2(0b000), > + access_gic_pmr }, > + /* ICC_BPR0_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b011), > + access_gic_bpr0 }, > + /* ICC_AP0R0_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b100), > + access_gic_ap0r }, > + /* ICC_AP0R1_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b101), > + access_gic_ap0r }, > + /* ICC_AP0R2_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b110), > + access_gic_ap0r }, > + /* ICC_AP0R3_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1000), Op2(0b111), > + access_gic_ap0r }, > + /* ICC_AP1R0_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b000), > + access_gic_ap1r }, > + /* ICC_AP1R1_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b001), > + access_gic_ap1r }, > + /* ICC_AP1R2_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b010), > + access_gic_ap1r }, > + /* ICC_AP1R3_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1001), Op2(0b011), > + access_gic_ap1r }, > + /* ICC_BPR1_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b011), > + access_gic_bpr1 }, > + /* ICC_CTLR_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b100), > + access_gic_ctlr }, > + /* ICC_IGRPEN0_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b110), > + access_gic_grpen0 }, > + /* ICC_GRPEN1_EL1 */ > + { Op0(0b11), Op1(0b000), CRn(0b1100), CRm(0b1100), Op2(0b111), > + access_gic_grpen1 }, > +}; > + > +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id, > + u64 *reg) > +{ > + struct sys_reg_params params; > + > + params.regval = le64_to_cpu(*reg); > + params.is_write = is_write; > + params.is_aarch32 = false; > + params.is_32bit = false; > + > + return find_reg_by_id(id, ¶ms, gic_v3_icc_reg_descs, > + ARRAY_SIZE(gic_v3_icc_reg_descs)) ? > + 0 : -ENXIO; > +} > + > +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id, > + u64 *reg) > +{ > + struct sys_reg_params params; > + const struct sys_reg_desc *r; > + > + if (is_write) > + params.regval = le64_to_cpu(*reg); > + params.is_write = is_write; > + params.is_aarch32 = false; > + params.is_32bit = false; > + > + r = find_reg_by_id(id, ¶ms, gic_v3_icc_reg_descs, > + ARRAY_SIZE(gic_v3_icc_reg_descs)); > + if (!r) > + return -ENXIO; > + > + if (!r->access(vcpu, ¶ms, r)) > + return -EINVAL; > + > + if (!is_write) > + *reg = cpu_to_le64(params.regval); > + > + return 0; > +} > + > diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h > index 8637690..23db38d 100644 > --- a/virt/kvm/arm/vgic/vgic.h > +++ b/virt/kvm/arm/vgic/vgic.h > @@ -79,6 +79,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > int offset, u32 *val); > int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write, > int offset, u32 *val); > +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, > + u64 id, u64 *val); > +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id, > + u64 *reg); > #else > static inline void vgic_v3_process_maintenance(struct kvm_vcpu *vcpu) > { > @@ -132,6 +136,8 @@ static inline int vgic_register_redist_iodevs(struct kvm *kvm, > } > #endif > > +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr); > +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr); > void kvm_register_vgic_device(unsigned long type); > int vgic_lazy_init(struct kvm *kvm); > int vgic_init(struct kvm *kvm); > -- > 1.7.9.5 > ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2016-08-03 8:42 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-07-20 13:02 [RFC PATCH v1 0/4] arm/arm64: vgic-new: Implement API for vGICv3 live migration vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 1/4] arm/arm64: vgic-new: Introduce 64-bit reg access support vijay.kilari at gmail.com 2016-07-20 14:02 ` Marc Zyngier 2016-07-20 13:02 ` [RFC PATCH v1 2/4] arm/arm64: vgic-new: Add distributor and redistributor access vijay.kilari at gmail.com 2016-08-02 14:43 ` Christoffer Dall 2016-08-03 8:33 ` Vijay Kilari 2016-08-03 8:42 ` Christoffer Dall 2016-07-20 13:02 ` [RFC PATCH v1 3/4] arm/arm64: vgic-new: Introduce find_reg_by_id() vijay.kilari at gmail.com 2016-07-20 13:02 ` [RFC PATCH v1 4/4] arm/arm64: vgic-new: Implement VGICv3 CPU interface access vijay.kilari at gmail.com 2016-08-02 14:46 ` Christoffer Dall
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).