* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
@ 2013-01-08 18:41 ` Christoffer Dall
2013-01-08 22:36 ` Scott Wood
2013-01-08 18:41 ` [PATCH v5 02/12] ARM: KVM: Keep track of currently running vcpus Christoffer Dall
` (11 subsequent siblings)
12 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:41 UTC (permalink / raw)
To: linux-arm-kernel
On ARM (and possibly other architectures) some bits are specific to the
model being emulated for the guest and user space needs a way to tell
the kernel about those bits. An example is mmio device base addresses,
where KVM must know the base address for a given device to properly
emulate mmio accesses within a certain address range or directly map a
device with virtualiation extensions into the guest address space.
We try to make this API slightly more generic than for our specific use,
but so far only the VGIC uses this feature.
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
Documentation/virtual/kvm/api.txt | 37 +++++++++++++++++++++++++++++++++++++
arch/arm/include/uapi/asm/kvm.h | 13 +++++++++++++
arch/arm/kvm/arm.c | 23 ++++++++++++++++++++++-
include/uapi/linux/kvm.h | 8 ++++++++
4 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 38066a7a..668956f 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2206,6 +2206,43 @@ This ioctl returns the guest registers that are supported for the
KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
+4.80 KVM_SET_DEVICE_ADDRESS
+
+Capability: KVM_CAP_SET_DEVICE_ADDRESS
+Architectures: arm
+Type: vm ioctl
+Parameters: struct kvm_device_address (in)
+Returns: 0 on success, -1 on error
+Errors:
+ ENODEV: The device id is unknown
+ ENXIO: Device not supported on current system
+ EEXIST: Address already set
+ E2BIG: Address outside guest physical address space
+
+struct kvm_device_address {
+ __u64 id;
+ __u64 addr;
+};
+
+Specify a device address in the guest's physical address space where guests
+can access emulated or directly exposed devices, which the host kernel needs
+to know about. The id field is an architecture specific identifier for a
+specific device.
+
+ARM divides the id field into two parts, a device id and an address type id
+specific to the individual device.
+
+ ?bits: | 63 ... 32 | 31 ... 16 | 15 ... 0 |
+ field: | 0x00000000 | device id | addr type id |
+
+ARM currently only require this when using the in-kernel GIC support for the
+hardware VGIC features, using KVM_ARM_DEVICE_VGIC_V2 as the device id. When
+setting the base address for the guest's mapping of the VGIC virtual CPU
+and distributor interface, the ioctl must be called after calling
+KVM_CREATE_IRQCHIP, but before calling KVM_RUN on any of the VCPUs. Calling
+this ioctl twice for any of the base addresses will return -EEXIST.
+
+
5. The kvm_run structure
------------------------
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 73b9615..09911a7 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -65,6 +65,19 @@ struct kvm_regs {
#define KVM_ARM_TARGET_CORTEX_A15 0
#define KVM_ARM_NUM_TARGETS 1
+/* KVM_SET_DEVICE_ADDRESS ioctl id encoding */
+#define KVM_DEVICE_TYPE_SHIFT 0
+#define KVM_DEVICE_TYPE_MASK (0xffff << KVM_DEVICE_TYPE_SHIFT)
+#define KVM_DEVICE_ID_SHIFT 16
+#define KVM_DEVICE_ID_MASK (0xffff << KVM_DEVICE_ID_SHIFT)
+
+/* Supported device IDs */
+#define KVM_ARM_DEVICE_VGIC_V2 0
+
+/* Supported VGIC address types */
+#define KVM_VGIC_V2_ADDR_TYPE_DIST 0
+#define KVM_VGIC_V2_ADDR_TYPE_CPU 1
+
struct kvm_vcpu_init {
__u32 target;
__u32 features[7];
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index f42d828..2f39b04 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -165,6 +165,8 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_COALESCED_MMIO:
r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break;
+ case KVM_CAP_SET_DEVICE_ADDR:
+ r = 1;
case KVM_CAP_NR_VCPUS:
r = num_online_cpus();
break;
@@ -805,10 +807,29 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
return -EINVAL;
}
+static int kvm_vm_ioctl_set_device_address(struct kvm *kvm,
+ struct kvm_device_address *dev_addr)
+{
+ return -ENODEV;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
- return -EINVAL;
+ struct kvm *kvm = filp->private_data;
+ void __user *argp = (void __user *)arg;
+
+ switch (ioctl) {
+ case KVM_SET_DEVICE_ADDRESS: {
+ struct kvm_device_address dev_addr;
+
+ if (copy_from_user(&dev_addr, argp, sizeof(dev_addr)))
+ return -EFAULT;
+ return kvm_vm_ioctl_set_device_address(kvm, &dev_addr);
+ }
+ default:
+ return -EINVAL;
+ }
}
static void cpu_init_hyp_mode(void *vector)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index dc63665..03248d1 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -636,6 +636,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_IRQFD_RESAMPLE 82
#define KVM_CAP_PPC_BOOKE_WATCHDOG 83
#define KVM_CAP_PPC_HTAB_FD 84
+#define KVM_CAP_SET_DEVICE_ADDR 85
#ifdef KVM_CAP_IRQ_ROUTING
@@ -783,6 +784,11 @@ struct kvm_msi {
__u8 pad[16];
};
+struct kvm_device_address {
+ __u64 id;
+ __u64 addr;
+};
+
/*
* ioctls for VM fds
*/
@@ -868,6 +874,8 @@ struct kvm_s390_ucas_mapping {
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
/* Available with KVM_CAP_PPC_HTAB_FD */
#define KVM_PPC_GET_HTAB_FD _IOW(KVMIO, 0xaa, struct kvm_get_htab_fd)
+/* Available with KVM_CAP_SET_DEVICE_ADDR */
+#define KVM_SET_DEVICE_ADDRESS _IOW(KVMIO, 0xab, struct kvm_device_address)
/*
* ioctls for vcpu fds
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 18:41 ` [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl Christoffer Dall
@ 2013-01-08 22:36 ` Scott Wood
2013-01-08 23:17 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Scott Wood @ 2013-01-08 22:36 UTC (permalink / raw)
To: linux-arm-kernel
On 01/08/2013 12:41:30 PM, Christoffer Dall wrote:
> On ARM (and possibly other architectures) some bits are specific to
> the
> model being emulated for the guest and user space needs a way to tell
> the kernel about those bits. An example is mmio device base
> addresses,
> where KVM must know the base address for a given device to properly
> emulate mmio accesses within a certain address range or directly map a
> device with virtualiation extensions into the guest address space.
>
> We try to make this API slightly more generic than for our specific
> use,
> but so far only the VGIC uses this feature.
>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
> ---
> Documentation/virtual/kvm/api.txt | 37
> +++++++++++++++++++++++++++++++++++++
> arch/arm/include/uapi/asm/kvm.h | 13 +++++++++++++
> arch/arm/kvm/arm.c | 23 ++++++++++++++++++++++-
> include/uapi/linux/kvm.h | 8 ++++++++
> 4 files changed, 80 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/virtual/kvm/api.txt
> b/Documentation/virtual/kvm/api.txt
> index 38066a7a..668956f 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -2206,6 +2206,43 @@ This ioctl returns the guest registers that
> are supported for the
> KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
>
>
> +4.80 KVM_SET_DEVICE_ADDRESS
> +
> +Capability: KVM_CAP_SET_DEVICE_ADDRESS
> +Architectures: arm
> +Type: vm ioctl
> +Parameters: struct kvm_device_address (in)
> +Returns: 0 on success, -1 on error
> +Errors:
> + ENODEV: The device id is unknown
> + ENXIO: Device not supported on current system
> + EEXIST: Address already set
> + E2BIG: Address outside guest physical address space
> +
> +struct kvm_device_address {
> + __u64 id;
> + __u64 addr;
> +};
What about this is really specific to addresses? Can't we set other
device parameters this way?
Sort of like a device equivalent of PPC's one-reg interface.
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 22:36 ` Scott Wood
@ 2013-01-08 23:17 ` Christoffer Dall
2013-01-08 23:29 ` Scott Wood
0 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 23:17 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Jan 8, 2013 at 5:36 PM, Scott Wood <scottwood@freescale.com> wrote:
> On 01/08/2013 12:41:30 PM, Christoffer Dall wrote:
>>
>> On ARM (and possibly other architectures) some bits are specific to the
>> model being emulated for the guest and user space needs a way to tell
>> the kernel about those bits. An example is mmio device base addresses,
>> where KVM must know the base address for a given device to properly
>> emulate mmio accesses within a certain address range or directly map a
>> device with virtualiation extensions into the guest address space.
>>
>> We try to make this API slightly more generic than for our specific use,
>> but so far only the VGIC uses this feature.
>>
>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>> ---
>> Documentation/virtual/kvm/api.txt | 37
>> +++++++++++++++++++++++++++++++++++++
>> arch/arm/include/uapi/asm/kvm.h | 13 +++++++++++++
>> arch/arm/kvm/arm.c | 23 ++++++++++++++++++++++-
>> include/uapi/linux/kvm.h | 8 ++++++++
>> 4 files changed, 80 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/virtual/kvm/api.txt
>> b/Documentation/virtual/kvm/api.txt
>> index 38066a7a..668956f 100644
>> --- a/Documentation/virtual/kvm/api.txt
>> +++ b/Documentation/virtual/kvm/api.txt
>> @@ -2206,6 +2206,43 @@ This ioctl returns the guest registers that are
>> supported for the
>> KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
>>
>>
>> +4.80 KVM_SET_DEVICE_ADDRESS
>> +
>> +Capability: KVM_CAP_SET_DEVICE_ADDRESS
>> +Architectures: arm
>> +Type: vm ioctl
>> +Parameters: struct kvm_device_address (in)
>> +Returns: 0 on success, -1 on error
>> +Errors:
>> + ENODEV: The device id is unknown
>> + ENXIO: Device not supported on current system
>> + EEXIST: Address already set
>> + E2BIG: Address outside guest physical address space
>> +
>> +struct kvm_device_address {
>> + __u64 id;
>> + __u64 addr;
>> +};
>
>
> What about this is really specific to addresses? Can't we set other device
> parameters this way?
>
> Sort of like a device equivalent of PPC's one-reg interface.
>
This has been discussed a number of times, and one or the other there
is a need for userspace to tell KVM to present memory-mapped devices
at a given address. It was also considered to make this specific to
irqchip initialization, but irqchips are different and a lot of that
code is x86-specific, so that approach was discarded.
This *could* look something like this:
struct kvm_device_param {
u64 dev_id;
u64 param_id;
u64 value;
};
but that has less clear, or at least less specific, semantics.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 23:17 ` Christoffer Dall
@ 2013-01-08 23:29 ` Scott Wood
2013-01-08 23:49 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Scott Wood @ 2013-01-08 23:29 UTC (permalink / raw)
To: linux-arm-kernel
On 01/08/2013 05:17:05 PM, Christoffer Dall wrote:
> On Tue, Jan 8, 2013 at 5:36 PM, Scott Wood <scottwood@freescale.com>
> wrote:
> > On 01/08/2013 12:41:30 PM, Christoffer Dall wrote:
> >> +struct kvm_device_address {
> >> + __u64 id;
> >> + __u64 addr;
> >> +};
> >
> >
> > What about this is really specific to addresses? Can't we set
> other device
> > parameters this way?
> >
> > Sort of like a device equivalent of PPC's one-reg interface.
> >
> This has been discussed a number of times,
Sorry, this patch was just pointed out to me today. I googled the
patch title but couldn't find this discussion.
> and one or the other there
> is a need for userspace to tell KVM to present memory-mapped devices
> at a given address. It was also considered to make this specific to
> irqchip initialization, but irqchips are different and a lot of that
> code is x86-specific, so that approach was discarded.
>
> This *could* look something like this:
>
> struct kvm_device_param {
> u64 dev_id;
> u64 param_id;
> u64 value;
> };
>
> but that has less clear, or at least less specific, semantics.
Why is it less clear? You need to have device-specific documentation
for what "id" means, so why not also an enumeration of "param"s? Or
just keep it as is, and rename "address" to "value". Whether "dev" and
"param" are combined is orthogonal from whether it's used for
non-address things.
If you leave it as "address", either we'll have it being used for
non-address things regardless of the name ("Not a typewriter!"), or
there'll end up being yet more unnecessary ioctls, or device-specific
things will end up getting shoved into CPU interfaces such as one-reg.
For example, on MPIC we need to be able to specify the version of the
chip to emulate in addition to the address at which it lives.
Also, why is it documented as an "arm" interface? Shouldn't it be a
generic interface, with other architectures currently not implementing
any IDs? What in the kvm_arch_vm_ioctl() wrapper is arm-specific?
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 23:29 ` Scott Wood
@ 2013-01-08 23:49 ` Christoffer Dall
2013-01-09 0:12 ` Scott Wood
2013-01-09 10:02 ` Alexander Graf
0 siblings, 2 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 23:49 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Jan 8, 2013 at 6:29 PM, Scott Wood <scottwood@freescale.com> wrote:
> On 01/08/2013 05:17:05 PM, Christoffer Dall wrote:
>>
>> On Tue, Jan 8, 2013 at 5:36 PM, Scott Wood <scottwood@freescale.com>
>> wrote:
>> > On 01/08/2013 12:41:30 PM, Christoffer Dall wrote:
>> >> +struct kvm_device_address {
>> >> + __u64 id;
>> >> + __u64 addr;
>> >> +};
>> >
>> >
>> > What about this is really specific to addresses? Can't we set other
>> > device
>> > parameters this way?
>> >
>> > Sort of like a device equivalent of PPC's one-reg interface.
>> >
>> This has been discussed a number of times,
>
>
> Sorry, this patch was just pointed out to me today. I googled the patch
> title but couldn't find this discussion.
>
>
I believe it was mainly discussed at the KVM Forum in person.
>> and one or the other there
>> is a need for userspace to tell KVM to present memory-mapped devices
>> at a given address. It was also considered to make this specific to
>> irqchip initialization, but irqchips are different and a lot of that
>> code is x86-specific, so that approach was discarded.
>>
>> This *could* look something like this:
>>
>> struct kvm_device_param {
>> u64 dev_id;
>> u64 param_id;
>> u64 value;
>> };
>>
>> but that has less clear, or at least less specific, semantics.
>
>
> Why is it less clear? You need to have device-specific documentation for
> what "id" means, so why not also an enumeration of "param"s? Or just keep
> it as is, and rename "address" to "value". Whether "dev" and "param" are
> combined is orthogonal from whether it's used for non-address things.
less clear in the sense that you have to look at more code to see what
it does. I'm not saying that it's too unclear, at all, I'm actually
fine with it, but to make my point, we can make an ioctl that's called
do_something() that takes a struct with val0, val1, val2, val3, ...
>
> If you leave it as "address", either we'll have it being used for
> non-address things regardless of the name ("Not a typewriter!"), or there'll
> end up being yet more unnecessary ioctls, or device-specific things will end
> up getting shoved into CPU interfaces such as one-reg. For example, on MPIC
> we need to be able to specify the version of the chip to emulate in addition
> to the address at which it lives.
>
> Also, why is it documented as an "arm" interface? Shouldn't it be a generic
> interface, with other architectures currently not implementing any IDs?
> What in the kvm_arch_vm_ioctl() wrapper is arm-specific?
>
As I remember the argument for keeping this the point was that there
were other preferred methods for other archs to do this, and that ARM
was the only platform that had this explicit need, but maybe I'm
making this up.
I'll let Peter and Alex respond this as well, and if they're fine with
changing it to what I proposed above, then let's do that, and we can
make it a non arm-specific interface.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 23:49 ` Christoffer Dall
@ 2013-01-09 0:12 ` Scott Wood
2013-01-09 10:02 ` Alexander Graf
1 sibling, 0 replies; 79+ messages in thread
From: Scott Wood @ 2013-01-09 0:12 UTC (permalink / raw)
To: linux-arm-kernel
On 01/08/2013 05:49:40 PM, Christoffer Dall wrote:
> On Tue, Jan 8, 2013 at 6:29 PM, Scott Wood <scottwood@freescale.com>
> wrote:
> > On 01/08/2013 05:17:05 PM, Christoffer Dall wrote:
> >> This *could* look something like this:
> >>
> >> struct kvm_device_param {
> >> u64 dev_id;
> >> u64 param_id;
> >> u64 value;
> >> };
> >>
> >> but that has less clear, or at least less specific, semantics.
> >
> >
> > Why is it less clear? You need to have device-specific
> documentation for
> > what "id" means, so why not also an enumeration of "param"s? Or
> just keep
> > it as is, and rename "address" to "value". Whether "dev" and
> "param" are
> > combined is orthogonal from whether it's used for non-address
> things.
>
> less clear in the sense that you have to look at more code to see what
> it does. I'm not saying that it's too unclear, at all, I'm actually
> fine with it, but to make my point, we can make an ioctl that's called
> do_something() that takes a struct with val0, val1, val2, val3, ...
Such an IOCTL would add nothing other than trading the limited and
cumbersome ioctl namespace for something structured a bit differently
(which isn't such a bad thing...). A set device attribute ioctl would
constrain it to "take this number and convey it to the enumerated
device for the enumerated configuration purpose". There's already room
for device-specific semantics since you can have multiple address types.
Regarding the dev/param split, it looks like you're doing the split
anyway -- might as well make them separate struct fields rather than an
architecture-specific bitfield encoding.
> > If you leave it as "address", either we'll have it being used for
> > non-address things regardless of the name ("Not a typewriter!"), or
> there'll
> > end up being yet more unnecessary ioctls, or device-specific things
> will end
> > up getting shoved into CPU interfaces such as one-reg. For
> example, on MPIC
> > we need to be able to specify the version of the chip to emulate in
> addition
> > to the address at which it lives.
> >
> > Also, why is it documented as an "arm" interface? Shouldn't it be
> a generic
> > interface, with other architectures currently not implementing any
> IDs?
> > What in the kvm_arch_vm_ioctl() wrapper is arm-specific?
> >
> As I remember the argument for keeping this the point was that there
> were other preferred methods for other archs to do this, and that ARM
> was the only platform that had this explicit need, but maybe I'm
> making this up.
Well, at least PPC has this explicit need as well. :-)
Only the toplevel mechanism would be generic; it would be up to each
device to decide which (if any) configuration parameters it wants to
expose through it.
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-08 23:49 ` Christoffer Dall
2013-01-09 0:12 ` Scott Wood
@ 2013-01-09 10:02 ` Alexander Graf
2013-01-09 14:48 ` Peter Maydell
1 sibling, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 10:02 UTC (permalink / raw)
To: linux-arm-kernel
Am 09.01.2013 um 00:49 schrieb Christoffer Dall <c.dall@virtualopensystems.com>:
> On Tue, Jan 8, 2013 at 6:29 PM, Scott Wood <scottwood@freescale.com> wrote:
>> On 01/08/2013 05:17:05 PM, Christoffer Dall wrote:
>>>
>>> On Tue, Jan 8, 2013 at 5:36 PM, Scott Wood <scottwood@freescale.com>
>>> wrote:
>>>> On 01/08/2013 12:41:30 PM, Christoffer Dall wrote:
>>>>> +struct kvm_device_address {
>>>>> + __u64 id;
>>>>> + __u64 addr;
>>>>> +};
>>>>
>>>>
>>>> What about this is really specific to addresses? Can't we set other
>>>> device
>>>> parameters this way?
>>>>
>>>> Sort of like a device equivalent of PPC's one-reg interface.
>>>>
>>> This has been discussed a number of times,
>>
>>
>> Sorry, this patch was just pointed out to me today. I googled the patch
>> title but couldn't find this discussion.
>>
>>
>
> I believe it was mainly discussed at the KVM Forum in person.
>
>>> and one or the other there
>>> is a need for userspace to tell KVM to present memory-mapped devices
>>> at a given address. It was also considered to make this specific to
>>> irqchip initialization, but irqchips are different and a lot of that
>>> code is x86-specific, so that approach was discarded.
>>>
>>> This *could* look something like this:
>>>
>>> struct kvm_device_param {
>>> u64 dev_id;
>>> u64 param_id;
>>> u64 value;
>>> };
>>>
>>> but that has less clear, or at least less specific, semantics.
>>
>>
>> Why is it less clear? You need to have device-specific documentation for
>> what "id" means, so why not also an enumeration of "param"s? Or just keep
>> it as is, and rename "address" to "value". Whether "dev" and "param" are
>> combined is orthogonal from whether it's used for non-address things.
>
> less clear in the sense that you have to look at more code to see what
> it does. I'm not saying that it's too unclear, at all, I'm actually
> fine with it, but to make my point, we can make an ioctl that's called
> do_something() that takes a struct with val0, val1, val2, val3, ...
>
>>
>> If you leave it as "address", either we'll have it being used for
>> non-address things regardless of the name ("Not a typewriter!"), or there'll
>> end up being yet more unnecessary ioctls, or device-specific things will end
>> up getting shoved into CPU interfaces such as one-reg. For example, on MPIC
>> we need to be able to specify the version of the chip to emulate in addition
>> to the address at which it lives.
>>
>> Also, why is it documented as an "arm" interface? Shouldn't it be a generic
>> interface, with other architectures currently not implementing any IDs?
>> What in the kvm_arch_vm_ioctl() wrapper is arm-specific?
>>
> As I remember the argument for keeping this the point was that there
> were other preferred methods for other archs to do this, and that ARM
> was the only platform that had this explicit need, but maybe I'm
> making this up.
>
> I'll let Peter and Alex respond this as well, and if they're fine with
> changing it to what I proposed above, then let's do that, and we can
> make it a non arm-specific interface.
I think we should make thus at least potentially generic. In fact, I wouldn't even mind calling it DEV_REG with the same semantics as ONE_REG, just that it also gets a unique dev id that gets created during in-kernel device creation and that it's a vm ioctl.
That way we wouldn't block our path to create two in-kernel irqchips one day.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 10:02 ` Alexander Graf
@ 2013-01-09 14:48 ` Peter Maydell
2013-01-09 14:58 ` Alexander Graf
0 siblings, 1 reply; 79+ messages in thread
From: Peter Maydell @ 2013-01-09 14:48 UTC (permalink / raw)
To: linux-arm-kernel
On 9 January 2013 10:02, Alexander Graf <agraf@suse.de> wrote:
> I think we should make thus at least potentially generic. In fact, I wouldn't even
> mind calling it DEV_REG with the same semantics as ONE_REG, just that it also
> gets a unique dev id that gets created during in-kernel device creation and that
> it's a vm ioctl.
Well, we might want a DEV_REG, but you might as well just make ONE_REG
OK as a VM ioctl, since there's no reason not to have not-per-cpu but not
device registers. ONE_REG already supports dividing up the ID space so
you can say which device or whatever is being used, because we had
things like GIC registers in mind when we put that API together.
However, this shouldn't be DEV_REG, because this isn't actually setting state
in the irqchip device, it's configuring the kernel's part of the system model
[compare wiring up irq routing, which also has a custom ioctl rather than a
generic one]. As such it definitely needs to happen only before the VM is
actually started and not during VM execution, unlike a DEV_REG which would
presumably be generally valid.
> That way we wouldn't block our path to create two in-kernel irqchips one day.
Er, SET_DEVICE_ADDRESS doesn't block us doing that; that's why it has
an ID parameter.
The discussion at the KVM forum, as I recall it was basically:
(a) some other ppc irqchips also want to set the base address for where
their memory mapped registers live, so this isn't a totally ARM specific
weirdness
(b) we didn't need to tangle up and delay the KVM ARM work with a vague
and unspecified desire for general configurability
-- PMM
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 14:48 ` Peter Maydell
@ 2013-01-09 14:58 ` Alexander Graf
2013-01-09 15:11 ` Peter Maydell
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 14:58 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 15:48, Peter Maydell wrote:
> On 9 January 2013 10:02, Alexander Graf <agraf@suse.de> wrote:
>> I think we should make thus at least potentially generic. In fact, I wouldn't even
>> mind calling it DEV_REG with the same semantics as ONE_REG, just that it also
>> gets a unique dev id that gets created during in-kernel device creation and that
>> it's a vm ioctl.
>
> Well, we might want a DEV_REG, but you might as well just make ONE_REG
> OK as a VM ioctl, since there's no reason not to have not-per-cpu but not
> device registers. ONE_REG already supports dividing up the ID space so
> you can say which device or whatever is being used, because we had
> things like GIC registers in mind when we put that API together.
ONE_REG's address space today doesn't include a field for the target device, because that's already predetermined by the fd you run it on. Hence my suggestion to add it as separate field, because then we keep the id space mask-free.
> However, this shouldn't be DEV_REG, because this isn't actually setting state
> in the irqchip device, it's configuring the kernel's part of the system model
> [compare wiring up irq routing, which also has a custom ioctl rather than a
> generic one]. As such it definitely needs to happen only before the VM is
> actually started and not during VM execution, unlike a DEV_REG which would
> presumably be generally valid.
The same can be true for ONE_REG/SET_SREGS. On PPC we support setting the PVR (CPU type) via SREGS because ONE_REG only came later. Setting the CPU type only makes sense before you first start using a vcpu. After that point it should be forbidden.
So in a way, the irqchip memory address location is a device register, as it has all the properties that a register on that device would have. It is
* per device
* one id for one purpose
* may or may not be writable during certain times
We would also keep the gates open to get a new register id one day for a second address. Imagine an interrupt controller that splits its address mappings into "PIC base registers" and "MSI target registers", though the MSI registers logically belong to the PIC. Then the ioctl as designed here gets stuck too.
>
>> That way we wouldn't block our path to create two in-kernel irqchips one day.
>
> Er, SET_DEVICE_ADDRESS doesn't block us doing that; that's why it has
> an ID parameter.
>
> The discussion at the KVM forum, as I recall it was basically:
> (a) some other ppc irqchips also want to set the base address for where
> their memory mapped registers live, so this isn't a totally ARM specific
> weirdness
Yes.
> (b) we didn't need to tangle up and delay the KVM ARM work with a vague
> and unspecified desire for general configurability
Yeah, that was the basic idea. Considering that the patch set hasn't been going in for another 2 months after that discussion indicates that this isn't too much of an issue though :).
In fact, I do recall myself pointing out that we need some padding in that ioctl to accomodate for later usages like the ones above. Then we could later rename (or copy name) it and everyone'd be happy.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 14:58 ` Alexander Graf
@ 2013-01-09 15:11 ` Peter Maydell
2013-01-09 15:17 ` Christoffer Dall
` (2 more replies)
0 siblings, 3 replies; 79+ messages in thread
From: Peter Maydell @ 2013-01-09 15:11 UTC (permalink / raw)
To: linux-arm-kernel
On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>
> On 09.01.2013, at 15:48, Peter Maydell wrote:
>
>> On 9 January 2013 10:02, Alexander Graf <agraf@suse.de> wrote:
>>> I think we should make thus at least potentially generic. In fact, I wouldn't even
>>> mind calling it DEV_REG with the same semantics as ONE_REG, just that it also
>>> gets a unique dev id that gets created during in-kernel device creation and that
>>> it's a vm ioctl.
>>
>> Well, we might want a DEV_REG, but you might as well just make ONE_REG
>> OK as a VM ioctl, since there's no reason not to have not-per-cpu but not
>> device registers. ONE_REG already supports dividing up the ID space so
>> you can say which device or whatever is being used, because we had
>> things like GIC registers in mind when we put that API together.
>
> ONE_REG's address space today doesn't include a field for the target device,
There's 16 bits of "register group type".
> because that's already predetermined by the fd you run it on. Hence my
> suggestion to add it as separate field, because then we keep the id space
> mask-free.
Er, it already has masks for the different parts of ID space. That's a feature,
not a bug.
>> However, this shouldn't be DEV_REG, because this isn't actually setting state
>> in the irqchip device, it's configuring the kernel's part of the system model
>> [compare wiring up irq routing, which also has a custom ioctl rather than a
>> generic one]. As such it definitely needs to happen only before the VM is
>> actually started and not during VM execution, unlike a DEV_REG which would
>> presumably be generally valid.
>
> The same can be true for ONE_REG/SET_SREGS. On PPC we support setting
> the PVR (CPU type) via SREGS because ONE_REG only came later. Setting
> the CPU type only makes sense before you first start using a vcpu. After that
> point it should be forbidden.
>
> So in a way, the irqchip memory address location is a device register, as it has
> all the properties that a register on that device would have. It is
>
> * per device
> * one id for one purpose
> * may or may not be writable during certain times
There's a distinction between "things you need to do in machine setup"
and "things you need to propagate for migration". It should be possible
to do a migration by doing a complete copy of all ONE_REG (and DEV_REG
if we have it) state from source to destination, so it needs to be always
possible to write them.
> We would also keep the gates open to get a new register id one day for a second
> address. Imagine an interrupt controller that splits its address mappings into
> "PIC base registers" and "MSI target registers", though the MSI registers logically
> belong to the PIC. Then the ioctl as designed here gets stuck too.
This is exactly what ARM's GIC has already. We have several memory regions
(one for the distributor, one for the cpu interface), and map them with several
calls to SET_DEVICE_ADDRESS.
>> (b) we didn't need to tangle up and delay the KVM ARM work with a vague
>> and unspecified desire for general configurability
>
> Yeah, that was the basic idea. Considering that the patch set hasn't been going
> in for another 2 months after that discussion indicates that this isn't too much of
> an issue though :).
We might get there faster if people didn't keep nitpicking the APIs at the
last minute :-)
-- PMM
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:11 ` Peter Maydell
@ 2013-01-09 15:17 ` Christoffer Dall
2013-01-09 15:20 ` Alexander Graf
2013-01-09 15:22 ` Marc Zyngier
2 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 15:17 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 9, 2013 at 10:11 AM, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>
>> On 09.01.2013, at 15:48, Peter Maydell wrote:
>>
>>> On 9 January 2013 10:02, Alexander Graf <agraf@suse.de> wrote:
>>>> I think we should make thus at least potentially generic. In fact, I wouldn't even
>>>> mind calling it DEV_REG with the same semantics as ONE_REG, just that it also
>>>> gets a unique dev id that gets created during in-kernel device creation and that
>>>> it's a vm ioctl.
>>>
>>> Well, we might want a DEV_REG, but you might as well just make ONE_REG
>>> OK as a VM ioctl, since there's no reason not to have not-per-cpu but not
>>> device registers. ONE_REG already supports dividing up the ID space so
>>> you can say which device or whatever is being used, because we had
>>> things like GIC registers in mind when we put that API together.
>>
>> ONE_REG's address space today doesn't include a field for the target device,
>
> There's 16 bits of "register group type".
>
>> because that's already predetermined by the fd you run it on. Hence my
>> suggestion to add it as separate field, because then we keep the id space
>> mask-free.
>
> Er, it already has masks for the different parts of ID space. That's a feature,
> not a bug.
>
>>> However, this shouldn't be DEV_REG, because this isn't actually setting state
>>> in the irqchip device, it's configuring the kernel's part of the system model
>>> [compare wiring up irq routing, which also has a custom ioctl rather than a
>>> generic one]. As such it definitely needs to happen only before the VM is
>>> actually started and not during VM execution, unlike a DEV_REG which would
>>> presumably be generally valid.
>>
>> The same can be true for ONE_REG/SET_SREGS. On PPC we support setting
>> the PVR (CPU type) via SREGS because ONE_REG only came later. Setting
>> the CPU type only makes sense before you first start using a vcpu. After that
>> point it should be forbidden.
>>
>> So in a way, the irqchip memory address location is a device register, as it has
>> all the properties that a register on that device would have. It is
>>
>> * per device
>> * one id for one purpose
>> * may or may not be writable during certain times
>
> There's a distinction between "things you need to do in machine setup"
> and "things you need to propagate for migration". It should be possible
> to do a migration by doing a complete copy of all ONE_REG (and DEV_REG
> if we have it) state from source to destination, so it needs to be always
> possible to write them.
>
>> We would also keep the gates open to get a new register id one day for a second
>> address. Imagine an interrupt controller that splits its address mappings into
>> "PIC base registers" and "MSI target registers", though the MSI registers logically
>> belong to the PIC. Then the ioctl as designed here gets stuck too.
>
> This is exactly what ARM's GIC has already. We have several memory regions
> (one for the distributor, one for the cpu interface), and map them with several
> calls to SET_DEVICE_ADDRESS.
>
>>> (b) we didn't need to tangle up and delay the KVM ARM work with a vague
>>> and unspecified desire for general configurability
>>
>> Yeah, that was the basic idea. Considering that the patch set hasn't been going
>> in for another 2 months after that discussion indicates that this isn't too much of
>> an issue though :).
>
well, now it's actually going in, and it seems like this is the last
issue. Let's hold our breaths here for a second and let me write up a
new device attribute API and send that out, and if you guys can
possibly live with it, then please consider accepting it.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:11 ` Peter Maydell
2013-01-09 15:17 ` Christoffer Dall
@ 2013-01-09 15:20 ` Alexander Graf
2013-01-09 15:22 ` Marc Zyngier
2 siblings, 0 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 15:20 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 16:11, Peter Maydell wrote:
> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>
>> On 09.01.2013, at 15:48, Peter Maydell wrote:
>>
>>> On 9 January 2013 10:02, Alexander Graf <agraf@suse.de> wrote:
>>>> I think we should make thus at least potentially generic. In fact, I wouldn't even
>>>> mind calling it DEV_REG with the same semantics as ONE_REG, just that it also
>>>> gets a unique dev id that gets created during in-kernel device creation and that
>>>> it's a vm ioctl.
>>>
>>> Well, we might want a DEV_REG, but you might as well just make ONE_REG
>>> OK as a VM ioctl, since there's no reason not to have not-per-cpu but not
>>> device registers. ONE_REG already supports dividing up the ID space so
>>> you can say which device or whatever is being used, because we had
>>> things like GIC registers in mind when we put that API together.
>>
>> ONE_REG's address space today doesn't include a field for the target device,
>
> There's 16 bits of "register group type".
That's "register group", not "device id". It's constant per id. If you have an ARM register, it will always stay an ARM register. It's not going to change. But if you reuse those bits for the device id, they are going to change, because a device id is similar to a handle that should be returned by the device create ioctl.
>
>> because that's already predetermined by the fd you run it on. Hence my
>> suggestion to add it as separate field, because then we keep the id space
>> mask-free.
>
> Er, it already has masks for the different parts of ID space. That's a feature,
> not a bug.
All those masks are constant masks and only used to make reading the IDs easier. They never change within a single type of a register.
>
>>> However, this shouldn't be DEV_REG, because this isn't actually setting state
>>> in the irqchip device, it's configuring the kernel's part of the system model
>>> [compare wiring up irq routing, which also has a custom ioctl rather than a
>>> generic one]. As such it definitely needs to happen only before the VM is
>>> actually started and not during VM execution, unlike a DEV_REG which would
>>> presumably be generally valid.
>>
>> The same can be true for ONE_REG/SET_SREGS. On PPC we support setting
>> the PVR (CPU type) via SREGS because ONE_REG only came later. Setting
>> the CPU type only makes sense before you first start using a vcpu. After that
>> point it should be forbidden.
>>
>> So in a way, the irqchip memory address location is a device register, as it has
>> all the properties that a register on that device would have. It is
>>
>> * per device
>> * one id for one purpose
>> * may or may not be writable during certain times
>
> There's a distinction between "things you need to do in machine setup"
> and "things you need to propagate for migration". It should be possible
> to do a migration by doing a complete copy of all ONE_REG (and DEV_REG
> if we have it) state from source to destination, so it needs to be always
> possible to write them.
You need to somewhere encode a list of registers you want to synchronize for migration anyways. Don't include this register in that list then.
>
>> We would also keep the gates open to get a new register id one day for a second
>> address. Imagine an interrupt controller that splits its address mappings into
>> "PIC base registers" and "MSI target registers", though the MSI registers logically
>> belong to the PIC. Then the ioctl as designed here gets stuck too.
>
> This is exactly what ARM's GIC has already. We have several memory regions
> (one for the distributor, one for the cpu interface), and map them with several
> calls to SET_DEVICE_ADDRESS.
I see.
>
>>> (b) we didn't need to tangle up and delay the KVM ARM work with a vague
>>> and unspecified desire for general configurability
>>
>> Yeah, that was the basic idea. Considering that the patch set hasn't been going
>> in for another 2 months after that discussion indicates that this isn't too much of
>> an issue though :).
>
> We might get there faster if people didn't keep nitpicking the APIs at the
> last minute :-)
I guess we just had too much bad experience with designing horrible APIs in the past on the PPC side ;).
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:11 ` Peter Maydell
2013-01-09 15:17 ` Christoffer Dall
2013-01-09 15:20 ` Alexander Graf
@ 2013-01-09 15:22 ` Marc Zyngier
2013-01-09 15:28 ` Alexander Graf
2 siblings, 1 reply; 79+ messages in thread
From: Marc Zyngier @ 2013-01-09 15:22 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, 9 Jan 2013 15:11:39 +0000, Peter Maydell
<peter.maydell@linaro.org>
wrote:
> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>
>> Yeah, that was the basic idea. Considering that the patch set hasn't
>> been going
>> in for another 2 months after that discussion indicates that this isn't
>> too much of
>> an issue though :).
>
> We might get there faster if people didn't keep nitpicking the APIs at
the
> last minute :-)
Exactly. We're trying hard to get the damned thing merged, and the
permanent API churn quickly becomes a burden.
My understanding was that a consensus was reached 2 months ago. Has
something fundamentally changed? Something that makes this API invalid? If
not, can we please keep this one?
Thanks,
M.
--
Who you jivin' with that Cosmik Debris?
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:22 ` Marc Zyngier
@ 2013-01-09 15:28 ` Alexander Graf
2013-01-09 15:50 ` Marc Zyngier
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 15:28 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 16:22, Marc Zyngier wrote:
> On Wed, 9 Jan 2013 15:11:39 +0000, Peter Maydell
> <peter.maydell@linaro.org>
> wrote:
>> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>>
>>> Yeah, that was the basic idea. Considering that the patch set hasn't
>>> been going
>>> in for another 2 months after that discussion indicates that this isn't
>>> too much of
>>> an issue though :).
>>
>> We might get there faster if people didn't keep nitpicking the APIs at
> the
>> last minute :-)
>
> Exactly. We're trying hard to get the damned thing merged, and the
> permanent API churn quickly becomes a burden.
As I said earlier, we have had a lot of experience in creating really bad APIs in the past.
But how about making this one specific? Call it KVM_ARM_SET_VGIC_ADDRESS, keep the rest as it is, resend it, and later we can come up with an actually generic interface.
Alex
>
> My understanding was that a consensus was reached 2 months ago. Has
> something fundamentally changed? Something that makes this API invalid? If
> not, can we please keep this one?
>
> Thanks,
>
> M.
> --
> Who you jivin' with that Cosmik Debris?
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:28 ` Alexander Graf
@ 2013-01-09 15:50 ` Marc Zyngier
2013-01-09 15:56 ` Alexander Graf
0 siblings, 1 reply; 79+ messages in thread
From: Marc Zyngier @ 2013-01-09 15:50 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, 9 Jan 2013 16:28:03 +0100, Alexander Graf <agraf@suse.de> wrote:
> On 09.01.2013, at 16:22, Marc Zyngier wrote:
>
>> On Wed, 9 Jan 2013 15:11:39 +0000, Peter Maydell
>> <peter.maydell@linaro.org>
>> wrote:
>>> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>>>
>>>> Yeah, that was the basic idea. Considering that the patch set hasn't
>>>> been going
>>>> in for another 2 months after that discussion indicates that this
isn't
>>>> too much of
>>>> an issue though :).
>>>
>>> We might get there faster if people didn't keep nitpicking the APIs at
>> the
>>> last minute :-)
>>
>> Exactly. We're trying hard to get the damned thing merged, and the
>> permanent API churn quickly becomes a burden.
>
> As I said earlier, we have had a lot of experience in creating really
bad
> APIs in the past.
Is this one really bad? Again, what changed in the meantime that makes you
think this API is wrong?
> But how about making this one specific? Call it
KVM_ARM_SET_VGIC_ADDRESS,
> keep the rest as it is, resend it, and later we can come up with an
> actually generic interface.
Let's pretend you never wrote that, shall we? ;-)
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:50 ` Marc Zyngier
@ 2013-01-09 15:56 ` Alexander Graf
2013-01-09 16:12 ` Marc Zyngier
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 15:56 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 16:50, Marc Zyngier wrote:
> On Wed, 9 Jan 2013 16:28:03 +0100, Alexander Graf <agraf@suse.de> wrote:
>> On 09.01.2013, at 16:22, Marc Zyngier wrote:
>>
>>> On Wed, 9 Jan 2013 15:11:39 +0000, Peter Maydell
>>> <peter.maydell@linaro.org>
>>> wrote:
>>>> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>>>>
>>>>> Yeah, that was the basic idea. Considering that the patch set hasn't
>>>>> been going
>>>>> in for another 2 months after that discussion indicates that this
> isn't
>>>>> too much of
>>>>> an issue though :).
>>>>
>>>> We might get there faster if people didn't keep nitpicking the APIs at
>>> the
>>>> last minute :-)
>>>
>>> Exactly. We're trying hard to get the damned thing merged, and the
>>> permanent API churn quickly becomes a burden.
>>
>> As I said earlier, we have had a lot of experience in creating really
> bad
>> APIs in the past.
>
> Is this one really bad? Again, what changed in the meantime that makes you
> think this API is wrong?
I complained about it 2 months ago already.
>> But how about making this one specific? Call it
> KVM_ARM_SET_VGIC_ADDRESS,
>> keep the rest as it is, resend it, and later we can come up with an
>> actually generic interface.
>
> Let's pretend you never wrote that, shall we? ;-)
Why? The worst thing that happened to us so far were interfaces that looked generic and extensible and turned out not to be (see SET_SREGS in the ppc code). We have 2 options to circumvent this:
1) Commonly agree on an interface that actually is extensible and use it throughout the code
2) Create a very specific interface for a single use case, keep it implemented "forever" and as soon as we need more, implement a more generic one that goes back to 1
Since the ARM patches have been out of tree for too long, I'm happy with going route 2 until we go back to square 1.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 15:56 ` Alexander Graf
@ 2013-01-09 16:12 ` Marc Zyngier
2013-01-09 16:29 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Marc Zyngier @ 2013-01-09 16:12 UTC (permalink / raw)
To: linux-arm-kernel
On 09/01/13 15:56, Alexander Graf wrote:
>
> On 09.01.2013, at 16:50, Marc Zyngier wrote:
>
>> On Wed, 9 Jan 2013 16:28:03 +0100, Alexander Graf <agraf@suse.de> wrote:
>>> On 09.01.2013, at 16:22, Marc Zyngier wrote:
>>>
>>>> On Wed, 9 Jan 2013 15:11:39 +0000, Peter Maydell
>>>> <peter.maydell@linaro.org>
>>>> wrote:
>>>>> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>>>>>
>>>>>> Yeah, that was the basic idea. Considering that the patch set hasn't
>>>>>> been going
>>>>>> in for another 2 months after that discussion indicates that this
>> isn't
>>>>>> too much of
>>>>>> an issue though :).
>>>>>
>>>>> We might get there faster if people didn't keep nitpicking the APIs at
>>>> the
>>>>> last minute :-)
>>>>
>>>> Exactly. We're trying hard to get the damned thing merged, and the
>>>> permanent API churn quickly becomes a burden.
>>>
>>> As I said earlier, we have had a lot of experience in creating really
>> bad
>>> APIs in the past.
>>
>> Is this one really bad? Again, what changed in the meantime that makes you
>> think this API is wrong?
>
> I complained about it 2 months ago already.
>
>>> But how about making this one specific? Call it
>> KVM_ARM_SET_VGIC_ADDRESS,
>>> keep the rest as it is, resend it, and later we can come up with an
>>> actually generic interface.
>>
>> Let's pretend you never wrote that, shall we? ;-)
>
> Why? The worst thing that happened to us so far were interfaces that looked generic and extensible and turned out not to be (see SET_SREGS in the ppc code). We have 2 options to circumvent this:
>
> 1) Commonly agree on an interface that actually is extensible and use it throughout the code
> 2) Create a very specific interface for a single use case, keep it implemented "forever" and as soon as we need more, implement a more generic one that goes back to 1
>
> Since the ARM patches have been out of tree for too long, I'm happy with going route 2 until we go back to square 1.
I really don't want to see that. Either we keep the API as it is, or we
change it for something that is really better and used by other
architectures. No point in turning it upside down if we're the only one
doing it.
Oh, and as we're aiming for 3.9, it'd better be ready soon...
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 16:12 ` Marc Zyngier
@ 2013-01-09 16:29 ` Christoffer Dall
0 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 16:29 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 9, 2013 at 11:12 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 09/01/13 15:56, Alexander Graf wrote:
>>
>> On 09.01.2013, at 16:50, Marc Zyngier wrote:
>>
>>> On Wed, 9 Jan 2013 16:28:03 +0100, Alexander Graf <agraf@suse.de> wrote:
>>>> On 09.01.2013, at 16:22, Marc Zyngier wrote:
>>>>
>>>>> On Wed, 9 Jan 2013 15:11:39 +0000, Peter Maydell
>>>>> <peter.maydell@linaro.org>
>>>>> wrote:
>>>>>> On 9 January 2013 14:58, Alexander Graf <agraf@suse.de> wrote:
>>>>>>>
>>>>>>> Yeah, that was the basic idea. Considering that the patch set hasn't
>>>>>>> been going
>>>>>>> in for another 2 months after that discussion indicates that this
>>> isn't
>>>>>>> too much of
>>>>>>> an issue though :).
>>>>>>
>>>>>> We might get there faster if people didn't keep nitpicking the APIs at
>>>>> the
>>>>>> last minute :-)
>>>>>
>>>>> Exactly. We're trying hard to get the damned thing merged, and the
>>>>> permanent API churn quickly becomes a burden.
>>>>
>>>> As I said earlier, we have had a lot of experience in creating really
>>> bad
>>>> APIs in the past.
>>>
>>> Is this one really bad? Again, what changed in the meantime that makes you
>>> think this API is wrong?
>>
>> I complained about it 2 months ago already.
>>
>>>> But how about making this one specific? Call it
>>> KVM_ARM_SET_VGIC_ADDRESS,
>>>> keep the rest as it is, resend it, and later we can come up with an
>>>> actually generic interface.
>>>
>>> Let's pretend you never wrote that, shall we? ;-)
>>
>> Why? The worst thing that happened to us so far were interfaces that looked generic and extensible and turned out not to be (see SET_SREGS in the ppc code). We have 2 options to circumvent this:
>>
>> 1) Commonly agree on an interface that actually is extensible and use it throughout the code
>> 2) Create a very specific interface for a single use case, keep it implemented "forever" and as soon as we need more, implement a more generic one that goes back to 1
>>
>> Since the ARM patches have been out of tree for too long, I'm happy with going route 2 until we go back to square 1.
>
> I really don't want to see that. Either we keep the API as it is, or we
> change it for something that is really better and used by other
> architectures. No point in turning it upside down if we're the only one
> doing it.
>
> Oh, and as we're aiming for 3.9, it'd better be ready soon...
>
ok, I renamed it to KVM_ARM_SET_DEVICE_ADDR, using the same binary
format, so user space tools remain compatible.
I'll be happy to help out with a more generic API, once the kvm/arm
patches reach upstream.
Thanks so far!
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 02/12] ARM: KVM: Keep track of currently running vcpus
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
2013-01-08 18:41 ` [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl Christoffer Dall
@ 2013-01-08 18:41 ` Christoffer Dall
2013-01-08 18:41 ` [PATCH v5 03/12] ARM: gic: define GICH offsets for VGIC support Christoffer Dall
` (10 subsequent siblings)
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:41 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
When an interrupt occurs for the guest, it is sometimes necessary
to find out which vcpu was running at that point.
Keep track of which vcpu is being run in kvm_arch_vcpu_ioctl_run(),
and allow the data to be retrieved using either:
- kvm_arm_get_running_vcpu(): returns the vcpu running at this point
on the current CPU. Can only be used in a non-preemptible context.
- kvm_arm_get_running_vcpus(): returns the per-CPU variable holding
the running vcpus, usable for per-CPU interrupts.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/kvm_host.h | 10 ++++++++++
arch/arm/kvm/arm.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 40 insertions(+)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index ca40795..7a9f1d5 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -155,4 +155,14 @@ static inline int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
{
return 0;
}
+
+struct kvm_vcpu *kvm_arm_get_running_vcpu(void);
+struct kvm_vcpu __percpu **kvm_get_running_vcpus(void);
+
+int kvm_arm_copy_coproc_indices(struct kvm_vcpu *vcpu, u64 __user *uindices);
+unsigned long kvm_arm_num_coproc_regs(struct kvm_vcpu *vcpu);
+struct kvm_one_reg;
+int kvm_arm_coproc_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *);
+int kvm_arm_coproc_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *);
+
#endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 2f39b04..5180f7b 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -53,11 +53,38 @@ static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page);
static struct vfp_hard_struct __percpu *kvm_host_vfp_state;
static unsigned long hyp_default_vectors;
+/* Per-CPU variable containing the currently running vcpu. */
+static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_arm_running_vcpu);
+
/* The VMID used in the VTTBR */
static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1);
static u8 kvm_next_vmid;
static DEFINE_SPINLOCK(kvm_vmid_lock);
+static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
+{
+ BUG_ON(preemptible());
+ __get_cpu_var(kvm_arm_running_vcpu) = vcpu;
+}
+
+/**
+ * kvm_arm_get_running_vcpu - get the vcpu running on the current CPU.
+ * Must be called from non-preemptible context
+ */
+struct kvm_vcpu *kvm_arm_get_running_vcpu(void)
+{
+ BUG_ON(preemptible());
+ return __get_cpu_var(kvm_arm_running_vcpu);
+}
+
+/**
+ * kvm_arm_get_running_vcpus - get the per-CPU array of currently running vcpus.
+ */
+struct kvm_vcpu __percpu **kvm_get_running_vcpus(void)
+{
+ return &kvm_arm_running_vcpu;
+}
+
int kvm_arch_hardware_enable(void *garbage)
{
return 0;
@@ -310,10 +337,13 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
cpumask_clear_cpu(cpu, &vcpu->arch.require_dcache_flush);
flush_cache_all(); /* We'd really want v7_flush_dcache_all() */
}
+
+ kvm_arm_set_running_vcpu(vcpu);
}
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
+ kvm_arm_set_running_vcpu(NULL);
}
int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 03/12] ARM: gic: define GICH offsets for VGIC support
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
2013-01-08 18:41 ` [PATCH v5 01/12] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl Christoffer Dall
2013-01-08 18:41 ` [PATCH v5 02/12] ARM: KVM: Keep track of currently running vcpus Christoffer Dall
@ 2013-01-08 18:41 ` Christoffer Dall
2013-01-08 18:41 ` [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code Christoffer Dall
` (9 subsequent siblings)
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:41 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
The GICH_* constants are defined by the GIC HW spec, and even though
they only be used by KVM to begin with, define them generically in gic.h.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/hardware/gic.h | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/arch/arm/include/asm/hardware/gic.h b/arch/arm/include/asm/hardware/gic.h
index 4b1ce6c..dd1add1 100644
--- a/arch/arm/include/asm/hardware/gic.h
+++ b/arch/arm/include/asm/hardware/gic.h
@@ -32,6 +32,31 @@
#define GIC_DIST_CONFIG 0xc00
#define GIC_DIST_SOFTINT 0xf00
+#define GICH_HCR 0x0
+#define GICH_VTR 0x4
+#define GICH_VMCR 0x8
+#define GICH_MISR 0x10
+#define GICH_EISR0 0x20
+#define GICH_EISR1 0x24
+#define GICH_ELRSR0 0x30
+#define GICH_ELRSR1 0x34
+#define GICH_APR 0xf0
+#define GICH_LR0 0x100
+
+#define GICH_HCR_EN (1 << 0)
+#define GICH_HCR_UIE (1 << 1)
+
+#define GICH_LR_VIRTUALID (0x3ff << 0)
+#define GICH_LR_PHYSID_CPUID_SHIFT (10)
+#define GICH_LR_PHYSID_CPUID (7 << GICH_LR_PHYSID_CPUID_SHIFT)
+#define GICH_LR_STATE (3 << 28)
+#define GICH_LR_PENDING_BIT (1 << 28)
+#define GICH_LR_ACTIVE_BIT (1 << 29)
+#define GICH_LR_EOI (1 << 19)
+
+#define GICH_MISR_EOI (1 << 0)
+#define GICH_MISR_U (1 << 1)
+
#ifndef __ASSEMBLY__
#include <linux/irqdomain.h>
struct device_node;
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (2 preceding siblings ...)
2013-01-08 18:41 ` [PATCH v5 03/12] ARM: gic: define GICH offsets for VGIC support Christoffer Dall
@ 2013-01-08 18:41 ` Christoffer Dall
2013-01-14 15:31 ` Will Deacon
2013-01-08 18:41 ` [PATCH v5 05/12] ARM: KVM: VGIC accept vcpu and dist base addresses from user space Christoffer Dall
` (8 subsequent siblings)
12 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:41 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
Wire the basic framework code for VGIC support and the initial in-kernel
MMIO support code for the VGIC, used for the distributor emulation.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/kvm_host.h | 8 ++
arch/arm/include/asm/kvm_vgic.h | 80 ++++++++++++++++++++++
arch/arm/kvm/Makefile | 1
arch/arm/kvm/arm.c | 27 +++++++
arch/arm/kvm/interrupts.S | 4 +
arch/arm/kvm/mmio.c | 3 +
arch/arm/kvm/vgic.c | 144 +++++++++++++++++++++++++++++++++++++++
7 files changed, 266 insertions(+), 1 deletion(-)
create mode 100644 arch/arm/include/asm/kvm_vgic.h
create mode 100644 arch/arm/kvm/vgic.c
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 7a9f1d5..149d62b 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -37,6 +37,8 @@
#define KVM_NR_PAGE_SIZES 1
#define KVM_PAGES_PER_HPAGE(x) (1UL<<31)
+#include <asm/kvm_vgic.h>
+
struct kvm_vcpu;
u32 *kvm_vcpu_reg(struct kvm_vcpu *vcpu, u8 reg_num, u32 mode);
int kvm_target_cpu(void);
@@ -58,6 +60,9 @@ struct kvm_arch {
/* Stage-2 page table */
pgd_t *pgd;
+
+ /* Interrupt controller */
+ struct vgic_dist vgic;
};
#define KVM_NR_MEM_OBJS 40
@@ -92,6 +97,9 @@ struct kvm_vcpu_arch {
struct vfp_hard_struct vfp_guest;
struct vfp_hard_struct *vfp_host;
+ /* VGIC state */
+ struct vgic_cpu vgic_cpu;
+
/*
* Anything that is not used directly from assembly code goes
* here.
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
new file mode 100644
index 0000000..fcfd530
--- /dev/null
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -0,0 +1,80 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __ASM_ARM_KVM_VGIC_H
+#define __ASM_ARM_KVM_VGIC_H
+
+#include <asm/hardware/gic.h>
+
+struct vgic_dist {
+};
+
+struct vgic_cpu {
+};
+
+struct kvm;
+struct kvm_vcpu;
+struct kvm_run;
+struct kvm_exit_mmio;
+
+#ifdef CONFIG_KVM_ARM_VGIC
+bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_exit_mmio *mmio);
+
+#else
+static inline int kvm_vgic_hyp_init(void)
+{
+ return 0;
+}
+
+static inline int kvm_vgic_init(struct kvm *kvm)
+{
+ return 0;
+}
+
+static inline int kvm_vgic_create(struct kvm *kvm)
+{
+ return 0;
+}
+
+static inline int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
+{
+ return 0;
+}
+
+static inline void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu) {}
+static inline void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu) {}
+
+static inline int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
+{
+ return 0;
+}
+
+static inline bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_exit_mmio *mmio)
+{
+ return false;
+}
+
+static inline int irqchip_in_kernel(struct kvm *kvm)
+{
+ return 0;
+}
+#endif
+
+#endif
diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index 44a5f4b..3c6620c 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -19,3 +19,4 @@ kvm-arm-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
obj-y += kvm-arm.o init.o interrupts.o
obj-y += arm.o guest.o mmu.o emulate.o reset.o
obj-y += coproc.o coproc_a15.o mmio.o decode.o
+obj-$(CONFIG_KVM_ARM_VGIC) += vgic.o
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 5180f7b..85c4cdf 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -61,6 +61,8 @@ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1);
static u8 kvm_next_vmid;
static DEFINE_SPINLOCK(kvm_vmid_lock);
+static bool vgic_present;
+
static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
{
BUG_ON(preemptible());
@@ -183,6 +185,9 @@ int kvm_dev_ioctl_check_extension(long ext)
{
int r;
switch (ext) {
+ case KVM_CAP_IRQCHIP:
+ r = vgic_present;
+ break;
case KVM_CAP_USER_MEMORY:
case KVM_CAP_SYNC_MMU:
case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
@@ -313,8 +318,16 @@ int __attribute_const__ kvm_target_cpu(void)
int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
{
+ int ret;
+
/* Force users to call KVM_ARM_VCPU_INIT */
vcpu->arch.target = -1;
+
+ /* Set up VGIC */
+ ret = kvm_vgic_vcpu_init(vcpu);
+ if (ret)
+ return ret;
+
return 0;
}
@@ -374,7 +387,7 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
*/
int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
{
- return !!v->arch.irq_lines;
+ return !!v->arch.irq_lines || kvm_vgic_vcpu_pending_irq(v);
}
int kvm_arch_vcpu_in_guest_mode(struct kvm_vcpu *v)
@@ -665,6 +678,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
update_vttbr(vcpu->kvm);
+ kvm_vgic_sync_to_cpu(vcpu);
+
local_irq_disable();
/*
@@ -677,6 +692,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
if (ret <= 0 || need_new_vmid_gen(vcpu->kvm)) {
local_irq_enable();
+ kvm_vgic_sync_from_cpu(vcpu);
continue;
}
@@ -715,6 +731,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
* Back from guest
*************************************************************/
+ kvm_vgic_sync_from_cpu(vcpu);
+
ret = handle_exit(vcpu, run, ret);
}
@@ -994,6 +1012,13 @@ static int init_hyp_mode(void)
}
}
+ /*
+ * Init HYP view of VGIC
+ */
+ err = kvm_vgic_hyp_init();
+ if (err)
+ goto out_free_vfp;
+
kvm_info("Hyp mode initialized successfully\n");
return 0;
out_free_vfp:
diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S
index 45570b8..9ff7904 100644
--- a/arch/arm/kvm/interrupts.S
+++ b/arch/arm/kvm/interrupts.S
@@ -94,6 +94,8 @@ ENTRY(__kvm_vcpu_run)
save_host_regs
+ restore_vgic_state
+
@ Store hardware CP15 state and load guest state
read_cp15_state store_to_vcpu = 0
write_cp15_state read_from_vcpu = 1
@@ -187,6 +189,8 @@ after_vfp_restore:
read_cp15_state store_to_vcpu = 1
write_cp15_state read_from_vcpu = 0
+ save_vgic_state
+
restore_host_regs
clrex @ Clear exclusive monitor
mov r0, r1 @ Return the return code
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index d6a4ca0..eadec78a 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -149,6 +149,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (mmio.is_write)
memcpy(mmio.data, vcpu_reg(vcpu, rt), mmio.len);
+ if (vgic_handle_mmio(vcpu, run, &mmio))
+ return 1;
+
kvm_prepare_mmio(run, &mmio);
return 0;
}
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
new file mode 100644
index 0000000..1feee5a
--- /dev/null
+++ b/arch/arm/kvm/vgic.c
@@ -0,0 +1,144 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <asm/kvm_emulate.h>
+
+#define ACCESS_READ_VALUE (1 << 0)
+#define ACCESS_READ_RAZ (0 << 0)
+#define ACCESS_READ_MASK(x) ((x) & (1 << 0))
+#define ACCESS_WRITE_IGNORED (0 << 1)
+#define ACCESS_WRITE_SETBIT (1 << 1)
+#define ACCESS_WRITE_CLEARBIT (2 << 1)
+#define ACCESS_WRITE_VALUE (3 << 1)
+#define ACCESS_WRITE_MASK(x) ((x) & (3 << 1))
+
+/**
+ * vgic_reg_access - access vgic register
+ * @mmio: pointer to the data describing the mmio access
+ * @reg: pointer to the virtual backing of vgic distributor data
+ * @offset: least significant 2 bits used for word offset
+ * @mode: ACCESS_ mode (see defines above)
+ *
+ * Helper to make vgic register access easier using one of the access
+ * modes defined for vgic register access
+ * (read,raz,write-ignored,setbit,clearbit,write)
+ */
+static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
+ phys_addr_t offset, int mode)
+{
+ int shift = (offset & 3) * 8;
+ u32 mask;
+ u32 regval;
+
+ /*
+ * Any alignment fault should have been delivered to the guest
+ * directly (ARM ARM B3.12.7 "Prioritization of aborts").
+ */
+
+ mask = (~0U) >> shift;
+ if (reg) {
+ regval = *reg;
+ } else {
+ BUG_ON(mode != (ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED));
+ regval = 0;
+ }
+
+ if (mmio->is_write) {
+ u32 data = (*((u32 *)mmio->data) & mask) << shift;
+ switch (ACCESS_WRITE_MASK(mode)) {
+ case ACCESS_WRITE_IGNORED:
+ return;
+
+ case ACCESS_WRITE_SETBIT:
+ regval |= data;
+ break;
+
+ case ACCESS_WRITE_CLEARBIT:
+ regval &= ~data;
+ break;
+
+ case ACCESS_WRITE_VALUE:
+ regval = (regval & ~(mask << shift)) | data;
+ break;
+ }
+ *reg = regval;
+ } else {
+ switch (ACCESS_READ_MASK(mode)) {
+ case ACCESS_READ_RAZ:
+ regval = 0;
+ /* fall through */
+
+ case ACCESS_READ_VALUE:
+ *((u32 *)mmio->data) = (regval >> shift) & mask;
+ }
+ }
+}
+
+/*
+ * I would have liked to use the kvm_bus_io_*() API instead, but it
+ * cannot cope with banked registers (only the VM pointer is passed
+ * around, and we need the vcpu). One of these days, someone please
+ * fix it!
+ */
+struct mmio_range {
+ phys_addr_t base;
+ unsigned long len;
+ bool (*handle_mmio)(struct kvm_vcpu *vcpu, struct kvm_exit_mmio *mmio,
+ phys_addr_t offset);
+};
+
+static const struct mmio_range vgic_ranges[] = {
+ {}
+};
+
+static const
+struct mmio_range *find_matching_range(const struct mmio_range *ranges,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t base)
+{
+ const struct mmio_range *r = ranges;
+ phys_addr_t addr = mmio->phys_addr - base;
+
+ while (r->len) {
+ if (addr >= r->base &&
+ (addr + mmio->len) <= (r->base + r->len))
+ return r;
+ r++;
+ }
+
+ return NULL;
+}
+
+/**
+ * vgic_handle_mmio - handle an in-kernel MMIO access
+ * @vcpu: pointer to the vcpu performing the access
+ * @run: pointer to the kvm_run structure
+ * @mmio: pointer to the data describing the access
+ *
+ * returns true if the MMIO access has been performed in kernel space,
+ * and false if it needs to be emulated in user space.
+ */
+bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_exit_mmio *mmio)
+{
+ return KVM_EXIT_MMIO;
+}
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code
2013-01-08 18:41 ` [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code Christoffer Dall
@ 2013-01-14 15:31 ` Will Deacon
2013-01-14 21:08 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Will Deacon @ 2013-01-14 15:31 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Jan 08, 2013 at 06:41:51PM +0000, Christoffer Dall wrote:
> From: Marc Zyngier <marc.zyngier@arm.com>
>
> Wire the basic framework code for VGIC support and the initial in-kernel
> MMIO support code for the VGIC, used for the distributor emulation.
[...]
> +/**
> + * vgic_reg_access - access vgic register
> + * @mmio: pointer to the data describing the mmio access
> + * @reg: pointer to the virtual backing of vgic distributor data
> + * @offset: least significant 2 bits used for word offset
> + * @mode: ACCESS_ mode (see defines above)
> + *
> + * Helper to make vgic register access easier using one of the access
> + * modes defined for vgic register access
> + * (read,raz,write-ignored,setbit,clearbit,write)
> + */
> +static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
> + phys_addr_t offset, int mode)
> +{
> + int shift = (offset & 3) * 8;
> + u32 mask;
> + u32 regval;
> +
> + /*
> + * Any alignment fault should have been delivered to the guest
> + * directly (ARM ARM B3.12.7 "Prioritization of aborts").
> + */
> +
> + mask = (~0U) >> shift;
> + if (reg) {
> + regval = *reg;
> + } else {
> + BUG_ON(mode != (ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED));
> + regval = 0;
> + }
> +
> + if (mmio->is_write) {
> + u32 data = (*((u32 *)mmio->data) & mask) << shift;
> + switch (ACCESS_WRITE_MASK(mode)) {
> + case ACCESS_WRITE_IGNORED:
> + return;
> +
> + case ACCESS_WRITE_SETBIT:
> + regval |= data;
> + break;
> +
> + case ACCESS_WRITE_CLEARBIT:
> + regval &= ~data;
> + break;
> +
> + case ACCESS_WRITE_VALUE:
> + regval = (regval & ~(mask << shift)) | data;
> + break;
> + }
> + *reg = regval;
> + } else {
> + switch (ACCESS_READ_MASK(mode)) {
> + case ACCESS_READ_RAZ:
> + regval = 0;
> + /* fall through */
> +
> + case ACCESS_READ_VALUE:
> + *((u32 *)mmio->data) = (regval >> shift) & mask;
> + }
> + }
> +}
As I mentioned previously, I suspect that this doesn't work with big-endian
systems. Whilst that's reasonable for the moment, a comment would be useful
for the unlucky soul that decides to do that work in future (or add
accessors for mmio->data as I suggested before).
Will
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code
2013-01-14 15:31 ` Will Deacon
@ 2013-01-14 21:08 ` Christoffer Dall
2013-01-14 21:28 ` [kvmarm] " Alexander Graf
` (2 more replies)
0 siblings, 3 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-14 21:08 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jan 14, 2013 at 10:31 AM, Will Deacon <will.deacon@arm.com> wrote:
> On Tue, Jan 08, 2013 at 06:41:51PM +0000, Christoffer Dall wrote:
>> From: Marc Zyngier <marc.zyngier@arm.com>
>>
>> Wire the basic framework code for VGIC support and the initial in-kernel
>> MMIO support code for the VGIC, used for the distributor emulation.
>
> [...]
>
>> +/**
>> + * vgic_reg_access - access vgic register
>> + * @mmio: pointer to the data describing the mmio access
>> + * @reg: pointer to the virtual backing of vgic distributor data
>> + * @offset: least significant 2 bits used for word offset
>> + * @mode: ACCESS_ mode (see defines above)
>> + *
>> + * Helper to make vgic register access easier using one of the access
>> + * modes defined for vgic register access
>> + * (read,raz,write-ignored,setbit,clearbit,write)
>> + */
>> +static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
>> + phys_addr_t offset, int mode)
>> +{
>> + int shift = (offset & 3) * 8;
>> + u32 mask;
>> + u32 regval;
>> +
>> + /*
>> + * Any alignment fault should have been delivered to the guest
>> + * directly (ARM ARM B3.12.7 "Prioritization of aborts").
>> + */
>> +
>> + mask = (~0U) >> shift;
>> + if (reg) {
>> + regval = *reg;
>> + } else {
>> + BUG_ON(mode != (ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED));
>> + regval = 0;
>> + }
>> +
>> + if (mmio->is_write) {
>> + u32 data = (*((u32 *)mmio->data) & mask) << shift;
>> + switch (ACCESS_WRITE_MASK(mode)) {
>> + case ACCESS_WRITE_IGNORED:
>> + return;
>> +
>> + case ACCESS_WRITE_SETBIT:
>> + regval |= data;
>> + break;
>> +
>> + case ACCESS_WRITE_CLEARBIT:
>> + regval &= ~data;
>> + break;
>> +
>> + case ACCESS_WRITE_VALUE:
>> + regval = (regval & ~(mask << shift)) | data;
>> + break;
>> + }
>> + *reg = regval;
>> + } else {
>> + switch (ACCESS_READ_MASK(mode)) {
>> + case ACCESS_READ_RAZ:
>> + regval = 0;
>> + /* fall through */
>> +
>> + case ACCESS_READ_VALUE:
>> + *((u32 *)mmio->data) = (regval >> shift) & mask;
>> + }
>> + }
>> +}
>
> As I mentioned previously, I suspect that this doesn't work with big-endian
> systems. Whilst that's reasonable for the moment, a comment would be useful
> for the unlucky soul that decides to do that work in future (or add
> accessors for mmio->data as I suggested before).
>
admittedly this really hurts my brain, but I think there's actually no
problem with endianness: whatever comes in mmio->data will have native
endianness and the vgic is always little-endian, so a guest would have
to make sure to do its own endianness conversion before writing data,
or did I get this backwards? (some nasty feeling about if the OS is
compiled in another endianness than the hardware everything may
break).
Anyhow, I think there's another bug in this code though. Please take a
look and see if you agree:
commit 3cab2b93a6f6acd3c043e584f23b94ab8f1bbd66
Author: Christoffer Dall <c.dall@virtualopensystems.com>
Date: Mon Jan 14 15:55:18 2013 -0500
KVM: ARM: Limit vgic read/writes to load/store length
The vgic read/write operations did not consider ldrb/strb masks, and
would therefore unintentionally overwrite parts of a register.
Consider for example a store of a single byte to a word-aligned address
of one of the priority registers, that would cause the 3 most
significant bytes to be overwritten with zeros.
Cc: Marc Zyniger <marc.zyngier@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 25daa07..5c1bcf5 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -233,6 +233,16 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu
*vcpu, int irq)
vcpu->arch.vgic_cpu.pending_shared);
}
+static u32 mmio_data_read(struct kvm_exit_mmio *mmio, u32 mask)
+{
+ return *((u32 *)mmio->data) & mask;
+}
+
+static void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
+{
+ *((u32 *)mmio->data) = value & mask;
+}
+
/**
* vgic_reg_access - access vgic register
* @mmio: pointer to the data describing the mmio access
@@ -247,8 +257,8 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu
*vcpu, int irq)
static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
phys_addr_t offset, int mode)
{
- int shift = (offset & 3) * 8;
- u32 mask;
+ int word_offset = (offset & 3) * 8;
+ u32 mask = (1UL << (mmio->len * 8)) - 1;
u32 regval;
/*
@@ -256,7 +266,6 @@ static void vgic_reg_access(struct kvm_exit_mmio
*mmio, u32 *reg,
* directly (ARM ARM B3.12.7 "Prioritization of aborts").
*/
- mask = (~0U) >> shift;
if (reg) {
regval = *reg;
} else {
@@ -265,7 +274,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
*mmio, u32 *reg,
}
if (mmio->is_write) {
- u32 data = (*((u32 *)mmio->data) & mask) << shift;
+ u32 data = mmio_data_read(mmio, mask) << word_offset;
switch (ACCESS_WRITE_MASK(mode)) {
case ACCESS_WRITE_IGNORED:
return;
@@ -279,7 +288,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
*mmio, u32 *reg,
break;
case ACCESS_WRITE_VALUE:
- regval = (regval & ~(mask << shift)) | data;
+ regval = (regval & ~(mask << word_offset)) | data;
break;
}
*reg = regval;
@@ -290,7 +299,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
*mmio, u32 *reg,
/* fall through */
case ACCESS_READ_VALUE:
- *((u32 *)mmio->data) = (regval >> shift) & mask;
+ mmio_data_write(mmio, mask, regval >> word_offset);
}
}
}
@@ -702,6 +711,12 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu,
struct kvm_run *run,
(mmio->phys_addr + mmio->len) > (base + KVM_VGIC_V2_DIST_SIZE))
return false;
+ /* We don't support ldrd / strd or ldm / stm to the emulated vgic */
+ if (mmio->len > 4) {
+ kvm_inject_dabt(vcpu, mmio->phys_addr);
+ return true;
+ }
+
range = find_matching_range(vgic_ranges, mmio, base);
if (unlikely(!range || !range->handle_mmio)) {
pr_warn("Unhandled access %d %08llx %d\n",
--
Thanks,
-Christoffer
^ permalink raw reply related [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code
2013-01-14 21:08 ` Christoffer Dall
@ 2013-01-14 21:28 ` Alexander Graf
2013-01-14 22:50 ` Will Deacon
2013-01-15 10:33 ` Marc Zyngier
2 siblings, 0 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-14 21:28 UTC (permalink / raw)
To: linux-arm-kernel
Am 14.01.2013 um 22:08 schrieb Christoffer Dall <c.dall@virtualopensystems.com>:
> On Mon, Jan 14, 2013 at 10:31 AM, Will Deacon <will.deacon@arm.com> wrote:
>> On Tue, Jan 08, 2013 at 06:41:51PM +0000, Christoffer Dall wrote:
>>> From: Marc Zyngier <marc.zyngier@arm.com>
>>>
>>> Wire the basic framework code for VGIC support and the initial in-kernel
>>> MMIO support code for the VGIC, used for the distributor emulation.
>>
>> [...]
>>
>>> +/**
>>> + * vgic_reg_access - access vgic register
>>> + * @mmio: pointer to the data describing the mmio access
>>> + * @reg: pointer to the virtual backing of vgic distributor data
>>> + * @offset: least significant 2 bits used for word offset
>>> + * @mode: ACCESS_ mode (see defines above)
>>> + *
>>> + * Helper to make vgic register access easier using one of the access
>>> + * modes defined for vgic register access
>>> + * (read,raz,write-ignored,setbit,clearbit,write)
>>> + */
>>> +static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
>>> + phys_addr_t offset, int mode)
>>> +{
>>> + int shift = (offset & 3) * 8;
>>> + u32 mask;
>>> + u32 regval;
>>> +
>>> + /*
>>> + * Any alignment fault should have been delivered to the guest
>>> + * directly (ARM ARM B3.12.7 "Prioritization of aborts").
>>> + */
>>> +
>>> + mask = (~0U) >> shift;
>>> + if (reg) {
>>> + regval = *reg;
>>> + } else {
>>> + BUG_ON(mode != (ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED));
>>> + regval = 0;
>>> + }
>>> +
>>> + if (mmio->is_write) {
>>> + u32 data = (*((u32 *)mmio->data) & mask) << shift;
>>> + switch (ACCESS_WRITE_MASK(mode)) {
>>> + case ACCESS_WRITE_IGNORED:
>>> + return;
>>> +
>>> + case ACCESS_WRITE_SETBIT:
>>> + regval |= data;
>>> + break;
>>> +
>>> + case ACCESS_WRITE_CLEARBIT:
>>> + regval &= ~data;
>>> + break;
>>> +
>>> + case ACCESS_WRITE_VALUE:
>>> + regval = (regval & ~(mask << shift)) | data;
>>> + break;
>>> + }
>>> + *reg = regval;
>>> + } else {
>>> + switch (ACCESS_READ_MASK(mode)) {
>>> + case ACCESS_READ_RAZ:
>>> + regval = 0;
>>> + /* fall through */
>>> +
>>> + case ACCESS_READ_VALUE:
>>> + *((u32 *)mmio->data) = (regval >> shift) & mask;
>>> + }
>>> + }
>>> +}
>>
>> As I mentioned previously, I suspect that this doesn't work with big-endian
>> systems. Whilst that's reasonable for the moment, a comment would be useful
>> for the unlucky soul that decides to do that work in future (or add
>> accessors for mmio->data as I suggested before).
>>
> admittedly this really hurts my brain, but I think there's actually no
> problem with endianness: whatever comes in mmio->data will have native
> endianness
IIRC we have a local endianness flag on ppc. Once you introduce big endian guests, you can just add one too and add a CAP for it. I wouldn't worry about it now though.
Alex
> and the vgic is always little-endian, so a guest would have
> to make sure to do its own endianness conversion before writing data,
> or did I get this backwards? (some nasty feeling about if the OS is
> compiled in another endianness than the hardware everything may
> break).
>
> Anyhow, I think there's another bug in this code though. Please take a
> look and see if you agree:
>
> commit 3cab2b93a6f6acd3c043e584f23b94ab8f1bbd66
> Author: Christoffer Dall <c.dall@virtualopensystems.com>
> Date: Mon Jan 14 15:55:18 2013 -0500
>
> KVM: ARM: Limit vgic read/writes to load/store length
>
> The vgic read/write operations did not consider ldrb/strb masks, and
> would therefore unintentionally overwrite parts of a register.
>
> Consider for example a store of a single byte to a word-aligned address
> of one of the priority registers, that would cause the 3 most
> significant bytes to be overwritten with zeros.
>
> Cc: Marc Zyniger <marc.zyngier@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>
> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
> index 25daa07..5c1bcf5 100644
> --- a/arch/arm/kvm/vgic.c
> +++ b/arch/arm/kvm/vgic.c
> @@ -233,6 +233,16 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu
> *vcpu, int irq)
> vcpu->arch.vgic_cpu.pending_shared);
> }
>
> +static u32 mmio_data_read(struct kvm_exit_mmio *mmio, u32 mask)
> +{
> + return *((u32 *)mmio->data) & mask;
> +}
> +
> +static void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
> +{
> + *((u32 *)mmio->data) = value & mask;
> +}
> +
> /**
> * vgic_reg_access - access vgic register
> * @mmio: pointer to the data describing the mmio access
> @@ -247,8 +257,8 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu
> *vcpu, int irq)
> static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
> phys_addr_t offset, int mode)
> {
> - int shift = (offset & 3) * 8;
> - u32 mask;
> + int word_offset = (offset & 3) * 8;
> + u32 mask = (1UL << (mmio->len * 8)) - 1;
> u32 regval;
>
> /*
> @@ -256,7 +266,6 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> * directly (ARM ARM B3.12.7 "Prioritization of aborts").
> */
>
> - mask = (~0U) >> shift;
> if (reg) {
> regval = *reg;
> } else {
> @@ -265,7 +274,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> }
>
> if (mmio->is_write) {
> - u32 data = (*((u32 *)mmio->data) & mask) << shift;
> + u32 data = mmio_data_read(mmio, mask) << word_offset;
> switch (ACCESS_WRITE_MASK(mode)) {
> case ACCESS_WRITE_IGNORED:
> return;
> @@ -279,7 +288,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> break;
>
> case ACCESS_WRITE_VALUE:
> - regval = (regval & ~(mask << shift)) | data;
> + regval = (regval & ~(mask << word_offset)) | data;
> break;
> }
> *reg = regval;
> @@ -290,7 +299,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> /* fall through */
>
> case ACCESS_READ_VALUE:
> - *((u32 *)mmio->data) = (regval >> shift) & mask;
> + mmio_data_write(mmio, mask, regval >> word_offset);
> }
> }
> }
> @@ -702,6 +711,12 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu,
> struct kvm_run *run,
> (mmio->phys_addr + mmio->len) > (base + KVM_VGIC_V2_DIST_SIZE))
> return false;
>
> + /* We don't support ldrd / strd or ldm / stm to the emulated vgic */
> + if (mmio->len > 4) {
> + kvm_inject_dabt(vcpu, mmio->phys_addr);
> + return true;
> + }
> +
> range = find_matching_range(vgic_ranges, mmio, base);
> if (unlikely(!range || !range->handle_mmio)) {
> pr_warn("Unhandled access %d %08llx %d\n",
> --
>
> Thanks,
> -Christoffer
> _______________________________________________
> kvmarm mailing list
> kvmarm at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code
2013-01-14 21:08 ` Christoffer Dall
2013-01-14 21:28 ` [kvmarm] " Alexander Graf
@ 2013-01-14 22:50 ` Will Deacon
2013-01-15 10:33 ` Marc Zyngier
2 siblings, 0 replies; 79+ messages in thread
From: Will Deacon @ 2013-01-14 22:50 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jan 14, 2013 at 09:08:54PM +0000, Christoffer Dall wrote:
> On Mon, Jan 14, 2013 at 10:31 AM, Will Deacon <will.deacon@arm.com> wrote:
> > On Tue, Jan 08, 2013 at 06:41:51PM +0000, Christoffer Dall wrote:
> >> + case ACCESS_READ_VALUE:
> >> + *((u32 *)mmio->data) = (regval >> shift) & mask;
> >> + }
> >> + }
> >> +}
> >
> > As I mentioned previously, I suspect that this doesn't work with big-endian
> > systems. Whilst that's reasonable for the moment, a comment would be useful
> > for the unlucky soul that decides to do that work in future (or add
> > accessors for mmio->data as I suggested before).
> >
> admittedly this really hurts my brain, but I think there's actually no
> problem with endianness: whatever comes in mmio->data will have native
> endianness and the vgic is always little-endian, so a guest would have
> to make sure to do its own endianness conversion before writing data,
> or did I get this backwards? (some nasty feeling about if the OS is
> compiled in another endianness than the hardware everything may
> break).
No, you're right. As long as the vgic is always little-endian the access
will be ok.
Sorry for the false alarm,
Will
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code
2013-01-14 21:08 ` Christoffer Dall
2013-01-14 21:28 ` [kvmarm] " Alexander Graf
2013-01-14 22:50 ` Will Deacon
@ 2013-01-15 10:33 ` Marc Zyngier
2 siblings, 0 replies; 79+ messages in thread
From: Marc Zyngier @ 2013-01-15 10:33 UTC (permalink / raw)
To: linux-arm-kernel
On 14/01/13 21:08, Christoffer Dall wrote:
> On Mon, Jan 14, 2013 at 10:31 AM, Will Deacon <will.deacon@arm.com> wrote:
>> As I mentioned previously, I suspect that this doesn't work with big-endian
>> systems. Whilst that's reasonable for the moment, a comment would be useful
>> for the unlucky soul that decides to do that work in future (or add
>> accessors for mmio->data as I suggested before).
>>
> admittedly this really hurts my brain, but I think there's actually no
> problem with endianness: whatever comes in mmio->data will have native
> endianness and the vgic is always little-endian, so a guest would have
> to make sure to do its own endianness conversion before writing data,
> or did I get this backwards? (some nasty feeling about if the OS is
> compiled in another endianness than the hardware everything may
> break).
>
> Anyhow, I think there's another bug in this code though. Please take a
> look and see if you agree:
>
> commit 3cab2b93a6f6acd3c043e584f23b94ab8f1bbd66
> Author: Christoffer Dall <c.dall@virtualopensystems.com>
> Date: Mon Jan 14 15:55:18 2013 -0500
>
> KVM: ARM: Limit vgic read/writes to load/store length
>
> The vgic read/write operations did not consider ldrb/strb masks, and
> would therefore unintentionally overwrite parts of a register.
>
> Consider for example a store of a single byte to a word-aligned address
> of one of the priority registers, that would cause the 3 most
> significant bytes to be overwritten with zeros.
>
> Cc: Marc Zyniger <marc.zyngier@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>
> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
> index 25daa07..5c1bcf5 100644
> --- a/arch/arm/kvm/vgic.c
> +++ b/arch/arm/kvm/vgic.c
> @@ -233,6 +233,16 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu
> *vcpu, int irq)
> vcpu->arch.vgic_cpu.pending_shared);
> }
>
> +static u32 mmio_data_read(struct kvm_exit_mmio *mmio, u32 mask)
> +{
> + return *((u32 *)mmio->data) & mask;
> +}
> +
> +static void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
> +{
> + *((u32 *)mmio->data) = value & mask;
> +}
> +
> /**
> * vgic_reg_access - access vgic register
> * @mmio: pointer to the data describing the mmio access
> @@ -247,8 +257,8 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu
> *vcpu, int irq)
> static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
> phys_addr_t offset, int mode)
> {
> - int shift = (offset & 3) * 8;
> - u32 mask;
> + int word_offset = (offset & 3) * 8;
> + u32 mask = (1UL << (mmio->len * 8)) - 1;
> u32 regval;
>
> /*
> @@ -256,7 +266,6 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> * directly (ARM ARM B3.12.7 "Prioritization of aborts").
> */
>
> - mask = (~0U) >> shift;
> if (reg) {
> regval = *reg;
> } else {
> @@ -265,7 +274,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> }
>
> if (mmio->is_write) {
> - u32 data = (*((u32 *)mmio->data) & mask) << shift;
> + u32 data = mmio_data_read(mmio, mask) << word_offset;
> switch (ACCESS_WRITE_MASK(mode)) {
> case ACCESS_WRITE_IGNORED:
> return;
> @@ -279,7 +288,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> break;
>
> case ACCESS_WRITE_VALUE:
> - regval = (regval & ~(mask << shift)) | data;
> + regval = (regval & ~(mask << word_offset)) | data;
> break;
> }
> *reg = regval;
> @@ -290,7 +299,7 @@ static void vgic_reg_access(struct kvm_exit_mmio
> *mmio, u32 *reg,
> /* fall through */
>
> case ACCESS_READ_VALUE:
> - *((u32 *)mmio->data) = (regval >> shift) & mask;
> + mmio_data_write(mmio, mask, regval >> word_offset);
> }
> }
> }
> @@ -702,6 +711,12 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu,
> struct kvm_run *run,
> (mmio->phys_addr + mmio->len) > (base + KVM_VGIC_V2_DIST_SIZE))
> return false;
>
> + /* We don't support ldrd / strd or ldm / stm to the emulated vgic */
> + if (mmio->len > 4) {
> + kvm_inject_dabt(vcpu, mmio->phys_addr);
> + return true;
> + }
> +
> range = find_matching_range(vgic_ranges, mmio, base);
> if (unlikely(!range || !range->handle_mmio)) {
> pr_warn("Unhandled access %d %08llx %d\n",
> --
>
> Thanks,
> -Christoffer
>
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 05/12] ARM: KVM: VGIC accept vcpu and dist base addresses from user space
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (3 preceding siblings ...)
2013-01-08 18:41 ` [PATCH v5 04/12] ARM: KVM: Initial VGIC infrastructure code Christoffer Dall
@ 2013-01-08 18:41 ` Christoffer Dall
2013-01-08 18:42 ` [PATCH v5 06/12] ARM: KVM: VGIC distributor handling Christoffer Dall
` (7 subsequent siblings)
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:41 UTC (permalink / raw)
To: linux-arm-kernel
User space defines the model to emulate to a guest and should therefore
decide which addresses are used for both the virtual CPU interface
directly mapped in the guest physical address space and for the emulated
distributor interface, which is mapped in software by the in-kernel VGIC
support.
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
Documentation/virtual/kvm/api.txt | 1 +
arch/arm/include/asm/kvm_vgic.h | 9 +++++
arch/arm/include/uapi/asm/kvm.h | 3 ++
arch/arm/kvm/arm.c | 14 ++++++++
arch/arm/kvm/vgic.c | 62 +++++++++++++++++++++++++++++++++++++
5 files changed, 88 insertions(+), 1 deletion(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 668956f..34d3ee9 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2218,6 +2218,7 @@ Errors:
ENXIO: Device not supported on current system
EEXIST: Address already set
E2BIG: Address outside guest physical address space
+ EBUSY: Address overlaps with other device range
struct kvm_device_address {
__u64 id;
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index fcfd530..270dcd2 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -22,6 +22,9 @@
#include <asm/hardware/gic.h>
struct vgic_dist {
+ /* Distributor and vcpu interface mapping in the guest */
+ phys_addr_t vgic_dist_base;
+ phys_addr_t vgic_cpu_base;
};
struct vgic_cpu {
@@ -33,6 +36,7 @@ struct kvm_run;
struct kvm_exit_mmio;
#ifdef CONFIG_KVM_ARM_VGIC
+int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_exit_mmio *mmio);
@@ -42,6 +46,11 @@ static inline int kvm_vgic_hyp_init(void)
return 0;
}
+static inline int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
+{
+ return 0;
+}
+
static inline int kvm_vgic_init(struct kvm *kvm)
{
return 0;
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 09911a7..94e893b 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -78,6 +78,9 @@ struct kvm_regs {
#define KVM_VGIC_V2_ADDR_TYPE_DIST 0
#define KVM_VGIC_V2_ADDR_TYPE_CPU 1
+#define KVM_VGIC_V2_DIST_SIZE 0x1000
+#define KVM_VGIC_V2_CPU_SIZE 0x2000
+
struct kvm_vcpu_init {
__u32 target;
__u32 features[7];
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 85c4cdf..4c2b057 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -858,7 +858,19 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
static int kvm_vm_ioctl_set_device_address(struct kvm *kvm,
struct kvm_device_address *dev_addr)
{
- return -ENODEV;
+ unsigned long dev_id, type;
+
+ dev_id = (dev_addr->id & KVM_DEVICE_ID_MASK) >> KVM_DEVICE_ID_SHIFT;
+ type = (dev_addr->id & KVM_DEVICE_TYPE_MASK) >> KVM_DEVICE_TYPE_SHIFT;
+
+ switch (dev_id) {
+ case KVM_ARM_DEVICE_VGIC_V2:
+ if (!vgic_present)
+ return -ENXIO;
+ return kvm_vgic_set_addr(kvm, type, dev_addr->addr);
+ default:
+ return -ENODEV;
+ }
}
long kvm_arch_vm_ioctl(struct file *filp,
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 1feee5a..cdb7671 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -22,6 +22,9 @@
#include <linux/io.h>
#include <asm/kvm_emulate.h>
+#define VGIC_ADDR_UNDEF (-1)
+#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
+
#define ACCESS_READ_VALUE (1 << 0)
#define ACCESS_READ_RAZ (0 << 0)
#define ACCESS_READ_MASK(x) ((x) & (1 << 0))
@@ -142,3 +145,62 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
{
return KVM_EXIT_MMIO;
}
+
+static bool vgic_ioaddr_overlap(struct kvm *kvm)
+{
+ phys_addr_t dist = kvm->arch.vgic.vgic_dist_base;
+ phys_addr_t cpu = kvm->arch.vgic.vgic_cpu_base;
+
+ if (IS_VGIC_ADDR_UNDEF(dist) || IS_VGIC_ADDR_UNDEF(cpu))
+ return 0;
+ if ((dist <= cpu && dist + KVM_VGIC_V2_DIST_SIZE > cpu) ||
+ (cpu <= dist && cpu + KVM_VGIC_V2_CPU_SIZE > dist))
+ return -EBUSY;
+ return 0;
+}
+
+static int vgic_ioaddr_assign(struct kvm *kvm, phys_addr_t *ioaddr,
+ phys_addr_t addr, phys_addr_t size)
+{
+ int ret;
+
+ if (!IS_VGIC_ADDR_UNDEF(*ioaddr))
+ return -EEXIST;
+ if (addr + size < addr)
+ return -EINVAL;
+
+ ret = vgic_ioaddr_overlap(kvm);
+ if (ret)
+ return ret;
+ *ioaddr = addr;
+ return ret;
+}
+
+int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
+{
+ int r = 0;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+
+ if (addr & ~KVM_PHYS_MASK)
+ return -E2BIG;
+
+ if (addr & ~PAGE_MASK)
+ return -EINVAL;
+
+ mutex_lock(&kvm->lock);
+ switch (type) {
+ case KVM_VGIC_V2_ADDR_TYPE_DIST:
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_dist_base,
+ addr, KVM_VGIC_V2_DIST_SIZE);
+ break;
+ case KVM_VGIC_V2_ADDR_TYPE_CPU:
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_cpu_base,
+ addr, KVM_VGIC_V2_CPU_SIZE);
+ break;
+ default:
+ r = -ENODEV;
+ }
+
+ mutex_unlock(&kvm->lock);
+ return r;
+}
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 06/12] ARM: KVM: VGIC distributor handling
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (4 preceding siblings ...)
2013-01-08 18:41 ` [PATCH v5 05/12] ARM: KVM: VGIC accept vcpu and dist base addresses from user space Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-14 15:39 ` Will Deacon
2013-01-08 18:42 ` [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management Christoffer Dall
` (6 subsequent siblings)
12 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
Add the GIC distributor emulation code. A number of the GIC features
are simply ignored as they are not required to boot a Linux guest.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/kvm_vgic.h | 82 +++++
arch/arm/kvm/vgic.c | 593 +++++++++++++++++++++++++++++++++++++++
2 files changed, 674 insertions(+), 1 deletion(-)
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index 270dcd2..9ff0d9c 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -19,12 +19,94 @@
#ifndef __ASM_ARM_KVM_VGIC_H
#define __ASM_ARM_KVM_VGIC_H
+#include <linux/kernel.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <linux/irqreturn.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
#include <asm/hardware/gic.h>
+#define VGIC_NR_IRQS 128
+#define VGIC_NR_SGIS 16
+#define VGIC_NR_PPIS 16
+#define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
+#define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
+#define VGIC_MAX_CPUS KVM_MAX_VCPUS
+
+/* Sanity checks... */
+#if (VGIC_MAX_CPUS > 8)
+#error Invalid number of CPU interfaces
+#endif
+
+#if (VGIC_NR_IRQS & 31)
+#error "VGIC_NR_IRQS must be a multiple of 32"
+#endif
+
+#if (VGIC_NR_IRQS > 1024)
+#error "VGIC_NR_IRQS must be <= 1024"
+#endif
+
+/*
+ * The GIC distributor registers describing interrupts have two parts:
+ * - 32 per-CPU interrupts (SGI + PPI)
+ * - a bunch of shared interrupts (SPI)
+ */
+struct vgic_bitmap {
+ union {
+ u32 reg[VGIC_NR_PRIVATE_IRQS / 32];
+ DECLARE_BITMAP(reg_ul, VGIC_NR_PRIVATE_IRQS);
+ } percpu[VGIC_MAX_CPUS];
+ union {
+ u32 reg[VGIC_NR_SHARED_IRQS / 32];
+ DECLARE_BITMAP(reg_ul, VGIC_NR_SHARED_IRQS);
+ } shared;
+};
+
+struct vgic_bytemap {
+ u32 percpu[VGIC_MAX_CPUS][VGIC_NR_PRIVATE_IRQS / 4];
+ u32 shared[VGIC_NR_SHARED_IRQS / 4];
+};
+
struct vgic_dist {
+#ifdef CONFIG_KVM_ARM_VGIC
+ spinlock_t lock;
+
+ /* Virtual control interface mapping */
+ void __iomem *vctrl_base;
+
/* Distributor and vcpu interface mapping in the guest */
phys_addr_t vgic_dist_base;
phys_addr_t vgic_cpu_base;
+
+ /* Distributor enabled */
+ u32 enabled;
+
+ /* Interrupt enabled (one bit per IRQ) */
+ struct vgic_bitmap irq_enabled;
+
+ /* Interrupt 'pin' level */
+ struct vgic_bitmap irq_state;
+
+ /* Level-triggered interrupt in progress */
+ struct vgic_bitmap irq_active;
+
+ /* Interrupt priority. Not used yet. */
+ struct vgic_bytemap irq_priority;
+
+ /* Level/edge triggered */
+ struct vgic_bitmap irq_cfg;
+
+ /* Source CPU per SGI and target CPU */
+ u8 irq_sgi_sources[VGIC_MAX_CPUS][16];
+
+ /* Target CPU for each IRQ */
+ u8 irq_spi_cpu[VGIC_NR_SHARED_IRQS];
+ struct vgic_bitmap irq_spi_target[VGIC_MAX_CPUS];
+
+ /* Bitmap indicating which CPU has something pending */
+ unsigned long irq_pending_on_cpu;
+#endif
};
struct vgic_cpu {
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index cdb7671..bd2bd7f 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -22,6 +22,43 @@
#include <linux/io.h>
#include <asm/kvm_emulate.h>
+/*
+ * How the whole thing works (courtesy of Christoffer Dall):
+ *
+ * - At any time, the dist->irq_pending_on_cpu is the oracle that knows if
+ * something is pending
+ * - VGIC pending interrupts are stored on the vgic.irq_state vgic
+ * bitmap (this bitmap is updated by both user land ioctls and guest
+ * mmio ops, and other in-kernel peripherals such as the
+ * arch. timers) and indicate the 'wire' state.
+ * - Every time the bitmap changes, the irq_pending_on_cpu oracle is
+ * recalculated
+ * - To calculate the oracle, we need info for each cpu from
+ * compute_pending_for_cpu, which considers:
+ * - PPI: dist->irq_state & dist->irq_enable
+ * - SPI: dist->irq_state & dist->irq_enable & dist->irq_spi_target
+ * - irq_spi_target is a 'formatted' version of the GICD_ICFGR
+ * registers, stored on each vcpu. We only keep one bit of
+ * information per interrupt, making sure that only one vcpu can
+ * accept the interrupt.
+ * - The same is true when injecting an interrupt, except that we only
+ * consider a single interrupt at a time. The irq_spi_cpu array
+ * contains the target CPU for each SPI.
+ *
+ * The handling of level interrupts adds some extra complexity. We
+ * need to track when the interrupt has been EOIed, so we can sample
+ * the 'line' again. This is achieved as such:
+ *
+ * - When a level interrupt is moved onto a vcpu, the corresponding
+ * bit in irq_active is set. As long as this bit is set, the line
+ * will be ignored for further interrupts. The interrupt is injected
+ * into the vcpu with the GICH_LR_EOI bit set (generate a
+ * maintenance interrupt on EOI).
+ * - When the interrupt is EOIed, the maintenance interrupt fires,
+ * and clears the corresponding bit in irq_active. This allow the
+ * interrupt line to be sampled again.
+ */
+
#define VGIC_ADDR_UNDEF (-1)
#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
@@ -34,6 +71,119 @@
#define ACCESS_WRITE_VALUE (3 << 1)
#define ACCESS_WRITE_MASK(x) ((x) & (3 << 1))
+static void vgic_update_state(struct kvm *kvm);
+static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg);
+
+static u32 *vgic_bitmap_get_reg(struct vgic_bitmap *x,
+ int cpuid, u32 offset)
+{
+ offset >>= 2;
+ if (!offset)
+ return x->percpu[cpuid].reg;
+ else
+ return x->shared.reg + offset - 1;
+}
+
+static int vgic_bitmap_get_irq_val(struct vgic_bitmap *x,
+ int cpuid, int irq)
+{
+ if (irq < VGIC_NR_PRIVATE_IRQS)
+ return test_bit(irq, x->percpu[cpuid].reg_ul);
+
+ return test_bit(irq - VGIC_NR_PRIVATE_IRQS, x->shared.reg_ul);
+}
+
+static void vgic_bitmap_set_irq_val(struct vgic_bitmap *x, int cpuid,
+ int irq, int val)
+{
+ unsigned long *reg;
+
+ if (irq < VGIC_NR_PRIVATE_IRQS) {
+ reg = x->percpu[cpuid].reg_ul;
+ } else {
+ reg = x->shared.reg_ul;
+ irq -= VGIC_NR_PRIVATE_IRQS;
+ }
+
+ if (val)
+ set_bit(irq, reg);
+ else
+ clear_bit(irq, reg);
+}
+
+static unsigned long *vgic_bitmap_get_cpu_map(struct vgic_bitmap *x, int cpuid)
+{
+ if (unlikely(cpuid >= VGIC_MAX_CPUS))
+ return NULL;
+ return x->percpu[cpuid].reg_ul;
+}
+
+static unsigned long *vgic_bitmap_get_shared_map(struct vgic_bitmap *x)
+{
+ return x->shared.reg_ul;
+}
+
+static u32 *vgic_bytemap_get_reg(struct vgic_bytemap *x, int cpuid, u32 offset)
+{
+ offset >>= 2;
+ BUG_ON(offset > (VGIC_NR_IRQS / 4));
+ if (offset < 4)
+ return x->percpu[cpuid] + offset;
+ else
+ return x->shared + offset - 8;
+}
+
+#define VGIC_CFG_LEVEL 0
+#define VGIC_CFG_EDGE 1
+
+static bool vgic_irq_is_edge(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int irq_val;
+
+ irq_val = vgic_bitmap_get_irq_val(&dist->irq_cfg, vcpu->vcpu_id, irq);
+ return irq_val == VGIC_CFG_EDGE;
+}
+
+static int vgic_irq_is_enabled(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ return vgic_bitmap_get_irq_val(&dist->irq_enabled, vcpu->vcpu_id, irq);
+}
+
+static void vgic_dist_irq_set(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ vgic_bitmap_set_irq_val(&dist->irq_state, vcpu->vcpu_id, irq, 1);
+}
+
+static void vgic_dist_irq_clear(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ vgic_bitmap_set_irq_val(&dist->irq_state, vcpu->vcpu_id, irq, 0);
+}
+
+static void vgic_cpu_irq_set(struct kvm_vcpu *vcpu, int irq)
+{
+ if (irq < VGIC_NR_PRIVATE_IRQS)
+ set_bit(irq, vcpu->arch.vgic_cpu.pending_percpu);
+ else
+ set_bit(irq - VGIC_NR_PRIVATE_IRQS,
+ vcpu->arch.vgic_cpu.pending_shared);
+}
+
+static void vgic_cpu_irq_clear(struct kvm_vcpu *vcpu, int irq)
+{
+ if (irq < VGIC_NR_PRIVATE_IRQS)
+ clear_bit(irq, vcpu->arch.vgic_cpu.pending_percpu);
+ else
+ clear_bit(irq - VGIC_NR_PRIVATE_IRQS,
+ vcpu->arch.vgic_cpu.pending_shared);
+}
+
/**
* vgic_reg_access - access vgic register
* @mmio: pointer to the data describing the mmio access
@@ -96,6 +246,294 @@ static void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
}
}
+static bool handle_mmio_misc(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ u32 reg;
+ u32 word_offset = offset & 3;
+
+ switch (offset & ~3) {
+ case 0: /* CTLR */
+ reg = vcpu->kvm->arch.vgic.enabled;
+ vgic_reg_access(mmio, ®, word_offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
+ if (mmio->is_write) {
+ vcpu->kvm->arch.vgic.enabled = reg & 1;
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+ break;
+
+ case 4: /* TYPER */
+ reg = (atomic_read(&vcpu->kvm->online_vcpus) - 1) << 5;
+ reg |= (VGIC_NR_IRQS >> 5) - 1;
+ vgic_reg_access(mmio, ®, word_offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
+ break;
+
+ case 8: /* IIDR */
+ reg = 0x4B00043B;
+ vgic_reg_access(mmio, ®, word_offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
+ break;
+ }
+
+ return false;
+}
+
+static bool handle_mmio_raz_wi(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ vgic_reg_access(mmio, NULL, offset,
+ ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED);
+ return false;
+}
+
+static bool handle_mmio_set_enable_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 *reg = vgic_bitmap_get_reg(&vcpu->kvm->arch.vgic.irq_enabled,
+ vcpu->vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_SETBIT);
+ if (mmio->is_write) {
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+
+ return false;
+}
+
+static bool handle_mmio_clear_enable_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 *reg = vgic_bitmap_get_reg(&vcpu->kvm->arch.vgic.irq_enabled,
+ vcpu->vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
+ if (mmio->is_write) {
+ if (offset < 4) /* Force SGI enabled */
+ *reg |= 0xffff;
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+
+ return false;
+}
+
+static bool handle_mmio_set_pending_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 *reg = vgic_bitmap_get_reg(&vcpu->kvm->arch.vgic.irq_state,
+ vcpu->vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_SETBIT);
+ if (mmio->is_write) {
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+
+ return false;
+}
+
+static bool handle_mmio_clear_pending_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 *reg = vgic_bitmap_get_reg(&vcpu->kvm->arch.vgic.irq_state,
+ vcpu->vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
+ if (mmio->is_write) {
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+
+ return false;
+}
+
+static bool handle_mmio_priority_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 *reg = vgic_bytemap_get_reg(&vcpu->kvm->arch.vgic.irq_priority,
+ vcpu->vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
+ return false;
+}
+
+#define GICD_ITARGETSR_SIZE 32
+#define GICD_CPUTARGETS_BITS 8
+#define GICD_IRQS_PER_ITARGETSR (GICD_ITARGETSR_SIZE / GICD_CPUTARGETS_BITS)
+static u32 vgic_get_target_reg(struct kvm *kvm, int irq)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu;
+ int i, c;
+ unsigned long *bmap;
+ u32 val = 0;
+
+ irq -= VGIC_NR_PRIVATE_IRQS;
+
+ kvm_for_each_vcpu(c, vcpu, kvm) {
+ bmap = vgic_bitmap_get_shared_map(&dist->irq_spi_target[c]);
+ for (i = 0; i < GICD_IRQS_PER_ITARGETSR; i++)
+ if (test_bit(irq + i, bmap))
+ val |= 1 << (c + i * 8);
+ }
+
+ return val;
+}
+
+static void vgic_set_target_reg(struct kvm *kvm, u32 val, int irq)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu;
+ int i, c;
+ unsigned long *bmap;
+ u32 target;
+
+ BUG_ON(irq & 3);
+ BUG_ON(irq < VGIC_NR_PRIVATE_IRQS);
+
+ irq -= VGIC_NR_PRIVATE_IRQS;
+
+ /*
+ * Pick the LSB in each byte. This ensures we target exactly
+ * one vcpu per IRQ. If the byte is null, assume we target
+ * CPU0.
+ */
+ for (i = 0; i < GICD_IRQS_PER_ITARGETSR; i++) {
+ int shift = i * GICD_CPUTARGETS_BITS;
+ target = ffs((val >> shift) & 0xffU);
+ target = target ? (target - 1) : 0;
+ dist->irq_spi_cpu[irq + i] = target;
+ kvm_for_each_vcpu(c, vcpu, kvm) {
+ bmap = vgic_bitmap_get_shared_map(&dist->irq_spi_target[c]);
+ if (c == target)
+ set_bit(irq + i, bmap);
+ else
+ clear_bit(irq + i, bmap);
+ }
+ }
+}
+
+static bool handle_mmio_target_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 reg;
+
+ /* We treat the banked interrupts targets as read-only */
+ if (offset < 32) {
+ u32 roreg = 1 << vcpu->vcpu_id;
+ roreg |= roreg << 8;
+ roreg |= roreg << 16;
+
+ vgic_reg_access(mmio, &roreg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
+ return false;
+ }
+
+ reg = vgic_get_target_reg(vcpu->kvm, offset & ~3U);
+ vgic_reg_access(mmio, ®, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
+ if (mmio->is_write) {
+ vgic_set_target_reg(vcpu->kvm, reg, offset & ~3U);
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+
+ return false;
+}
+
+static u32 vgic_cfg_expand(u16 val)
+{
+ u32 res = 0;
+ int i;
+
+ /*
+ * Turn a 16bit value like abcd...mnop into a 32bit word
+ * a0b0c0d0...m0n0o0p0, which is what the HW cfg register is.
+ */
+ for (i = 0; i < 16; i++)
+ res |= ((val >> i) & VGIC_CFG_EDGE) << (2 * i + 1);
+
+ return res;
+}
+
+static u16 vgic_cfg_compress(u32 val)
+{
+ u16 res = 0;
+ int i;
+
+ /*
+ * Turn a 32bit word a0b0c0d0...m0n0o0p0 into 16bit value like
+ * abcd...mnop which is what we really care about.
+ */
+ for (i = 0; i < 16; i++)
+ res |= ((val >> (i * 2 + 1)) & VGIC_CFG_EDGE) << i;
+
+ return res;
+}
+
+/*
+ * The distributor uses 2 bits per IRQ for the CFG register, but the
+ * LSB is always 0. As such, we only keep the upper bit, and use the
+ * two above functions to compress/expand the bits
+ */
+static bool handle_mmio_cfg_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ u32 val;
+ u32 *reg = vgic_bitmap_get_reg(&vcpu->kvm->arch.vgic.irq_cfg,
+ vcpu->vcpu_id, offset >> 1);
+ if (offset & 2)
+ val = *reg >> 16;
+ else
+ val = *reg & 0xffff;
+
+ val = vgic_cfg_expand(val);
+ vgic_reg_access(mmio, &val, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
+ if (mmio->is_write) {
+ if (offset < 4) {
+ *reg = ~0U; /* Force PPIs/SGIs to 1 */
+ return false;
+ }
+
+ val = vgic_cfg_compress(val);
+ if (offset & 2) {
+ *reg &= 0xffff;
+ *reg |= val << 16;
+ } else {
+ *reg &= 0xffff << 16;
+ *reg |= val;
+ }
+ }
+
+ return false;
+}
+
+static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ u32 reg;
+ vgic_reg_access(mmio, ®, offset,
+ ACCESS_READ_RAZ | ACCESS_WRITE_VALUE);
+ if (mmio->is_write) {
+ vgic_dispatch_sgi(vcpu, reg);
+ vgic_update_state(vcpu->kvm);
+ return true;
+ }
+
+ return false;
+}
+
/*
* I would have liked to use the kvm_bus_io_*() API instead, but it
* cannot cope with banked registers (only the VM pointer is passed
@@ -110,6 +548,66 @@ struct mmio_range {
};
static const struct mmio_range vgic_ranges[] = {
+ { /* CTRL, TYPER, IIDR */
+ .base = 0,
+ .len = 12,
+ .handle_mmio = handle_mmio_misc,
+ },
+ { /* IGROUPRn */
+ .base = 0x80,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_raz_wi,
+ },
+ { /* ISENABLERn */
+ .base = 0x100,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_set_enable_reg,
+ },
+ { /* ICENABLERn */
+ .base = 0x180,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_clear_enable_reg,
+ },
+ { /* ISPENDRn */
+ .base = 0x200,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_set_pending_reg,
+ },
+ { /* ICPENDRn */
+ .base = 0x280,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_clear_pending_reg,
+ },
+ { /* ISACTIVERn */
+ .base = 0x300,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_raz_wi,
+ },
+ { /* ICACTIVERn */
+ .base = 0x380,
+ .len = VGIC_NR_IRQS / 8,
+ .handle_mmio = handle_mmio_raz_wi,
+ },
+ { /* IPRIORITYRn */
+ .base = 0x400,
+ .len = VGIC_NR_IRQS,
+ .handle_mmio = handle_mmio_priority_reg,
+ },
+ { /* ITARGETSRn */
+ .base = 0x800,
+ .len = VGIC_NR_IRQS,
+ .handle_mmio = handle_mmio_target_reg,
+ },
+ { /* ICFGRn */
+ .base = 0xC00,
+ .len = VGIC_NR_IRQS / 4,
+ .handle_mmio = handle_mmio_cfg_reg,
+ },
+ { /* SGIRn */
+ .base = 0xF00,
+ .len = 4,
+ .handle_mmio = handle_mmio_sgi_reg,
+ },
{}
};
@@ -143,7 +641,100 @@ struct mmio_range *find_matching_range(const struct mmio_range *ranges,
bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_exit_mmio *mmio)
{
- return KVM_EXIT_MMIO;
+ const struct mmio_range *range;
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ unsigned long base = dist->vgic_dist_base;
+ bool updated_state;
+ unsigned long offset;
+
+ if (!irqchip_in_kernel(vcpu->kvm) ||
+ mmio->phys_addr < base ||
+ (mmio->phys_addr + mmio->len) > (base + KVM_VGIC_V2_DIST_SIZE))
+ return false;
+
+ range = find_matching_range(vgic_ranges, mmio, base);
+ if (unlikely(!range || !range->handle_mmio)) {
+ pr_warn("Unhandled access %d %08llx %d\n",
+ mmio->is_write, mmio->phys_addr, mmio->len);
+ return false;
+ }
+
+ spin_lock(&vcpu->kvm->arch.vgic.lock);
+ offset = mmio->phys_addr - range->base - base;
+ updated_state = range->handle_mmio(vcpu, mmio, offset);
+ spin_unlock(&vcpu->kvm->arch.vgic.lock);
+ kvm_prepare_mmio(run, mmio);
+ kvm_handle_mmio_return(vcpu, run);
+
+ return true;
+}
+
+static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ int nrcpus = atomic_read(&kvm->online_vcpus);
+ u8 target_cpus;
+ int sgi, mode, c, vcpu_id;
+
+ vcpu_id = vcpu->vcpu_id;
+
+ sgi = reg & 0xf;
+ target_cpus = (reg >> 16) & 0xff;
+ mode = (reg >> 24) & 3;
+
+ switch (mode) {
+ case 0:
+ if (!target_cpus)
+ return;
+
+ case 1:
+ target_cpus = ((1 << nrcpus) - 1) & ~(1 << vcpu_id) & 0xff;
+ break;
+
+ case 2:
+ target_cpus = 1 << vcpu_id;
+ break;
+ }
+
+ kvm_for_each_vcpu(c, vcpu, kvm) {
+ if (target_cpus & 1) {
+ /* Flag the SGI as pending */
+ vgic_dist_irq_set(vcpu, sgi);
+ dist->irq_sgi_sources[c][sgi] |= 1 << vcpu_id;
+ kvm_debug("SGI%d from CPU%d to CPU%d\n", sgi, vcpu_id, c);
+ }
+
+ target_cpus >>= 1;
+ }
+}
+
+static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
+{
+ return 0;
+}
+
+/*
+ * Update the interrupt state and determine which CPUs have pending
+ * interrupts. Must be called with distributor lock held.
+ */
+static void vgic_update_state(struct kvm *kvm)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu;
+ int c;
+
+ if (!dist->enabled) {
+ set_bit(0, &dist->irq_pending_on_cpu);
+ return;
+ }
+
+ kvm_for_each_vcpu(c, vcpu, kvm) {
+ if (compute_pending_for_cpu(vcpu)) {
+ pr_debug("CPU%d has pending interrupts\n", c);
+ set_bit(c, &dist->irq_pending_on_cpu);
+ }
+ }
}
static bool vgic_ioaddr_overlap(struct kvm *kvm)
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 06/12] ARM: KVM: VGIC distributor handling
2013-01-08 18:42 ` [PATCH v5 06/12] ARM: KVM: VGIC distributor handling Christoffer Dall
@ 2013-01-14 15:39 ` Will Deacon
2013-01-14 21:55 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Will Deacon @ 2013-01-14 15:39 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Jan 08, 2013 at 06:42:04PM +0000, Christoffer Dall wrote:
> From: Marc Zyngier <marc.zyngier@arm.com>
>
> Add the GIC distributor emulation code. A number of the GIC features
> are simply ignored as they are not required to boot a Linux guest.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
> ---
> arch/arm/include/asm/kvm_vgic.h | 82 +++++
> arch/arm/kvm/vgic.c | 593 +++++++++++++++++++++++++++++++++++++++
> 2 files changed, 674 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
> index 270dcd2..9ff0d9c 100644
> --- a/arch/arm/include/asm/kvm_vgic.h
> +++ b/arch/arm/include/asm/kvm_vgic.h
> @@ -19,12 +19,94 @@
> #ifndef __ASM_ARM_KVM_VGIC_H
> #define __ASM_ARM_KVM_VGIC_H
>
> +#include <linux/kernel.h>
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +#include <linux/irqreturn.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
> #include <asm/hardware/gic.h>
>
> +#define VGIC_NR_IRQS 128
> +#define VGIC_NR_SGIS 16
Now that you have this, you can use it in a few places (see also the BUG_ONs
in vgic_queue_irq).
> +#define VGIC_NR_PPIS 16
> +#define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
> +#define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
> +#define VGIC_MAX_CPUS KVM_MAX_VCPUS
> +
> +/* Sanity checks... */
> +#if (VGIC_MAX_CPUS > 8)
> +#error Invalid number of CPU interfaces
> +#endif
> +
> +#if (VGIC_NR_IRQS & 31)
> +#error "VGIC_NR_IRQS must be a multiple of 32"
> +#endif
> +
> +#if (VGIC_NR_IRQS > 1024)
> +#error "VGIC_NR_IRQS must be <= 1024"
> +#endif
> +
> +/*
> + * The GIC distributor registers describing interrupts have two parts:
> + * - 32 per-CPU interrupts (SGI + PPI)
> + * - a bunch of shared interrupts (SPI)
> + */
> +struct vgic_bitmap {
> + union {
> + u32 reg[VGIC_NR_PRIVATE_IRQS / 32];
> + DECLARE_BITMAP(reg_ul, VGIC_NR_PRIVATE_IRQS);
> + } percpu[VGIC_MAX_CPUS];
> + union {
> + u32 reg[VGIC_NR_SHARED_IRQS / 32];
> + DECLARE_BITMAP(reg_ul, VGIC_NR_SHARED_IRQS);
> + } shared;
> +};
> +
> +struct vgic_bytemap {
> + u32 percpu[VGIC_MAX_CPUS][VGIC_NR_PRIVATE_IRQS / 4];
> + u32 shared[VGIC_NR_SHARED_IRQS / 4];
> +};
> +
> struct vgic_dist {
> +#ifdef CONFIG_KVM_ARM_VGIC
> + spinlock_t lock;
> +
> + /* Virtual control interface mapping */
> + void __iomem *vctrl_base;
> +
> /* Distributor and vcpu interface mapping in the guest */
> phys_addr_t vgic_dist_base;
> phys_addr_t vgic_cpu_base;
> +
> + /* Distributor enabled */
> + u32 enabled;
> +
> + /* Interrupt enabled (one bit per IRQ) */
> + struct vgic_bitmap irq_enabled;
> +
> + /* Interrupt 'pin' level */
> + struct vgic_bitmap irq_state;
> +
> + /* Level-triggered interrupt in progress */
> + struct vgic_bitmap irq_active;
> +
> + /* Interrupt priority. Not used yet. */
> + struct vgic_bytemap irq_priority;
> +
> + /* Level/edge triggered */
> + struct vgic_bitmap irq_cfg;
> +
> + /* Source CPU per SGI and target CPU */
> + u8 irq_sgi_sources[VGIC_MAX_CPUS][16];
VGIC_NR_SGIS
> +static u32 vgic_get_target_reg(struct kvm *kvm, int irq)
> +{
> + struct vgic_dist *dist = &kvm->arch.vgic;
> + struct kvm_vcpu *vcpu;
> + int i, c;
> + unsigned long *bmap;
> + u32 val = 0;
> +
> + irq -= VGIC_NR_PRIVATE_IRQS;
> +
> + kvm_for_each_vcpu(c, vcpu, kvm) {
> + bmap = vgic_bitmap_get_shared_map(&dist->irq_spi_target[c]);
> + for (i = 0; i < GICD_IRQS_PER_ITARGETSR; i++)
> + if (test_bit(irq + i, bmap))
> + val |= 1 << (c + i * 8);
> + }
> +
> + return val;
> +}
> +
> +static void vgic_set_target_reg(struct kvm *kvm, u32 val, int irq)
> +{
> + struct vgic_dist *dist = &kvm->arch.vgic;
> + struct kvm_vcpu *vcpu;
> + int i, c;
> + unsigned long *bmap;
> + u32 target;
> +
> + BUG_ON(irq & 3);
> + BUG_ON(irq < VGIC_NR_PRIVATE_IRQS);
This is now different to vgic_Get_target_reg, which doesn't have the
BUG_ONs. Can we remove these ones too?
> + irq -= VGIC_NR_PRIVATE_IRQS;
> +
> + /*
> + * Pick the LSB in each byte. This ensures we target exactly
> + * one vcpu per IRQ. If the byte is null, assume we target
> + * CPU0.
> + */
> + for (i = 0; i < GICD_IRQS_PER_ITARGETSR; i++) {
> + int shift = i * GICD_CPUTARGETS_BITS;
> + target = ffs((val >> shift) & 0xffU);
> + target = target ? (target - 1) : 0;
> + dist->irq_spi_cpu[irq + i] = target;
> + kvm_for_each_vcpu(c, vcpu, kvm) {
> + bmap = vgic_bitmap_get_shared_map(&dist->irq_spi_target[c]);
> + if (c == target)
> + set_bit(irq + i, bmap);
> + else
> + clear_bit(irq + i, bmap);
> + }
> + }
> +}
[...]
> static const struct mmio_range vgic_ranges[] = {
> + { /* CTRL, TYPER, IIDR */
> + .base = 0,
> + .len = 12,
> + .handle_mmio = handle_mmio_misc,
> + },
> + { /* IGROUPRn */
> + .base = 0x80,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_raz_wi,
> + },
> + { /* ISENABLERn */
> + .base = 0x100,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_set_enable_reg,
> + },
> + { /* ICENABLERn */
> + .base = 0x180,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_clear_enable_reg,
> + },
> + { /* ISPENDRn */
> + .base = 0x200,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_set_pending_reg,
> + },
> + { /* ICPENDRn */
> + .base = 0x280,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_clear_pending_reg,
> + },
> + { /* ISACTIVERn */
> + .base = 0x300,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_raz_wi,
> + },
> + { /* ICACTIVERn */
> + .base = 0x380,
> + .len = VGIC_NR_IRQS / 8,
> + .handle_mmio = handle_mmio_raz_wi,
> + },
> + { /* IPRIORITYRn */
> + .base = 0x400,
> + .len = VGIC_NR_IRQS,
> + .handle_mmio = handle_mmio_priority_reg,
> + },
> + { /* ITARGETSRn */
> + .base = 0x800,
> + .len = VGIC_NR_IRQS,
> + .handle_mmio = handle_mmio_target_reg,
> + },
> + { /* ICFGRn */
> + .base = 0xC00,
> + .len = VGIC_NR_IRQS / 4,
> + .handle_mmio = handle_mmio_cfg_reg,
> + },
> + { /* SGIRn */
> + .base = 0xF00,
> + .len = 4,
> + .handle_mmio = handle_mmio_sgi_reg,
> + },
> {}
> };
You've added named definitions for these constants to the GIC header file,
so please replace these immediates with those and delete the comments.
Will
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 06/12] ARM: KVM: VGIC distributor handling
2013-01-14 15:39 ` Will Deacon
@ 2013-01-14 21:55 ` Christoffer Dall
0 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-14 21:55 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jan 14, 2013 at 10:39 AM, Will Deacon <will.deacon@arm.com> wrote:
> On Tue, Jan 08, 2013 at 06:42:04PM +0000, Christoffer Dall wrote:
>> From: Marc Zyngier <marc.zyngier@arm.com>
>>
>> Add the GIC distributor emulation code. A number of the GIC features
>> are simply ignored as they are not required to boot a Linux guest.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>> ---
>> arch/arm/include/asm/kvm_vgic.h | 82 +++++
>> arch/arm/kvm/vgic.c | 593 +++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 674 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>> index 270dcd2..9ff0d9c 100644
>> --- a/arch/arm/include/asm/kvm_vgic.h
>> +++ b/arch/arm/include/asm/kvm_vgic.h
>> @@ -19,12 +19,94 @@
>> #ifndef __ASM_ARM_KVM_VGIC_H
>> #define __ASM_ARM_KVM_VGIC_H
>>
>> +#include <linux/kernel.h>
>> +#include <linux/kvm.h>
>> +#include <linux/kvm_host.h>
>> +#include <linux/irqreturn.h>
>> +#include <linux/spinlock.h>
>> +#include <linux/types.h>
>> #include <asm/hardware/gic.h>
>>
>> +#define VGIC_NR_IRQS 128
>> +#define VGIC_NR_SGIS 16
>
> Now that you have this, you can use it in a few places (see also the BUG_ONs
> in vgic_queue_irq).
>
>> +#define VGIC_NR_PPIS 16
>> +#define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
>> +#define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
>> +#define VGIC_MAX_CPUS KVM_MAX_VCPUS
>> +
>> +/* Sanity checks... */
>> +#if (VGIC_MAX_CPUS > 8)
>> +#error Invalid number of CPU interfaces
>> +#endif
>> +
>> +#if (VGIC_NR_IRQS & 31)
>> +#error "VGIC_NR_IRQS must be a multiple of 32"
>> +#endif
>> +
>> +#if (VGIC_NR_IRQS > 1024)
>> +#error "VGIC_NR_IRQS must be <= 1024"
>> +#endif
>> +
>> +/*
>> + * The GIC distributor registers describing interrupts have two parts:
>> + * - 32 per-CPU interrupts (SGI + PPI)
>> + * - a bunch of shared interrupts (SPI)
>> + */
>> +struct vgic_bitmap {
>> + union {
>> + u32 reg[VGIC_NR_PRIVATE_IRQS / 32];
>> + DECLARE_BITMAP(reg_ul, VGIC_NR_PRIVATE_IRQS);
>> + } percpu[VGIC_MAX_CPUS];
>> + union {
>> + u32 reg[VGIC_NR_SHARED_IRQS / 32];
>> + DECLARE_BITMAP(reg_ul, VGIC_NR_SHARED_IRQS);
>> + } shared;
>> +};
>> +
>> +struct vgic_bytemap {
>> + u32 percpu[VGIC_MAX_CPUS][VGIC_NR_PRIVATE_IRQS / 4];
>> + u32 shared[VGIC_NR_SHARED_IRQS / 4];
>> +};
>> +
>> struct vgic_dist {
>> +#ifdef CONFIG_KVM_ARM_VGIC
>> + spinlock_t lock;
>> +
>> + /* Virtual control interface mapping */
>> + void __iomem *vctrl_base;
>> +
>> /* Distributor and vcpu interface mapping in the guest */
>> phys_addr_t vgic_dist_base;
>> phys_addr_t vgic_cpu_base;
>> +
>> + /* Distributor enabled */
>> + u32 enabled;
>> +
>> + /* Interrupt enabled (one bit per IRQ) */
>> + struct vgic_bitmap irq_enabled;
>> +
>> + /* Interrupt 'pin' level */
>> + struct vgic_bitmap irq_state;
>> +
>> + /* Level-triggered interrupt in progress */
>> + struct vgic_bitmap irq_active;
>> +
>> + /* Interrupt priority. Not used yet. */
>> + struct vgic_bytemap irq_priority;
>> +
>> + /* Level/edge triggered */
>> + struct vgic_bitmap irq_cfg;
>> +
>> + /* Source CPU per SGI and target CPU */
>> + u8 irq_sgi_sources[VGIC_MAX_CPUS][16];
>
> VGIC_NR_SGIS
>
>
>> +static u32 vgic_get_target_reg(struct kvm *kvm, int irq)
>> +{
>> + struct vgic_dist *dist = &kvm->arch.vgic;
>> + struct kvm_vcpu *vcpu;
>> + int i, c;
>> + unsigned long *bmap;
>> + u32 val = 0;
>> +
>> + irq -= VGIC_NR_PRIVATE_IRQS;
>> +
>> + kvm_for_each_vcpu(c, vcpu, kvm) {
>> + bmap = vgic_bitmap_get_shared_map(&dist->irq_spi_target[c]);
>> + for (i = 0; i < GICD_IRQS_PER_ITARGETSR; i++)
>> + if (test_bit(irq + i, bmap))
>> + val |= 1 << (c + i * 8);
>> + }
>> +
>> + return val;
>> +}
>> +
>> +static void vgic_set_target_reg(struct kvm *kvm, u32 val, int irq)
>> +{
>> + struct vgic_dist *dist = &kvm->arch.vgic;
>> + struct kvm_vcpu *vcpu;
>> + int i, c;
>> + unsigned long *bmap;
>> + u32 target;
>> +
>> + BUG_ON(irq & 3);
>> + BUG_ON(irq < VGIC_NR_PRIVATE_IRQS);
>
> This is now different to vgic_Get_target_reg, which doesn't have the
> BUG_ONs. Can we remove these ones too?
>
>> + irq -= VGIC_NR_PRIVATE_IRQS;
>> +
>> + /*
>> + * Pick the LSB in each byte. This ensures we target exactly
>> + * one vcpu per IRQ. If the byte is null, assume we target
>> + * CPU0.
>> + */
>> + for (i = 0; i < GICD_IRQS_PER_ITARGETSR; i++) {
>> + int shift = i * GICD_CPUTARGETS_BITS;
>> + target = ffs((val >> shift) & 0xffU);
>> + target = target ? (target - 1) : 0;
>> + dist->irq_spi_cpu[irq + i] = target;
>> + kvm_for_each_vcpu(c, vcpu, kvm) {
>> + bmap = vgic_bitmap_get_shared_map(&dist->irq_spi_target[c]);
>> + if (c == target)
>> + set_bit(irq + i, bmap);
>> + else
>> + clear_bit(irq + i, bmap);
>> + }
>> + }
>> +}
>
> [...]
>
>> static const struct mmio_range vgic_ranges[] = {
>> + { /* CTRL, TYPER, IIDR */
>> + .base = 0,
>> + .len = 12,
>> + .handle_mmio = handle_mmio_misc,
>> + },
>> + { /* IGROUPRn */
>> + .base = 0x80,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_raz_wi,
>> + },
>> + { /* ISENABLERn */
>> + .base = 0x100,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_set_enable_reg,
>> + },
>> + { /* ICENABLERn */
>> + .base = 0x180,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_clear_enable_reg,
>> + },
>> + { /* ISPENDRn */
>> + .base = 0x200,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_set_pending_reg,
>> + },
>> + { /* ICPENDRn */
>> + .base = 0x280,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_clear_pending_reg,
>> + },
>> + { /* ISACTIVERn */
>> + .base = 0x300,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_raz_wi,
>> + },
>> + { /* ICACTIVERn */
>> + .base = 0x380,
>> + .len = VGIC_NR_IRQS / 8,
>> + .handle_mmio = handle_mmio_raz_wi,
>> + },
>> + { /* IPRIORITYRn */
>> + .base = 0x400,
>> + .len = VGIC_NR_IRQS,
>> + .handle_mmio = handle_mmio_priority_reg,
>> + },
>> + { /* ITARGETSRn */
>> + .base = 0x800,
>> + .len = VGIC_NR_IRQS,
>> + .handle_mmio = handle_mmio_target_reg,
>> + },
>> + { /* ICFGRn */
>> + .base = 0xC00,
>> + .len = VGIC_NR_IRQS / 4,
>> + .handle_mmio = handle_mmio_cfg_reg,
>> + },
>> + { /* SGIRn */
>> + .base = 0xF00,
>> + .len = 4,
>> + .handle_mmio = handle_mmio_sgi_reg,
>> + },
>> {}
>> };
>
> You've added named definitions for these constants to the GIC header file,
> so please replace these immediates with those and delete the comments.
>
The following two commits should address your concerns:
commit ff4648faa6fd3342ce72e537a8068ab21d4085c8
Author: Christoffer Dall <c.dall@virtualopensystems.com>
Date: Mon Jan 14 16:53:21 2013 -0500
KVM: ARM: vgic: Use defines instead of hardcoded numbers.
Address reviewer comments.
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index f5f270b..1ace491 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -99,7 +99,7 @@ struct vgic_dist {
struct vgic_bitmap irq_cfg;
/* Source CPU per SGI and target CPU */
- u8 irq_sgi_sources[VGIC_MAX_CPUS][16];
+ u8 irq_sgi_sources[VGIC_MAX_CPUS][VGIC_NR_SGIS];
/* Target CPU for each IRQ */
u8 irq_spi_cpu[VGIC_NR_SHARED_IRQS];
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 25daa07..a0d283c 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -447,9 +447,6 @@ static void vgic_set_target_reg(struct kvm *kvm,
u32 val, int irq)
unsigned long *bmap;
u32 target;
- BUG_ON(irq & 3);
- BUG_ON(irq < VGIC_NR_PRIVATE_IRQS);
-
irq -= VGIC_NR_PRIVATE_IRQS;
/*
@@ -598,63 +595,63 @@ struct mmio_range {
};
static const struct mmio_range vgic_ranges[] = {
- { /* CTRL, TYPER, IIDR */
- .base = 0,
+ {
+ .base = GIC_DIST_CTRL,
.len = 12,
.handle_mmio = handle_mmio_misc,
},
- { /* IGROUPRn */
- .base = 0x80,
+ {
+ .base = GIC_DIST_IGROUP,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_raz_wi,
},
- { /* ISENABLERn */
- .base = 0x100,
+ {
+ .base = GIC_DIST_ENABLE_SET,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_set_enable_reg,
},
- { /* ICENABLERn */
- .base = 0x180,
+ {
+ .base = GIC_DIST_ENABLE_CLEAR,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_clear_enable_reg,
},
- { /* ISPENDRn */
- .base = 0x200,
+ {
+ .base = GIC_DIST_PENDING_SET,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_set_pending_reg,
},
- { /* ICPENDRn */
- .base = 0x280,
+ {
+ .base = GIC_DIST_PENDING_CLEAR,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_clear_pending_reg,
},
- { /* ISACTIVERn */
- .base = 0x300,
+ {
+ .base = GIC_DIST_ACTIVE_SET,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_raz_wi,
},
- { /* ICACTIVERn */
- .base = 0x380,
+ {
+ .base = GIC_DIST_ACTIVE_CLEAR,
.len = VGIC_NR_IRQS / 8,
.handle_mmio = handle_mmio_raz_wi,
},
- { /* IPRIORITYRn */
- .base = 0x400,
+ {
+ .base = GIC_DIST_PRI,
.len = VGIC_NR_IRQS,
.handle_mmio = handle_mmio_priority_reg,
},
- { /* ITARGETSRn */
- .base = 0x800,
+ {
+ .base = GIC_DIST_TARGET,
.len = VGIC_NR_IRQS,
.handle_mmio = handle_mmio_target_reg,
},
- { /* ICFGRn */
- .base = 0xC00,
+ {
+ .base = GIC_DIST_CONFIG,
.len = VGIC_NR_IRQS / 4,
.handle_mmio = handle_mmio_cfg_reg,
},
- { /* SGIRn */
- .base = 0xF00,
+ {
+ .base = GIC_DIST_SOFTINT,
.len = 4,
.handle_mmio = handle_mmio_sgi_reg,
},
@@ -856,7 +853,7 @@ static bool vgic_queue_irq(struct kvm_vcpu *vcpu,
u8 sgi_source_id, int irq)
/* Sanitize the input... */
BUG_ON(sgi_source_id & ~7);
- BUG_ON(sgi_source_id && irq > 15);
+ BUG_ON(sgi_source_id && irq >= VGIC_NR_SGIS);
BUG_ON(irq >= VGIC_NR_IRQS);
kvm_debug("Queue IRQ%d\n", irq);
--
commit 940c2382e1d1cb6831d35ceeccb02c3d3f76a45c
Author: Christoffer Dall <c.dall@virtualopensystems.com>
Date: Mon Jan 14 16:51:30 2013 -0500
ARM: gic: add missing distributor defintions
Add missing register map offsets for the distributor and rename
GIC_DIST_ACTIVE_BIT to GIC_DIST_ACTIVE_SET to be consistent.
Cc: Marc Zyniger <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
diff --git a/arch/arm/include/asm/hardware/gic.h
b/arch/arm/include/asm/hardware/gic.h
index dd1add1..6cad421 100644
--- a/arch/arm/include/asm/hardware/gic.h
+++ b/arch/arm/include/asm/hardware/gic.h
@@ -22,11 +22,13 @@
#define GIC_DIST_CTRL 0x000
#define GIC_DIST_CTR 0x004
+#define GIC_DIST_IGROUP 0x080
#define GIC_DIST_ENABLE_SET 0x100
#define GIC_DIST_ENABLE_CLEAR 0x180
#define GIC_DIST_PENDING_SET 0x200
#define GIC_DIST_PENDING_CLEAR 0x280
-#define GIC_DIST_ACTIVE_BIT 0x300
+#define GIC_DIST_ACTIVE_SET 0x300
+#define GIC_DIST_ACTIVE_CLEAR 0x380
#define GIC_DIST_PRI 0x400
#define GIC_DIST_TARGET 0x800
#define GIC_DIST_CONFIG 0xc00
--
Thanks,
-Christoffer
^ permalink raw reply related [flat|nested] 79+ messages in thread
* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (5 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 06/12] ARM: KVM: VGIC distributor handling Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-14 15:42 ` Will Deacon
2013-01-08 18:42 ` [PATCH v5 08/12] ARM: KVM: vgic: retire queued, disabled interrupts Christoffer Dall
` (5 subsequent siblings)
12 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
Add VGIC virtual CPU interface code, picking pending interrupts
from the distributor and stashing them in the VGIC control interface
list registers.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/kvm_vgic.h | 30 ++++
arch/arm/kvm/vgic.c | 327 +++++++++++++++++++++++++++++++++++++++
2 files changed, 356 insertions(+), 1 deletion(-)
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index 9ff0d9c..b3133c4 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -110,8 +110,33 @@ struct vgic_dist {
};
struct vgic_cpu {
+#ifdef CONFIG_KVM_ARM_VGIC
+ /* per IRQ to LR mapping */
+ u8 vgic_irq_lr_map[VGIC_NR_IRQS];
+
+ /* Pending interrupts on this VCPU */
+ DECLARE_BITMAP( pending_percpu, VGIC_NR_PRIVATE_IRQS);
+ DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
+
+ /* Bitmap of used/free list registers */
+ DECLARE_BITMAP( lr_used, 64);
+
+ /* Number of list registers on this CPU */
+ int nr_lr;
+
+ /* CPU vif control registers for world switch */
+ u32 vgic_hcr;
+ u32 vgic_vmcr;
+ u32 vgic_misr; /* Saved only */
+ u32 vgic_eisr[2]; /* Saved only */
+ u32 vgic_elrsr[2]; /* Saved only */
+ u32 vgic_apr;
+ u32 vgic_lr[64]; /* A15 has only 4... */
+#endif
};
+#define LR_EMPTY 0xff
+
struct kvm;
struct kvm_vcpu;
struct kvm_run;
@@ -119,9 +144,14 @@ struct kvm_exit_mmio;
#ifdef CONFIG_KVM_ARM_VGIC
int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
+void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
+void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
+int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_exit_mmio *mmio);
+#define irqchip_in_kernel(k) (!!((k)->arch.vgic.vctrl_base))
+
#else
static inline int kvm_vgic_hyp_init(void)
{
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index bd2bd7f..58237d5 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -152,6 +152,34 @@ static int vgic_irq_is_enabled(struct kvm_vcpu *vcpu, int irq)
return vgic_bitmap_get_irq_val(&dist->irq_enabled, vcpu->vcpu_id, irq);
}
+static int vgic_irq_is_active(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ return vgic_bitmap_get_irq_val(&dist->irq_active, vcpu->vcpu_id, irq);
+}
+
+static void vgic_irq_set_active(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 1);
+}
+
+static void vgic_irq_clear_active(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 0);
+}
+
+static int vgic_dist_irq_is_pending(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ return vgic_bitmap_get_irq_val(&dist->irq_state, vcpu->vcpu_id, irq);
+}
+
static void vgic_dist_irq_set(struct kvm_vcpu *vcpu, int irq)
{
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
@@ -711,7 +739,30 @@ static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
{
- return 0;
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ unsigned long *pending, *enabled, *pend_percpu, *pend_shared;
+ unsigned long pending_private, pending_shared;
+ int vcpu_id;
+
+ vcpu_id = vcpu->vcpu_id;
+ pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
+ pend_shared = vcpu->arch.vgic_cpu.pending_shared;
+
+ pending = vgic_bitmap_get_cpu_map(&dist->irq_state, vcpu_id);
+ enabled = vgic_bitmap_get_cpu_map(&dist->irq_enabled, vcpu_id);
+ bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
+
+ pending = vgic_bitmap_get_shared_map(&dist->irq_state);
+ enabled = vgic_bitmap_get_shared_map(&dist->irq_enabled);
+ bitmap_and(pend_shared, pending, enabled, VGIC_NR_SHARED_IRQS);
+ bitmap_and(pend_shared, pend_shared,
+ vgic_bitmap_get_shared_map(&dist->irq_spi_target[vcpu_id]),
+ VGIC_NR_SHARED_IRQS);
+
+ pending_private = find_first_bit(pend_percpu, VGIC_NR_PRIVATE_IRQS);
+ pending_shared = find_first_bit(pend_shared, VGIC_NR_SHARED_IRQS);
+ return (pending_private < VGIC_NR_PRIVATE_IRQS ||
+ pending_shared < VGIC_NR_SHARED_IRQS);
}
/*
@@ -737,6 +788,280 @@ static void vgic_update_state(struct kvm *kvm)
}
}
+#define LR_CPUID(lr) \
+ (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
+#define MK_LR_PEND(src, irq) \
+ (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
+/*
+ * Queue an interrupt to a CPU virtual interface. Return true on success,
+ * or false if it wasn't possible to queue it.
+ */
+static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ int lr;
+
+ /* Sanitize the input... */
+ BUG_ON(sgi_source_id & ~7);
+ BUG_ON(sgi_source_id && irq > 15);
+ BUG_ON(irq >= VGIC_NR_IRQS);
+
+ kvm_debug("Queue IRQ%d\n", irq);
+
+ lr = vgic_cpu->vgic_irq_lr_map[irq];
+
+ /* Do we have an active interrupt for the same CPUID? */
+ if (lr != LR_EMPTY &&
+ (LR_CPUID(vgic_cpu->vgic_lr[lr]) == sgi_source_id)) {
+ kvm_debug("LR%d piggyback for IRQ%d %x\n",
+ lr, irq, vgic_cpu->vgic_lr[lr]);
+ BUG_ON(!test_bit(lr, vgic_cpu->lr_used));
+ vgic_cpu->vgic_lr[lr] |= GICH_LR_PENDING_BIT;
+
+ goto out;
+ }
+
+ /* Try to use another LR for this interrupt */
+ lr = find_first_zero_bit((unsigned long *)vgic_cpu->lr_used,
+ vgic_cpu->nr_lr);
+ if (lr >= vgic_cpu->nr_lr)
+ return false;
+
+ kvm_debug("LR%d allocated for IRQ%d %x\n", lr, irq, sgi_source_id);
+ vgic_cpu->vgic_lr[lr] = MK_LR_PEND(sgi_source_id, irq);
+ vgic_cpu->vgic_irq_lr_map[irq] = lr;
+ set_bit(lr, vgic_cpu->lr_used);
+
+out:
+ if (!vgic_irq_is_edge(vcpu, irq))
+ vgic_cpu->vgic_lr[lr] |= GICH_LR_EOI;
+
+ return true;
+}
+
+static bool vgic_queue_sgi(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ unsigned long sources;
+ int vcpu_id = vcpu->vcpu_id;
+ int c;
+
+ sources = dist->irq_sgi_sources[vcpu_id][irq];
+
+ for_each_set_bit(c, &sources, VGIC_MAX_CPUS) {
+ if (vgic_queue_irq(vcpu, c, irq))
+ clear_bit(c, &sources);
+ }
+
+ dist->irq_sgi_sources[vcpu_id][irq] = sources;
+
+ /*
+ * If the sources bitmap has been cleared it means that we
+ * could queue all the SGIs onto link registers (see the
+ * clear_bit above), and therefore we are done with them in
+ * our emulated gic and can get rid of them.
+ */
+ if (!sources) {
+ vgic_dist_irq_clear(vcpu, irq);
+ vgic_cpu_irq_clear(vcpu, irq);
+ return true;
+ }
+
+ return false;
+}
+
+static bool vgic_queue_hwirq(struct kvm_vcpu *vcpu, int irq)
+{
+ if (vgic_irq_is_active(vcpu, irq))
+ return true; /* level interrupt, already queued */
+
+ if (vgic_queue_irq(vcpu, 0, irq)) {
+ if (vgic_irq_is_edge(vcpu, irq)) {
+ vgic_dist_irq_clear(vcpu, irq);
+ vgic_cpu_irq_clear(vcpu, irq);
+ } else {
+ vgic_irq_set_active(vcpu, irq);
+ }
+
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Fill the list registers with pending interrupts before running the
+ * guest.
+ */
+static void __kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int i, vcpu_id;
+ int overflow = 0;
+
+ vcpu_id = vcpu->vcpu_id;
+
+ /*
+ * We may not have any pending interrupt, or the interrupts
+ * may have been serviced from another vcpu. In all cases,
+ * move along.
+ */
+ if (!kvm_vgic_vcpu_pending_irq(vcpu)) {
+ pr_debug("CPU%d has no pending interrupt\n", vcpu_id);
+ goto epilog;
+ }
+
+ /* SGIs */
+ for_each_set_bit(i, vgic_cpu->pending_percpu, VGIC_NR_SGIS) {
+ if (!vgic_queue_sgi(vcpu, i))
+ overflow = 1;
+ }
+
+ /* PPIs */
+ for_each_set_bit_from(i, vgic_cpu->pending_percpu, VGIC_NR_PRIVATE_IRQS) {
+ if (!vgic_queue_hwirq(vcpu, i))
+ overflow = 1;
+ }
+
+ /* SPIs */
+ for_each_set_bit(i, vgic_cpu->pending_shared, VGIC_NR_SHARED_IRQS) {
+ if (!vgic_queue_hwirq(vcpu, i + VGIC_NR_PRIVATE_IRQS))
+ overflow = 1;
+ }
+
+epilog:
+ if (overflow) {
+ vgic_cpu->vgic_hcr |= GICH_HCR_UIE;
+ } else {
+ vgic_cpu->vgic_hcr &= ~GICH_HCR_UIE;
+ /*
+ * We're about to run this VCPU, and we've consumed
+ * everything the distributor had in store for
+ * us. Claim we don't have anything pending. We'll
+ * adjust that if needed while exiting.
+ */
+ clear_bit(vcpu_id, &dist->irq_pending_on_cpu);
+ }
+}
+
+static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ bool level_pending = false;
+
+ kvm_debug("MISR = %08x\n", vgic_cpu->vgic_misr);
+
+ /*
+ * We do not need to take the distributor lock here, since the only
+ * action we perform is clearing the irq_active_bit for an EOIed
+ * level interrupt. There is a potential race with
+ * the queuing of an interrupt in __kvm_sync_to_cpu(), where we check
+ * if the interrupt is already active. Two possibilities:
+ *
+ * - The queuing is occurring on the same vcpu: cannot happen,
+ * as we're already in the context of this vcpu, and
+ * executing the handler
+ * - The interrupt has been migrated to another vcpu, and we
+ * ignore this interrupt for this run. Big deal. It is still
+ * pending though, and will get considered when this vcpu
+ * exits.
+ */
+ if (vgic_cpu->vgic_misr & GICH_MISR_EOI) {
+ /*
+ * Some level interrupts have been EOIed. Clear their
+ * active bit.
+ */
+ int lr, irq;
+
+ for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_eisr,
+ vgic_cpu->nr_lr) {
+ irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
+
+ vgic_irq_clear_active(vcpu, irq);
+ vgic_cpu->vgic_lr[lr] &= ~GICH_LR_EOI;
+
+ /* Any additional pending interrupt? */
+ if (vgic_dist_irq_is_pending(vcpu, irq)) {
+ vgic_cpu_irq_set(vcpu, irq);
+ level_pending = true;
+ } else {
+ vgic_cpu_irq_clear(vcpu, irq);
+ }
+ }
+ }
+
+ if (vgic_cpu->vgic_misr & GICH_MISR_U)
+ vgic_cpu->vgic_hcr &= ~GICH_HCR_UIE;
+
+ return level_pending;
+}
+
+/*
+ * Sync back the VGIC state after a guest run. We do not really touch
+ * the distributor here (the irq_pending_on_cpu bit is safe to set),
+ * so there is no need for taking its lock.
+ */
+static void __kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int lr, pending;
+ bool level_pending;
+
+ level_pending = vgic_process_maintenance(vcpu);
+
+ /* Clear mappings for empty LRs */
+ for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_elrsr,
+ vgic_cpu->nr_lr) {
+ int irq;
+
+ if (!test_and_clear_bit(lr, vgic_cpu->lr_used))
+ continue;
+
+ irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
+
+ BUG_ON(irq >= VGIC_NR_IRQS);
+ vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
+ }
+
+ /* Check if we still have something up our sleeve... */
+ pending = find_first_zero_bit((unsigned long *)vgic_cpu->vgic_elrsr,
+ vgic_cpu->nr_lr);
+ if (level_pending || pending < vgic_cpu->nr_lr)
+ set_bit(vcpu->vcpu_id, &dist->irq_pending_on_cpu);
+}
+
+void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return;
+
+ spin_lock(&dist->lock);
+ __kvm_vgic_sync_to_cpu(vcpu);
+ spin_unlock(&dist->lock);
+}
+
+void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu)
+{
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return;
+
+ __kvm_vgic_sync_from_cpu(vcpu);
+}
+
+int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return 0;
+
+ return test_bit(vcpu->vcpu_id, &dist->irq_pending_on_cpu);
+}
+
static bool vgic_ioaddr_overlap(struct kvm *kvm)
{
phys_addr_t dist = kvm->arch.vgic.vgic_dist_base;
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-08 18:42 ` [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management Christoffer Dall
@ 2013-01-14 15:42 ` Will Deacon
2013-01-14 22:02 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Will Deacon @ 2013-01-14 15:42 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Jan 08, 2013 at 06:42:11PM +0000, Christoffer Dall wrote:
> From: Marc Zyngier <marc.zyngier@arm.com>
>
> Add VGIC virtual CPU interface code, picking pending interrupts
> from the distributor and stashing them in the VGIC control interface
> list registers.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
> ---
> arch/arm/include/asm/kvm_vgic.h | 30 ++++
> arch/arm/kvm/vgic.c | 327 +++++++++++++++++++++++++++++++++++++++
> 2 files changed, 356 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
> index 9ff0d9c..b3133c4 100644
> --- a/arch/arm/include/asm/kvm_vgic.h
> +++ b/arch/arm/include/asm/kvm_vgic.h
> @@ -110,8 +110,33 @@ struct vgic_dist {
> };
>
> struct vgic_cpu {
> +#ifdef CONFIG_KVM_ARM_VGIC
> + /* per IRQ to LR mapping */
> + u8 vgic_irq_lr_map[VGIC_NR_IRQS];
> +
> + /* Pending interrupts on this VCPU */
> + DECLARE_BITMAP( pending_percpu, VGIC_NR_PRIVATE_IRQS);
> + DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
> +
> + /* Bitmap of used/free list registers */
> + DECLARE_BITMAP( lr_used, 64);
> +
> + /* Number of list registers on this CPU */
> + int nr_lr;
> +
> + /* CPU vif control registers for world switch */
> + u32 vgic_hcr;
> + u32 vgic_vmcr;
> + u32 vgic_misr; /* Saved only */
> + u32 vgic_eisr[2]; /* Saved only */
> + u32 vgic_elrsr[2]; /* Saved only */
> + u32 vgic_apr;
> + u32 vgic_lr[64]; /* A15 has only 4... */
Have a #define for the maximum number of list registers.
> +#endif
> };
>
> +#define LR_EMPTY 0xff
> +
> struct kvm;
> struct kvm_vcpu;
> struct kvm_run;
> @@ -119,9 +144,14 @@ struct kvm_exit_mmio;
>
> #ifdef CONFIG_KVM_ARM_VGIC
> int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
> +void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
> +void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
Same comment as for the arch timer (flush/sync).
> +int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
> bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> struct kvm_exit_mmio *mmio);
>
> +#define irqchip_in_kernel(k) (!!((k)->arch.vgic.vctrl_base))
> +
> #else
> static inline int kvm_vgic_hyp_init(void)
> {
> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
> index bd2bd7f..58237d5 100644
> --- a/arch/arm/kvm/vgic.c
> +++ b/arch/arm/kvm/vgic.c
> @@ -152,6 +152,34 @@ static int vgic_irq_is_enabled(struct kvm_vcpu *vcpu, int irq)
> return vgic_bitmap_get_irq_val(&dist->irq_enabled, vcpu->vcpu_id, irq);
> }
>
> +static int vgic_irq_is_active(struct kvm_vcpu *vcpu, int irq)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> +
> + return vgic_bitmap_get_irq_val(&dist->irq_active, vcpu->vcpu_id, irq);
> +}
> +
> +static void vgic_irq_set_active(struct kvm_vcpu *vcpu, int irq)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> +
> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 1);
> +}
> +
> +static void vgic_irq_clear_active(struct kvm_vcpu *vcpu, int irq)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> +
> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 0);
> +}
> +
> +static int vgic_dist_irq_is_pending(struct kvm_vcpu *vcpu, int irq)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> +
> + return vgic_bitmap_get_irq_val(&dist->irq_state, vcpu->vcpu_id, irq);
> +}
> +
> static void vgic_dist_irq_set(struct kvm_vcpu *vcpu, int irq)
> {
> struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> @@ -711,7 +739,30 @@ static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
>
> static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
> {
> - return 0;
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + unsigned long *pending, *enabled, *pend_percpu, *pend_shared;
> + unsigned long pending_private, pending_shared;
> + int vcpu_id;
> +
> + vcpu_id = vcpu->vcpu_id;
> + pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
> + pend_shared = vcpu->arch.vgic_cpu.pending_shared;
> +
> + pending = vgic_bitmap_get_cpu_map(&dist->irq_state, vcpu_id);
> + enabled = vgic_bitmap_get_cpu_map(&dist->irq_enabled, vcpu_id);
> + bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
> +
> + pending = vgic_bitmap_get_shared_map(&dist->irq_state);
> + enabled = vgic_bitmap_get_shared_map(&dist->irq_enabled);
> + bitmap_and(pend_shared, pending, enabled, VGIC_NR_SHARED_IRQS);
> + bitmap_and(pend_shared, pend_shared,
> + vgic_bitmap_get_shared_map(&dist->irq_spi_target[vcpu_id]),
> + VGIC_NR_SHARED_IRQS);
> +
> + pending_private = find_first_bit(pend_percpu, VGIC_NR_PRIVATE_IRQS);
> + pending_shared = find_first_bit(pend_shared, VGIC_NR_SHARED_IRQS);
> + return (pending_private < VGIC_NR_PRIVATE_IRQS ||
> + pending_shared < VGIC_NR_SHARED_IRQS);
> }
>
> /*
> @@ -737,6 +788,280 @@ static void vgic_update_state(struct kvm *kvm)
> }
> }
>
> +#define LR_CPUID(lr) \
> + (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> +#define MK_LR_PEND(src, irq) \
> + (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
> +/*
> + * Queue an interrupt to a CPU virtual interface. Return true on success,
> + * or false if it wasn't possible to queue it.
> + */
> +static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
> +{
> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> + int lr;
> +
> + /* Sanitize the input... */
> + BUG_ON(sgi_source_id & ~7);
> + BUG_ON(sgi_source_id && irq > 15);
You can use your new NR_SGIS definition here.
Will
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-14 15:42 ` Will Deacon
@ 2013-01-14 22:02 ` Christoffer Dall
2013-01-15 11:00 ` Marc Zyngier
0 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-14 22:02 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jan 14, 2013 at 10:42 AM, Will Deacon <will.deacon@arm.com> wrote:
> On Tue, Jan 08, 2013 at 06:42:11PM +0000, Christoffer Dall wrote:
>> From: Marc Zyngier <marc.zyngier@arm.com>
>>
>> Add VGIC virtual CPU interface code, picking pending interrupts
>> from the distributor and stashing them in the VGIC control interface
>> list registers.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>> ---
>> arch/arm/include/asm/kvm_vgic.h | 30 ++++
>> arch/arm/kvm/vgic.c | 327 +++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 356 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>> index 9ff0d9c..b3133c4 100644
>> --- a/arch/arm/include/asm/kvm_vgic.h
>> +++ b/arch/arm/include/asm/kvm_vgic.h
>> @@ -110,8 +110,33 @@ struct vgic_dist {
>> };
>>
>> struct vgic_cpu {
>> +#ifdef CONFIG_KVM_ARM_VGIC
>> + /* per IRQ to LR mapping */
>> + u8 vgic_irq_lr_map[VGIC_NR_IRQS];
>> +
>> + /* Pending interrupts on this VCPU */
>> + DECLARE_BITMAP( pending_percpu, VGIC_NR_PRIVATE_IRQS);
>> + DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>> +
>> + /* Bitmap of used/free list registers */
>> + DECLARE_BITMAP( lr_used, 64);
>> +
>> + /* Number of list registers on this CPU */
>> + int nr_lr;
>> +
>> + /* CPU vif control registers for world switch */
>> + u32 vgic_hcr;
>> + u32 vgic_vmcr;
>> + u32 vgic_misr; /* Saved only */
>> + u32 vgic_eisr[2]; /* Saved only */
>> + u32 vgic_elrsr[2]; /* Saved only */
>> + u32 vgic_apr;
>> + u32 vgic_lr[64]; /* A15 has only 4... */
>
> Have a #define for the maximum number of list registers.
>
>> +#endif
>> };
>>
>> +#define LR_EMPTY 0xff
>> +
>> struct kvm;
>> struct kvm_vcpu;
>> struct kvm_run;
>> @@ -119,9 +144,14 @@ struct kvm_exit_mmio;
>>
>> #ifdef CONFIG_KVM_ARM_VGIC
>> int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
>> +void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
>> +void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
>
> Same comment as for the arch timer (flush/sync).
>
>> +int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>> bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> struct kvm_exit_mmio *mmio);
>>
>> +#define irqchip_in_kernel(k) (!!((k)->arch.vgic.vctrl_base))
>> +
>> #else
>> static inline int kvm_vgic_hyp_init(void)
>> {
>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>> index bd2bd7f..58237d5 100644
>> --- a/arch/arm/kvm/vgic.c
>> +++ b/arch/arm/kvm/vgic.c
>> @@ -152,6 +152,34 @@ static int vgic_irq_is_enabled(struct kvm_vcpu *vcpu, int irq)
>> return vgic_bitmap_get_irq_val(&dist->irq_enabled, vcpu->vcpu_id, irq);
>> }
>>
>> +static int vgic_irq_is_active(struct kvm_vcpu *vcpu, int irq)
>> +{
>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> +
>> + return vgic_bitmap_get_irq_val(&dist->irq_active, vcpu->vcpu_id, irq);
>> +}
>> +
>> +static void vgic_irq_set_active(struct kvm_vcpu *vcpu, int irq)
>> +{
>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> +
>> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 1);
>> +}
>> +
>> +static void vgic_irq_clear_active(struct kvm_vcpu *vcpu, int irq)
>> +{
>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> +
>> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 0);
>> +}
>> +
>> +static int vgic_dist_irq_is_pending(struct kvm_vcpu *vcpu, int irq)
>> +{
>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> +
>> + return vgic_bitmap_get_irq_val(&dist->irq_state, vcpu->vcpu_id, irq);
>> +}
>> +
>> static void vgic_dist_irq_set(struct kvm_vcpu *vcpu, int irq)
>> {
>> struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> @@ -711,7 +739,30 @@ static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
>>
>> static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
>> {
>> - return 0;
>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> + unsigned long *pending, *enabled, *pend_percpu, *pend_shared;
>> + unsigned long pending_private, pending_shared;
>> + int vcpu_id;
>> +
>> + vcpu_id = vcpu->vcpu_id;
>> + pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
>> + pend_shared = vcpu->arch.vgic_cpu.pending_shared;
>> +
>> + pending = vgic_bitmap_get_cpu_map(&dist->irq_state, vcpu_id);
>> + enabled = vgic_bitmap_get_cpu_map(&dist->irq_enabled, vcpu_id);
>> + bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
>> +
>> + pending = vgic_bitmap_get_shared_map(&dist->irq_state);
>> + enabled = vgic_bitmap_get_shared_map(&dist->irq_enabled);
>> + bitmap_and(pend_shared, pending, enabled, VGIC_NR_SHARED_IRQS);
>> + bitmap_and(pend_shared, pend_shared,
>> + vgic_bitmap_get_shared_map(&dist->irq_spi_target[vcpu_id]),
>> + VGIC_NR_SHARED_IRQS);
>> +
>> + pending_private = find_first_bit(pend_percpu, VGIC_NR_PRIVATE_IRQS);
>> + pending_shared = find_first_bit(pend_shared, VGIC_NR_SHARED_IRQS);
>> + return (pending_private < VGIC_NR_PRIVATE_IRQS ||
>> + pending_shared < VGIC_NR_SHARED_IRQS);
>> }
>>
>> /*
>> @@ -737,6 +788,280 @@ static void vgic_update_state(struct kvm *kvm)
>> }
>> }
>>
>> +#define LR_CPUID(lr) \
>> + (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
>> +#define MK_LR_PEND(src, irq) \
>> + (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
>> +/*
>> + * Queue an interrupt to a CPU virtual interface. Return true on success,
>> + * or false if it wasn't possible to queue it.
>> + */
>> +static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
>> +{
>> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>> + int lr;
>> +
>> + /* Sanitize the input... */
>> + BUG_ON(sgi_source_id & ~7);
>> + BUG_ON(sgi_source_id && irq > 15);
>
> You can use your new NR_SGIS definition here.
>
This should address the remaining comments:
commit 43957095ec5476beb198f4c4630dfc3e2f3951db
Author: Christoffer Dall <c.dall@virtualopensystems.com>
Date: Mon Jan 14 16:59:38 2013 -0500
KVM: ARM: vgic: Define VGIC_MAX_LRS
Define maximum number of link registers we can handle instead of using
literals in the code. If an architecture reports more link registers
than we support, only use the number we can support.
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index 1ace491..f9d1977 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -33,6 +33,7 @@
#define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
#define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
#define VGIC_MAX_CPUS KVM_MAX_VCPUS
+#define VGIC_MAX_LRS 64
/* Sanity checks... */
#if (VGIC_MAX_CPUS > 8)
@@ -120,7 +121,7 @@ struct vgic_cpu {
DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
/* Bitmap of used/free list registers */
- DECLARE_BITMAP( lr_used, 64);
+ DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
/* Number of list registers on this CPU */
int nr_lr;
@@ -132,7 +133,7 @@ struct vgic_cpu {
u32 vgic_eisr[2]; /* Saved only */
u32 vgic_elrsr[2]; /* Saved only */
u32 vgic_apr;
- u32 vgic_lr[64]; /* A15 has only 4... */
+ u32 vgic_lr[VGIC_MAX_LRS];
#endif
};
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index a0d283c..90a99fd 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
+ if (vgic_nr_lr > VGIC_MAX_LRS)
+ vgic_nr_lr = VGIC_MAX_LRS; /* TODO: Clear remaining LRs */
ret = create_hyp_io_mappings(vgic_vctrl_base,
vgic_vctrl_base + resource_size(&vctrl_res),
--
Thanks,
-Christoffer
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-14 22:02 ` Christoffer Dall
@ 2013-01-15 11:00 ` Marc Zyngier
2013-01-15 14:31 ` Christoffer Dall
2013-01-16 15:29 ` Christoffer Dall
0 siblings, 2 replies; 79+ messages in thread
From: Marc Zyngier @ 2013-01-15 11:00 UTC (permalink / raw)
To: linux-arm-kernel
On 14/01/13 22:02, Christoffer Dall wrote:
> On Mon, Jan 14, 2013 at 10:42 AM, Will Deacon <will.deacon@arm.com> wrote:
>> On Tue, Jan 08, 2013 at 06:42:11PM +0000, Christoffer Dall wrote:
>>> From: Marc Zyngier <marc.zyngier@arm.com>
>>>
>>> Add VGIC virtual CPU interface code, picking pending interrupts
>>> from the distributor and stashing them in the VGIC control interface
>>> list registers.
>>>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>>> ---
>>> arch/arm/include/asm/kvm_vgic.h | 30 ++++
>>> arch/arm/kvm/vgic.c | 327 +++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 356 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>>> index 9ff0d9c..b3133c4 100644
>>> --- a/arch/arm/include/asm/kvm_vgic.h
>>> +++ b/arch/arm/include/asm/kvm_vgic.h
>>> @@ -110,8 +110,33 @@ struct vgic_dist {
>>> };
>>>
>>> struct vgic_cpu {
>>> +#ifdef CONFIG_KVM_ARM_VGIC
>>> + /* per IRQ to LR mapping */
>>> + u8 vgic_irq_lr_map[VGIC_NR_IRQS];
>>> +
>>> + /* Pending interrupts on this VCPU */
>>> + DECLARE_BITMAP( pending_percpu, VGIC_NR_PRIVATE_IRQS);
>>> + DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>> +
>>> + /* Bitmap of used/free list registers */
>>> + DECLARE_BITMAP( lr_used, 64);
>>> +
>>> + /* Number of list registers on this CPU */
>>> + int nr_lr;
>>> +
>>> + /* CPU vif control registers for world switch */
>>> + u32 vgic_hcr;
>>> + u32 vgic_vmcr;
>>> + u32 vgic_misr; /* Saved only */
>>> + u32 vgic_eisr[2]; /* Saved only */
>>> + u32 vgic_elrsr[2]; /* Saved only */
>>> + u32 vgic_apr;
>>> + u32 vgic_lr[64]; /* A15 has only 4... */
>>
>> Have a #define for the maximum number of list registers.
>>
>>> +#endif
>>> };
>>>
>>> +#define LR_EMPTY 0xff
>>> +
>>> struct kvm;
>>> struct kvm_vcpu;
>>> struct kvm_run;
>>> @@ -119,9 +144,14 @@ struct kvm_exit_mmio;
>>>
>>> #ifdef CONFIG_KVM_ARM_VGIC
>>> int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
>>> +void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
>>> +void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
>>
>> Same comment as for the arch timer (flush/sync).
>>
>>> +int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>>> bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>> struct kvm_exit_mmio *mmio);
>>>
>>> +#define irqchip_in_kernel(k) (!!((k)->arch.vgic.vctrl_base))
>>> +
>>> #else
>>> static inline int kvm_vgic_hyp_init(void)
>>> {
>>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>>> index bd2bd7f..58237d5 100644
>>> --- a/arch/arm/kvm/vgic.c
>>> +++ b/arch/arm/kvm/vgic.c
>>> @@ -152,6 +152,34 @@ static int vgic_irq_is_enabled(struct kvm_vcpu *vcpu, int irq)
>>> return vgic_bitmap_get_irq_val(&dist->irq_enabled, vcpu->vcpu_id, irq);
>>> }
>>>
>>> +static int vgic_irq_is_active(struct kvm_vcpu *vcpu, int irq)
>>> +{
>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> +
>>> + return vgic_bitmap_get_irq_val(&dist->irq_active, vcpu->vcpu_id, irq);
>>> +}
>>> +
>>> +static void vgic_irq_set_active(struct kvm_vcpu *vcpu, int irq)
>>> +{
>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> +
>>> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 1);
>>> +}
>>> +
>>> +static void vgic_irq_clear_active(struct kvm_vcpu *vcpu, int irq)
>>> +{
>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> +
>>> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 0);
>>> +}
>>> +
>>> +static int vgic_dist_irq_is_pending(struct kvm_vcpu *vcpu, int irq)
>>> +{
>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> +
>>> + return vgic_bitmap_get_irq_val(&dist->irq_state, vcpu->vcpu_id, irq);
>>> +}
>>> +
>>> static void vgic_dist_irq_set(struct kvm_vcpu *vcpu, int irq)
>>> {
>>> struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> @@ -711,7 +739,30 @@ static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
>>>
>>> static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
>>> {
>>> - return 0;
>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> + unsigned long *pending, *enabled, *pend_percpu, *pend_shared;
>>> + unsigned long pending_private, pending_shared;
>>> + int vcpu_id;
>>> +
>>> + vcpu_id = vcpu->vcpu_id;
>>> + pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
>>> + pend_shared = vcpu->arch.vgic_cpu.pending_shared;
>>> +
>>> + pending = vgic_bitmap_get_cpu_map(&dist->irq_state, vcpu_id);
>>> + enabled = vgic_bitmap_get_cpu_map(&dist->irq_enabled, vcpu_id);
>>> + bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
>>> +
>>> + pending = vgic_bitmap_get_shared_map(&dist->irq_state);
>>> + enabled = vgic_bitmap_get_shared_map(&dist->irq_enabled);
>>> + bitmap_and(pend_shared, pending, enabled, VGIC_NR_SHARED_IRQS);
>>> + bitmap_and(pend_shared, pend_shared,
>>> + vgic_bitmap_get_shared_map(&dist->irq_spi_target[vcpu_id]),
>>> + VGIC_NR_SHARED_IRQS);
>>> +
>>> + pending_private = find_first_bit(pend_percpu, VGIC_NR_PRIVATE_IRQS);
>>> + pending_shared = find_first_bit(pend_shared, VGIC_NR_SHARED_IRQS);
>>> + return (pending_private < VGIC_NR_PRIVATE_IRQS ||
>>> + pending_shared < VGIC_NR_SHARED_IRQS);
>>> }
>>>
>>> /*
>>> @@ -737,6 +788,280 @@ static void vgic_update_state(struct kvm *kvm)
>>> }
>>> }
>>>
>>> +#define LR_CPUID(lr) \
>>> + (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
>>> +#define MK_LR_PEND(src, irq) \
>>> + (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
>>> +/*
>>> + * Queue an interrupt to a CPU virtual interface. Return true on success,
>>> + * or false if it wasn't possible to queue it.
>>> + */
>>> +static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
>>> +{
>>> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>>> + int lr;
>>> +
>>> + /* Sanitize the input... */
>>> + BUG_ON(sgi_source_id & ~7);
>>> + BUG_ON(sgi_source_id && irq > 15);
>>
>> You can use your new NR_SGIS definition here.
>>
>
> This should address the remaining comments:
>
> commit 43957095ec5476beb198f4c4630dfc3e2f3951db
> Author: Christoffer Dall <c.dall@virtualopensystems.com>
> Date: Mon Jan 14 16:59:38 2013 -0500
>
> KVM: ARM: vgic: Define VGIC_MAX_LRS
>
> Define maximum number of link registers we can handle instead of using
> literals in the code. If an architecture reports more link registers
> than we support, only use the number we can support.
>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>
> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
> index 1ace491..f9d1977 100644
> --- a/arch/arm/include/asm/kvm_vgic.h
> +++ b/arch/arm/include/asm/kvm_vgic.h
> @@ -33,6 +33,7 @@
> #define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
> #define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
> #define VGIC_MAX_CPUS KVM_MAX_VCPUS
> +#define VGIC_MAX_LRS 64
Consider this instead (for the reason below)
#define VGIC_MAX_LRS (1 << 7)
> /* Sanity checks... */
> #if (VGIC_MAX_CPUS > 8)
> @@ -120,7 +121,7 @@ struct vgic_cpu {
> DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>
> /* Bitmap of used/free list registers */
> - DECLARE_BITMAP( lr_used, 64);
> + DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
>
> /* Number of list registers on this CPU */
> int nr_lr;
> @@ -132,7 +133,7 @@ struct vgic_cpu {
> u32 vgic_eisr[2]; /* Saved only */
> u32 vgic_elrsr[2]; /* Saved only */
> u32 vgic_apr;
> - u32 vgic_lr[64]; /* A15 has only 4... */
> + u32 vgic_lr[VGIC_MAX_LRS];
> #endif
> };
>
> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
> index a0d283c..90a99fd 100644
> --- a/arch/arm/kvm/vgic.c
> +++ b/arch/arm/kvm/vgic.c
> @@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
>
> vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
> vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
There is a bug here. It should be:
vgic_nr_lr = (vgic_nr_lr & 0x2f) + 1;
> + if (vgic_nr_lr > VGIC_MAX_LRS)
> + vgic_nr_lr = VGIC_MAX_LRS; /* TODO: Clear remaining LRs */
Why? VGIC_MAX_LRS isn't a configurable value, but a maximum value
defined by the specification. This is the maximum you can fit in a 6 bit
field, plus one (1 << 7, exactly).
> ret = create_hyp_io_mappings(vgic_vctrl_base,
> vgic_vctrl_base + resource_size(&vctrl_res),
> --
>
> Thanks,
> -Christoffer
>
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-15 11:00 ` Marc Zyngier
@ 2013-01-15 14:31 ` Christoffer Dall
2013-01-16 15:29 ` Christoffer Dall
1 sibling, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-15 14:31 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Jan 15, 2013 at 6:00 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 14/01/13 22:02, Christoffer Dall wrote:
>> On Mon, Jan 14, 2013 at 10:42 AM, Will Deacon <will.deacon@arm.com> wrote:
>>> On Tue, Jan 08, 2013 at 06:42:11PM +0000, Christoffer Dall wrote:
>>>> From: Marc Zyngier <marc.zyngier@arm.com>
>>>>
>>>> Add VGIC virtual CPU interface code, picking pending interrupts
>>>> from the distributor and stashing them in the VGIC control interface
>>>> list registers.
>>>>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>>>> ---
>>>> arch/arm/include/asm/kvm_vgic.h | 30 ++++
>>>> arch/arm/kvm/vgic.c | 327 +++++++++++++++++++++++++++++++++++++++
>>>> 2 files changed, 356 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>>>> index 9ff0d9c..b3133c4 100644
>>>> --- a/arch/arm/include/asm/kvm_vgic.h
>>>> +++ b/arch/arm/include/asm/kvm_vgic.h
>>>> @@ -110,8 +110,33 @@ struct vgic_dist {
>>>> };
>>>>
>>>> struct vgic_cpu {
>>>> +#ifdef CONFIG_KVM_ARM_VGIC
>>>> + /* per IRQ to LR mapping */
>>>> + u8 vgic_irq_lr_map[VGIC_NR_IRQS];
>>>> +
>>>> + /* Pending interrupts on this VCPU */
>>>> + DECLARE_BITMAP( pending_percpu, VGIC_NR_PRIVATE_IRQS);
>>>> + DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>>> +
>>>> + /* Bitmap of used/free list registers */
>>>> + DECLARE_BITMAP( lr_used, 64);
>>>> +
>>>> + /* Number of list registers on this CPU */
>>>> + int nr_lr;
>>>> +
>>>> + /* CPU vif control registers for world switch */
>>>> + u32 vgic_hcr;
>>>> + u32 vgic_vmcr;
>>>> + u32 vgic_misr; /* Saved only */
>>>> + u32 vgic_eisr[2]; /* Saved only */
>>>> + u32 vgic_elrsr[2]; /* Saved only */
>>>> + u32 vgic_apr;
>>>> + u32 vgic_lr[64]; /* A15 has only 4... */
>>>
>>> Have a #define for the maximum number of list registers.
>>>
>>>> +#endif
>>>> };
>>>>
>>>> +#define LR_EMPTY 0xff
>>>> +
>>>> struct kvm;
>>>> struct kvm_vcpu;
>>>> struct kvm_run;
>>>> @@ -119,9 +144,14 @@ struct kvm_exit_mmio;
>>>>
>>>> #ifdef CONFIG_KVM_ARM_VGIC
>>>> int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
>>>> +void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
>>>> +void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
>>>
>>> Same comment as for the arch timer (flush/sync).
>>>
>>>> +int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>>>> bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>>> struct kvm_exit_mmio *mmio);
>>>>
>>>> +#define irqchip_in_kernel(k) (!!((k)->arch.vgic.vctrl_base))
>>>> +
>>>> #else
>>>> static inline int kvm_vgic_hyp_init(void)
>>>> {
>>>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>>>> index bd2bd7f..58237d5 100644
>>>> --- a/arch/arm/kvm/vgic.c
>>>> +++ b/arch/arm/kvm/vgic.c
>>>> @@ -152,6 +152,34 @@ static int vgic_irq_is_enabled(struct kvm_vcpu *vcpu, int irq)
>>>> return vgic_bitmap_get_irq_val(&dist->irq_enabled, vcpu->vcpu_id, irq);
>>>> }
>>>>
>>>> +static int vgic_irq_is_active(struct kvm_vcpu *vcpu, int irq)
>>>> +{
>>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>>> +
>>>> + return vgic_bitmap_get_irq_val(&dist->irq_active, vcpu->vcpu_id, irq);
>>>> +}
>>>> +
>>>> +static void vgic_irq_set_active(struct kvm_vcpu *vcpu, int irq)
>>>> +{
>>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>>> +
>>>> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 1);
>>>> +}
>>>> +
>>>> +static void vgic_irq_clear_active(struct kvm_vcpu *vcpu, int irq)
>>>> +{
>>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>>> +
>>>> + vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 0);
>>>> +}
>>>> +
>>>> +static int vgic_dist_irq_is_pending(struct kvm_vcpu *vcpu, int irq)
>>>> +{
>>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>>> +
>>>> + return vgic_bitmap_get_irq_val(&dist->irq_state, vcpu->vcpu_id, irq);
>>>> +}
>>>> +
>>>> static void vgic_dist_irq_set(struct kvm_vcpu *vcpu, int irq)
>>>> {
>>>> struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>>> @@ -711,7 +739,30 @@ static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
>>>>
>>>> static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
>>>> {
>>>> - return 0;
>>>> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>>> + unsigned long *pending, *enabled, *pend_percpu, *pend_shared;
>>>> + unsigned long pending_private, pending_shared;
>>>> + int vcpu_id;
>>>> +
>>>> + vcpu_id = vcpu->vcpu_id;
>>>> + pend_percpu = vcpu->arch.vgic_cpu.pending_percpu;
>>>> + pend_shared = vcpu->arch.vgic_cpu.pending_shared;
>>>> +
>>>> + pending = vgic_bitmap_get_cpu_map(&dist->irq_state, vcpu_id);
>>>> + enabled = vgic_bitmap_get_cpu_map(&dist->irq_enabled, vcpu_id);
>>>> + bitmap_and(pend_percpu, pending, enabled, VGIC_NR_PRIVATE_IRQS);
>>>> +
>>>> + pending = vgic_bitmap_get_shared_map(&dist->irq_state);
>>>> + enabled = vgic_bitmap_get_shared_map(&dist->irq_enabled);
>>>> + bitmap_and(pend_shared, pending, enabled, VGIC_NR_SHARED_IRQS);
>>>> + bitmap_and(pend_shared, pend_shared,
>>>> + vgic_bitmap_get_shared_map(&dist->irq_spi_target[vcpu_id]),
>>>> + VGIC_NR_SHARED_IRQS);
>>>> +
>>>> + pending_private = find_first_bit(pend_percpu, VGIC_NR_PRIVATE_IRQS);
>>>> + pending_shared = find_first_bit(pend_shared, VGIC_NR_SHARED_IRQS);
>>>> + return (pending_private < VGIC_NR_PRIVATE_IRQS ||
>>>> + pending_shared < VGIC_NR_SHARED_IRQS);
>>>> }
>>>>
>>>> /*
>>>> @@ -737,6 +788,280 @@ static void vgic_update_state(struct kvm *kvm)
>>>> }
>>>> }
>>>>
>>>> +#define LR_CPUID(lr) \
>>>> + (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
>>>> +#define MK_LR_PEND(src, irq) \
>>>> + (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
>>>> +/*
>>>> + * Queue an interrupt to a CPU virtual interface. Return true on success,
>>>> + * or false if it wasn't possible to queue it.
>>>> + */
>>>> +static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
>>>> +{
>>>> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
>>>> + int lr;
>>>> +
>>>> + /* Sanitize the input... */
>>>> + BUG_ON(sgi_source_id & ~7);
>>>> + BUG_ON(sgi_source_id && irq > 15);
>>>
>>> You can use your new NR_SGIS definition here.
>>>
>>
>> This should address the remaining comments:
>>
>> commit 43957095ec5476beb198f4c4630dfc3e2f3951db
>> Author: Christoffer Dall <c.dall@virtualopensystems.com>
>> Date: Mon Jan 14 16:59:38 2013 -0500
>>
>> KVM: ARM: vgic: Define VGIC_MAX_LRS
>>
>> Define maximum number of link registers we can handle instead of using
>> literals in the code. If an architecture reports more link registers
>> than we support, only use the number we can support.
>>
>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>>
>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>> index 1ace491..f9d1977 100644
>> --- a/arch/arm/include/asm/kvm_vgic.h
>> +++ b/arch/arm/include/asm/kvm_vgic.h
>> @@ -33,6 +33,7 @@
>> #define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
>> #define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
>> #define VGIC_MAX_CPUS KVM_MAX_VCPUS
>> +#define VGIC_MAX_LRS 64
>
> Consider this instead (for the reason below)
> #define VGIC_MAX_LRS (1 << 7)
>
>> /* Sanity checks... */
>> #if (VGIC_MAX_CPUS > 8)
>> @@ -120,7 +121,7 @@ struct vgic_cpu {
>> DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>
>> /* Bitmap of used/free list registers */
>> - DECLARE_BITMAP( lr_used, 64);
>> + DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
>>
>> /* Number of list registers on this CPU */
>> int nr_lr;
>> @@ -132,7 +133,7 @@ struct vgic_cpu {
>> u32 vgic_eisr[2]; /* Saved only */
>> u32 vgic_elrsr[2]; /* Saved only */
>> u32 vgic_apr;
>> - u32 vgic_lr[64]; /* A15 has only 4... */
>> + u32 vgic_lr[VGIC_MAX_LRS];
>> #endif
>> };
>>
>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>> index a0d283c..90a99fd 100644
>> --- a/arch/arm/kvm/vgic.c
>> +++ b/arch/arm/kvm/vgic.c
>> @@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
>>
>> vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
>> vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
>
> There is a bug here. It should be:
> vgic_nr_lr = (vgic_nr_lr & 0x2f) + 1;
>
>> + if (vgic_nr_lr > VGIC_MAX_LRS)
>> + vgic_nr_lr = VGIC_MAX_LRS; /* TODO: Clear remaining LRs */
>
> Why? VGIC_MAX_LRS isn't a configurable value, but a maximum value
> defined by the specification. This is the maximum you can fit in a 6 bit
> field, plus one (1 << 7, exactly).
>
I figures this was something that could potentially change with a
newer gic architecture etc., and then it was good to have the check,
but you're right, then everything would break.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-15 11:00 ` Marc Zyngier
2013-01-15 14:31 ` Christoffer Dall
@ 2013-01-16 15:29 ` Christoffer Dall
2013-01-16 16:09 ` Marc Zyngier
1 sibling, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-16 15:29 UTC (permalink / raw)
To: linux-arm-kernel
[...]
>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>> index 1ace491..f9d1977 100644
>> --- a/arch/arm/include/asm/kvm_vgic.h
>> +++ b/arch/arm/include/asm/kvm_vgic.h
>> @@ -33,6 +33,7 @@
>> #define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
>> #define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
>> #define VGIC_MAX_CPUS KVM_MAX_VCPUS
>> +#define VGIC_MAX_LRS 64
>
> Consider this instead (for the reason below)
> #define VGIC_MAX_LRS (1 << 7)
>
so here you mean (1 << 6), right?
>> /* Sanity checks... */
>> #if (VGIC_MAX_CPUS > 8)
>> @@ -120,7 +121,7 @@ struct vgic_cpu {
>> DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>
>> /* Bitmap of used/free list registers */
>> - DECLARE_BITMAP( lr_used, 64);
>> + DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
>>
>> /* Number of list registers on this CPU */
>> int nr_lr;
>> @@ -132,7 +133,7 @@ struct vgic_cpu {
>> u32 vgic_eisr[2]; /* Saved only */
>> u32 vgic_elrsr[2]; /* Saved only */
>> u32 vgic_apr;
>> - u32 vgic_lr[64]; /* A15 has only 4... */
>> + u32 vgic_lr[VGIC_MAX_LRS];
>> #endif
>> };
>>
>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>> index a0d283c..90a99fd 100644
>> --- a/arch/arm/kvm/vgic.c
>> +++ b/arch/arm/kvm/vgic.c
>> @@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
>>
>> vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
>> vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
>
> There is a bug here. It should be:
> vgic_nr_lr = (vgic_nr_lr & 0x2f) + 1;
>
and here you mean (vgic_nr_lr & 0x3f) + 1
right?
>> + if (vgic_nr_lr > VGIC_MAX_LRS)
>> + vgic_nr_lr = VGIC_MAX_LRS; /* TODO: Clear remaining LRs */
>
> Why? VGIC_MAX_LRS isn't a configurable value, but a maximum value
> defined by the specification. This is the maximum you can fit in a 6 bit
> field, plus one (1 << 7, exactly).
>
>> ret = create_hyp_io_mappings(vgic_vctrl_base,
>> vgic_vctrl_base + resource_size(&vctrl_res),
>> --
>>
>> Thanks,
>> -Christoffer
>>
>
>
> --
> Jazz is not dead. It just smells funny...
>
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-16 15:29 ` Christoffer Dall
@ 2013-01-16 16:09 ` Marc Zyngier
2013-01-16 16:13 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Marc Zyngier @ 2013-01-16 16:09 UTC (permalink / raw)
To: linux-arm-kernel
On 16/01/13 15:29, Christoffer Dall wrote:
> [...]
>
>>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>>> index 1ace491..f9d1977 100644
>>> --- a/arch/arm/include/asm/kvm_vgic.h
>>> +++ b/arch/arm/include/asm/kvm_vgic.h
>>> @@ -33,6 +33,7 @@
>>> #define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
>>> #define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
>>> #define VGIC_MAX_CPUS KVM_MAX_VCPUS
>>> +#define VGIC_MAX_LRS 64
>>
>> Consider this instead (for the reason below)
>> #define VGIC_MAX_LRS (1 << 7)
>>
>
> so here you mean (1 << 6), right?
No. We have a 6 bit field that contains (NR_LRS - 1). So the maximum
value is (0b111111 + 1), which is (1 << 7).
>
>>> /* Sanity checks... */
>>> #if (VGIC_MAX_CPUS > 8)
>>> @@ -120,7 +121,7 @@ struct vgic_cpu {
>>> DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>>
>>> /* Bitmap of used/free list registers */
>>> - DECLARE_BITMAP( lr_used, 64);
>>> + DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
>>>
>>> /* Number of list registers on this CPU */
>>> int nr_lr;
>>> @@ -132,7 +133,7 @@ struct vgic_cpu {
>>> u32 vgic_eisr[2]; /* Saved only */
>>> u32 vgic_elrsr[2]; /* Saved only */
>>> u32 vgic_apr;
>>> - u32 vgic_lr[64]; /* A15 has only 4... */
>>> + u32 vgic_lr[VGIC_MAX_LRS];
>>> #endif
>>> };
>>>
>>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>>> index a0d283c..90a99fd 100644
>>> --- a/arch/arm/kvm/vgic.c
>>> +++ b/arch/arm/kvm/vgic.c
>>> @@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
>>>
>>> vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
>>> vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
>>
>> There is a bug here. It should be:
>> vgic_nr_lr = (vgic_nr_lr & 0x2f) + 1;
>>
>
> and here you mean (vgic_nr_lr & 0x3f) + 1
> right?
Neither. 0x2f is the right value. See the GIC spec, 5.3.2, GICH_VTR
register.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-16 16:09 ` Marc Zyngier
@ 2013-01-16 16:13 ` Christoffer Dall
2013-01-16 16:17 ` [kvmarm] " Marc Zyngier
0 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-16 16:13 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 16, 2013 at 11:09 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/01/13 15:29, Christoffer Dall wrote:
>> [...]
>>
>>>> diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
>>>> index 1ace491..f9d1977 100644
>>>> --- a/arch/arm/include/asm/kvm_vgic.h
>>>> +++ b/arch/arm/include/asm/kvm_vgic.h
>>>> @@ -33,6 +33,7 @@
>>>> #define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
>>>> #define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
>>>> #define VGIC_MAX_CPUS KVM_MAX_VCPUS
>>>> +#define VGIC_MAX_LRS 64
>>>
>>> Consider this instead (for the reason below)
>>> #define VGIC_MAX_LRS (1 << 7)
>>>
>>
>> so here you mean (1 << 6), right?
>
> No. We have a 6 bit field that contains (NR_LRS - 1). So the maximum
> value is (0b111111 + 1), which is (1 << 7).
>
eh, (1 << 7) is 128, and we have a maximum value of 63 (which plus the
one is 64). You can verify this by thinking about having four bits, is
a halfword, which we use hex numbers to deal with, so the number of
values you can decode there is 16, then you have two more bits, which
each doubles the number of values, so this becomes 64 values total,
ie. from 0 through 63. :)
>>
>>>> /* Sanity checks... */
>>>> #if (VGIC_MAX_CPUS > 8)
>>>> @@ -120,7 +121,7 @@ struct vgic_cpu {
>>>> DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>>>
>>>> /* Bitmap of used/free list registers */
>>>> - DECLARE_BITMAP( lr_used, 64);
>>>> + DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
>>>>
>>>> /* Number of list registers on this CPU */
>>>> int nr_lr;
>>>> @@ -132,7 +133,7 @@ struct vgic_cpu {
>>>> u32 vgic_eisr[2]; /* Saved only */
>>>> u32 vgic_elrsr[2]; /* Saved only */
>>>> u32 vgic_apr;
>>>> - u32 vgic_lr[64]; /* A15 has only 4... */
>>>> + u32 vgic_lr[VGIC_MAX_LRS];
>>>> #endif
>>>> };
>>>>
>>>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>>>> index a0d283c..90a99fd 100644
>>>> --- a/arch/arm/kvm/vgic.c
>>>> +++ b/arch/arm/kvm/vgic.c
>>>> @@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
>>>>
>>>> vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
>>>> vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
>>>
>>> There is a bug here. It should be:
>>> vgic_nr_lr = (vgic_nr_lr & 0x2f) + 1;
>>>
>>
>> and here you mean (vgic_nr_lr & 0x3f) + 1
>> right?
>
> Neither. 0x2f is the right value. See the GIC spec, 5.3.2, GICH_VTR
> register.
>
I'm looking at it, and I don't understand why you don't want to
consider bit[4] ?
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management
2013-01-16 16:13 ` Christoffer Dall
@ 2013-01-16 16:17 ` Marc Zyngier
0 siblings, 0 replies; 79+ messages in thread
From: Marc Zyngier @ 2013-01-16 16:17 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, 16 Jan 2013 11:13:08 -0500, Christoffer Dall
<c.dall@virtualopensystems.com> wrote:
> On Wed, Jan 16, 2013 at 11:09 AM, Marc Zyngier <marc.zyngier@arm.com>
> wrote:
>> On 16/01/13 15:29, Christoffer Dall wrote:
>>> [...]
>>>
>>>>> diff --git a/arch/arm/include/asm/kvm_vgic.h
>>>>> b/arch/arm/include/asm/kvm_vgic.h
>>>>> index 1ace491..f9d1977 100644
>>>>> --- a/arch/arm/include/asm/kvm_vgic.h
>>>>> +++ b/arch/arm/include/asm/kvm_vgic.h
>>>>> @@ -33,6 +33,7 @@
>>>>> #define VGIC_NR_PRIVATE_IRQS (VGIC_NR_SGIS + VGIC_NR_PPIS)
>>>>> #define VGIC_NR_SHARED_IRQS (VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
>>>>> #define VGIC_MAX_CPUS KVM_MAX_VCPUS
>>>>> +#define VGIC_MAX_LRS 64
>>>>
>>>> Consider this instead (for the reason below)
>>>> #define VGIC_MAX_LRS (1 << 7)
>>>>
>>>
>>> so here you mean (1 << 6), right?
>>
>> No. We have a 6 bit field that contains (NR_LRS - 1). So the maximum
>> value is (0b111111 + 1), which is (1 << 7).
>>
>
> eh, (1 << 7) is 128, and we have a maximum value of 63 (which plus the
> one is 64). You can verify this by thinking about having four bits, is
> a halfword, which we use hex numbers to deal with, so the number of
> values you can decode there is 16, then you have two more bits, which
> each doubles the number of values, so this becomes 64 values total,
> ie. from 0 through 63. :)
>
Blah. Ignore me, I'm being stupid.
>
>
>>>
>>>>> /* Sanity checks... */
>>>>> #if (VGIC_MAX_CPUS > 8)
>>>>> @@ -120,7 +121,7 @@ struct vgic_cpu {
>>>>> DECLARE_BITMAP( pending_shared, VGIC_NR_SHARED_IRQS);
>>>>>
>>>>> /* Bitmap of used/free list registers */
>>>>> - DECLARE_BITMAP( lr_used, 64);
>>>>> + DECLARE_BITMAP( lr_used, VGIC_MAX_LRS);
>>>>>
>>>>> /* Number of list registers on this CPU */
>>>>> int nr_lr;
>>>>> @@ -132,7 +133,7 @@ struct vgic_cpu {
>>>>> u32 vgic_eisr[2]; /* Saved only */
>>>>> u32 vgic_elrsr[2]; /* Saved only */
>>>>> u32 vgic_apr;
>>>>> - u32 vgic_lr[64]; /* A15 has only 4... */
>>>>> + u32 vgic_lr[VGIC_MAX_LRS];
>>>>> #endif
>>>>> };
>>>>>
>>>>> diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
>>>>> index a0d283c..90a99fd 100644
>>>>> --- a/arch/arm/kvm/vgic.c
>>>>> +++ b/arch/arm/kvm/vgic.c
>>>>> @@ -1345,6 +1345,8 @@ int kvm_vgic_hyp_init(void)
>>>>>
>>>>> vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
>>>>> vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
>>>>
>>>> There is a bug here. It should be:
>>>> vgic_nr_lr = (vgic_nr_lr & 0x2f) + 1;
>>>>
>>>
>>> and here you mean (vgic_nr_lr & 0x3f) + 1
>>> right?
>>
>> Neither. 0x2f is the right value. See the GIC spec, 5.3.2, GICH_VTR
>> register.
>>
> I'm looking at it, and I don't understand why you don't want to
> consider bit[4] ?
Because it's not a prime number? ;-)
I think I should stay away from patches these days, I'm clearly not
thinking straight. Thanks for coping with my lack of brain.
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5 08/12] ARM: KVM: vgic: retire queued, disabled interrupts
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (6 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 07/12] ARM: KVM: VGIC virtual CPU interface management Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-08 18:42 ` [PATCH v5 09/12] ARM: KVM: VGIC interrupt injection Christoffer Dall
` (4 subsequent siblings)
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
An interrupt may have been disabled after being made pending on the
CPU interface (the classic case is a timer running while we're
rebooting the guest - the interrupt would kick as soon as the CPU
interface gets enabled, with deadly consequences).
The solution is to examine already active LRs, and check the
interrupt is still enabled. If not, just retire it.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/kvm/vgic.c | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 58237d5..49e8b27 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -71,6 +71,7 @@
#define ACCESS_WRITE_VALUE (3 << 1)
#define ACCESS_WRITE_MASK(x) ((x) & (3 << 1))
+static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu);
static void vgic_update_state(struct kvm *kvm);
static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg);
@@ -344,6 +345,7 @@ static bool handle_mmio_clear_enable_reg(struct kvm_vcpu *vcpu,
if (mmio->is_write) {
if (offset < 4) /* Force SGI enabled */
*reg |= 0xffff;
+ vgic_retire_disabled_irqs(vcpu);
vgic_update_state(vcpu->kvm);
return true;
}
@@ -792,6 +794,34 @@ static void vgic_update_state(struct kvm *kvm)
(((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
#define MK_LR_PEND(src, irq) \
(GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
+
+/*
+ * An interrupt may have been disabled after being made pending on the
+ * CPU interface (the classic case is a timer running while we're
+ * rebooting the guest - the interrupt would kick as soon as the CPU
+ * interface gets enabled, with deadly consequences).
+ *
+ * The solution is to examine already active LRs, and check the
+ * interrupt is still enabled. If not, just retire it.
+ */
+static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ int lr;
+
+ for_each_set_bit(lr, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
+ int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
+
+ if (!vgic_irq_is_enabled(vcpu, irq)) {
+ vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
+ clear_bit(lr, vgic_cpu->lr_used);
+ vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
+ if (vgic_irq_is_active(vcpu, irq))
+ vgic_irq_clear_active(vcpu, irq);
+ }
+ }
+}
+
/*
* Queue an interrupt to a CPU virtual interface. Return true on success,
* or false if it wasn't possible to queue it.
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 09/12] ARM: KVM: VGIC interrupt injection
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (7 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 08/12] ARM: KVM: vgic: retire queued, disabled interrupts Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-08 18:42 ` [PATCH v5 10/12] ARM: KVM: VGIC control interface world switch Christoffer Dall
` (3 subsequent siblings)
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
Plug the interrupt injection code. Interrupts can now be generated
from user space.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/kvm_vgic.h | 8 +++
arch/arm/kvm/arm.c | 55 +++++++++++++++---
arch/arm/kvm/vgic.c | 117 +++++++++++++++++++++++++++++++++++++++
3 files changed, 170 insertions(+), 10 deletions(-)
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index b3133c4..9ff0e52 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -146,6 +146,8 @@ struct kvm_exit_mmio;
int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
+int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
+ bool level);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_exit_mmio *mmio);
@@ -181,6 +183,12 @@ static inline int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
static inline void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu) {}
static inline void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu) {}
+static inline int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
+ unsigned int irq_num, bool level)
+{
+ return 0;
+}
+
static inline int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
{
return 0;
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 4c2b057..8dd949c 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -788,20 +788,49 @@ int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level)
trace_kvm_irq_line(irq_type, vcpu_idx, irq_num, irq_level->level);
- if (irq_type != KVM_ARM_IRQ_TYPE_CPU)
- return -EINVAL;
+ switch (irq_type) {
+ case KVM_ARM_IRQ_TYPE_CPU:
+ if (irqchip_in_kernel(kvm))
+ return -ENXIO;
- if (vcpu_idx >= nrcpus)
- return -EINVAL;
+ if (vcpu_idx >= nrcpus)
+ return -EINVAL;
- vcpu = kvm_get_vcpu(kvm, vcpu_idx);
- if (!vcpu)
- return -EINVAL;
+ vcpu = kvm_get_vcpu(kvm, vcpu_idx);
+ if (!vcpu)
+ return -EINVAL;
- if (irq_num > KVM_ARM_IRQ_CPU_FIQ)
- return -EINVAL;
+ if (irq_num > KVM_ARM_IRQ_CPU_FIQ)
+ return -EINVAL;
+
+ return vcpu_interrupt_line(vcpu, irq_num, level);
+ case KVM_ARM_IRQ_TYPE_PPI:
+ if (!irqchip_in_kernel(kvm))
+ return -ENXIO;
+
+ if (vcpu_idx >= nrcpus)
+ return -EINVAL;
+
+ vcpu = kvm_get_vcpu(kvm, vcpu_idx);
+ if (!vcpu)
+ return -EINVAL;
+
+ if (irq_num < VGIC_NR_SGIS || irq_num >= VGIC_NR_PRIVATE_IRQS)
+ return -EINVAL;
- return vcpu_interrupt_line(vcpu, irq_num, level);
+ return kvm_vgic_inject_irq(kvm, vcpu->vcpu_id, irq_num, level);
+ case KVM_ARM_IRQ_TYPE_SPI:
+ if (!irqchip_in_kernel(kvm))
+ return -ENXIO;
+
+ if (irq_num < VGIC_NR_PRIVATE_IRQS ||
+ irq_num > KVM_ARM_IRQ_GIC_MAX)
+ return -EINVAL;
+
+ return kvm_vgic_inject_irq(kvm, 0, irq_num, level);
+ }
+
+ return -EINVAL;
}
long kvm_arch_vcpu_ioctl(struct file *filp,
@@ -880,6 +909,12 @@ long kvm_arch_vm_ioctl(struct file *filp,
void __user *argp = (void __user *)arg;
switch (ioctl) {
+ case KVM_CREATE_IRQCHIP: {
+ if (vgic_present)
+ return kvm_vgic_create(kvm);
+ else
+ return -ENXIO;
+ }
case KVM_SET_DEVICE_ADDRESS: {
struct kvm_device_address dev_addr;
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 49e8b27..65e5282 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -73,6 +73,7 @@
static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu);
static void vgic_update_state(struct kvm *kvm);
+static void vgic_kick_vcpus(struct kvm *kvm);
static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg);
static u32 *vgic_bitmap_get_reg(struct vgic_bitmap *x,
@@ -696,6 +697,9 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
kvm_prepare_mmio(run, mmio);
kvm_handle_mmio_return(vcpu, run);
+ if (updated_state)
+ vgic_kick_vcpus(vcpu->kvm);
+
return true;
}
@@ -1092,6 +1096,119 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
return test_bit(vcpu->vcpu_id, &dist->irq_pending_on_cpu);
}
+static void vgic_kick_vcpus(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu;
+ int c;
+
+ /*
+ * We've injected an interrupt, time to find out who deserves
+ * a good kick...
+ */
+ kvm_for_each_vcpu(c, vcpu, kvm) {
+ if (kvm_vgic_vcpu_pending_irq(vcpu))
+ kvm_vcpu_kick(vcpu);
+ }
+}
+
+static int vgic_validate_injection(struct kvm_vcpu *vcpu, int irq, int level)
+{
+ int is_edge = vgic_irq_is_edge(vcpu, irq);
+ int state = vgic_dist_irq_is_pending(vcpu, irq);
+
+ /*
+ * Only inject an interrupt if:
+ * - edge triggered and we have a rising edge
+ * - level triggered and we change level
+ */
+ if (is_edge)
+ return level > state;
+ else
+ return level != state;
+}
+
+static bool vgic_update_irq_state(struct kvm *kvm, int cpuid,
+ unsigned int irq_num, bool level)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu;
+ int is_edge, is_level;
+ int enabled;
+ bool ret = true;
+
+ spin_lock(&dist->lock);
+
+ vcpu = kvm_get_vcpu(kvm, cpuid);
+ is_edge = vgic_irq_is_edge(vcpu, irq_num);
+ is_level = !is_edge;
+
+ if (!vgic_validate_injection(vcpu, irq_num, level)) {
+ ret = false;
+ goto out;
+ }
+
+ if (irq_num >= VGIC_NR_PRIVATE_IRQS) {
+ cpuid = dist->irq_spi_cpu[irq_num - VGIC_NR_PRIVATE_IRQS];
+ vcpu = kvm_get_vcpu(kvm, cpuid);
+ }
+
+ kvm_debug("Inject IRQ%d level %d CPU%d\n", irq_num, level, cpuid);
+
+ if (level)
+ vgic_dist_irq_set(vcpu, irq_num);
+ else
+ vgic_dist_irq_clear(vcpu, irq_num);
+
+ enabled = vgic_irq_is_enabled(vcpu, irq_num);
+
+ if (!enabled) {
+ ret = false;
+ goto out;
+ }
+
+ if (is_level && vgic_irq_is_active(vcpu, irq_num)) {
+ /*
+ * Level interrupt in progress, will be picked up
+ * when EOId.
+ */
+ ret = false;
+ goto out;
+ }
+
+ if (level) {
+ vgic_cpu_irq_set(vcpu, irq_num);
+ set_bit(cpuid, &dist->irq_pending_on_cpu);
+ }
+
+out:
+ spin_unlock(&dist->lock);
+
+ return ret;
+}
+
+/**
+ * kvm_vgic_inject_irq - Inject an IRQ from a device to the vgic
+ * @kvm: The VM structure pointer
+ * @cpuid: The CPU for PPIs
+ * @irq_num: The IRQ number that is assigned to the device
+ * @level: Edge-triggered: true: to trigger the interrupt
+ * false: to ignore the call
+ * Level-sensitive true: activates an interrupt
+ * false: deactivates an interrupt
+ *
+ * The GIC is not concerned with devices being active-LOW or active-HIGH for
+ * level-sensitive interrupts. You can think of the level parameter as 1
+ * being HIGH and 0 being LOW and all devices being active-HIGH.
+ */
+int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
+ bool level)
+{
+ if (vgic_update_irq_state(kvm, cpuid, irq_num, level))
+ vgic_kick_vcpus(kvm);
+
+ return 0;
+}
+
static bool vgic_ioaddr_overlap(struct kvm *kvm)
{
phys_addr_t dist = kvm->arch.vgic.vgic_dist_base;
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 10/12] ARM: KVM: VGIC control interface world switch
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (8 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 09/12] ARM: KVM: VGIC interrupt injection Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-08 18:42 ` [PATCH v5 11/12] ARM: KVM: VGIC initialisation code Christoffer Dall
` (2 subsequent siblings)
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
Enable the VGIC control interface to be save-restored on world switch.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/kernel/asm-offsets.c | 12 ++++++
arch/arm/kvm/interrupts_head.S | 74 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 86 insertions(+)
diff --git a/arch/arm/kernel/asm-offsets.c b/arch/arm/kernel/asm-offsets.c
index c8b3272..17cea2e 100644
--- a/arch/arm/kernel/asm-offsets.c
+++ b/arch/arm/kernel/asm-offsets.c
@@ -169,6 +169,18 @@ int main(void)
DEFINE(VCPU_HxFAR, offsetof(struct kvm_vcpu, arch.hxfar));
DEFINE(VCPU_HPFAR, offsetof(struct kvm_vcpu, arch.hpfar));
DEFINE(VCPU_HYP_PC, offsetof(struct kvm_vcpu, arch.hyp_pc));
+#ifdef CONFIG_KVM_ARM_VGIC
+ DEFINE(VCPU_VGIC_CPU, offsetof(struct kvm_vcpu, arch.vgic_cpu));
+ DEFINE(VGIC_CPU_HCR, offsetof(struct vgic_cpu, vgic_hcr));
+ DEFINE(VGIC_CPU_VMCR, offsetof(struct vgic_cpu, vgic_vmcr));
+ DEFINE(VGIC_CPU_MISR, offsetof(struct vgic_cpu, vgic_misr));
+ DEFINE(VGIC_CPU_EISR, offsetof(struct vgic_cpu, vgic_eisr));
+ DEFINE(VGIC_CPU_ELRSR, offsetof(struct vgic_cpu, vgic_elrsr));
+ DEFINE(VGIC_CPU_APR, offsetof(struct vgic_cpu, vgic_apr));
+ DEFINE(VGIC_CPU_LR, offsetof(struct vgic_cpu, vgic_lr));
+ DEFINE(VGIC_CPU_NR_LR, offsetof(struct vgic_cpu, nr_lr));
+ DEFINE(KVM_VGIC_VCTRL, offsetof(struct kvm, arch.vgic.vctrl_base));
+#endif
DEFINE(KVM_VTTBR, offsetof(struct kvm, arch.vttbr));
#endif
return 0;
diff --git a/arch/arm/kvm/interrupts_head.S b/arch/arm/kvm/interrupts_head.S
index f59a580..b4276ed 100644
--- a/arch/arm/kvm/interrupts_head.S
+++ b/arch/arm/kvm/interrupts_head.S
@@ -1,3 +1,5 @@
+#include <asm/hardware/gic.h>
+
#define VCPU_USR_REG(_reg_nr) (VCPU_USR_REGS + (_reg_nr * 4))
#define VCPU_USR_SP (VCPU_USR_REG(13))
#define VCPU_USR_LR (VCPU_USR_REG(14))
@@ -371,6 +373,49 @@ vcpu .req r0 @ vcpu pointer always in r0
* Assumes vcpu pointer in vcpu reg
*/
.macro save_vgic_state
+#ifdef CONFIG_KVM_ARM_VGIC
+ /* Get VGIC VCTRL base into r2 */
+ ldr r2, [vcpu, #VCPU_KVM]
+ ldr r2, [r2, #KVM_VGIC_VCTRL]
+ cmp r2, #0
+ beq 2f
+
+ /* Compute the address of struct vgic_cpu */
+ add r11, vcpu, #VCPU_VGIC_CPU
+
+ /* Save all interesting registers */
+ ldr r3, [r2, #GICH_HCR]
+ ldr r4, [r2, #GICH_VMCR]
+ ldr r5, [r2, #GICH_MISR]
+ ldr r6, [r2, #GICH_EISR0]
+ ldr r7, [r2, #GICH_EISR1]
+ ldr r8, [r2, #GICH_ELRSR0]
+ ldr r9, [r2, #GICH_ELRSR1]
+ ldr r10, [r2, #GICH_APR]
+
+ str r3, [r11, #VGIC_CPU_HCR]
+ str r4, [r11, #VGIC_CPU_VMCR]
+ str r5, [r11, #VGIC_CPU_MISR]
+ str r6, [r11, #VGIC_CPU_EISR]
+ str r7, [r11, #(VGIC_CPU_EISR + 4)]
+ str r8, [r11, #VGIC_CPU_ELRSR]
+ str r9, [r11, #(VGIC_CPU_ELRSR + 4)]
+ str r10, [r11, #VGIC_CPU_APR]
+
+ /* Clear GICH_HCR */
+ mov r5, #0
+ str r5, [r2, #GICH_HCR]
+
+ /* Save list registers */
+ add r2, r2, #GICH_LR0
+ add r3, r11, #VGIC_CPU_LR
+ ldr r4, [r11, #VGIC_CPU_NR_LR]
+1: ldr r6, [r2], #4
+ str r6, [r3], #4
+ subs r4, r4, #1
+ bne 1b
+2:
+#endif
.endm
/*
@@ -379,6 +424,35 @@ vcpu .req r0 @ vcpu pointer always in r0
* Assumes vcpu pointer in vcpu reg
*/
.macro restore_vgic_state
+#ifdef CONFIG_KVM_ARM_VGIC
+ /* Get VGIC VCTRL base into r2 */
+ ldr r2, [vcpu, #VCPU_KVM]
+ ldr r2, [r2, #KVM_VGIC_VCTRL]
+ cmp r2, #0
+ beq 2f
+
+ /* Compute the address of struct vgic_cpu */
+ add r11, vcpu, #VCPU_VGIC_CPU
+
+ /* We only restore a minimal set of registers */
+ ldr r3, [r11, #VGIC_CPU_HCR]
+ ldr r4, [r11, #VGIC_CPU_VMCR]
+ ldr r8, [r11, #VGIC_CPU_APR]
+
+ str r3, [r2, #GICH_HCR]
+ str r4, [r2, #GICH_VMCR]
+ str r8, [r2, #GICH_APR]
+
+ /* Restore list registers */
+ add r2, r2, #GICH_LR0
+ add r3, r11, #VGIC_CPU_LR
+ ldr r4, [r11, #VGIC_CPU_NR_LR]
+1: ldr r6, [r3], #4
+ str r6, [r2], #4
+ subs r4, r4, #1
+ bne 1b
+2:
+#endif
.endm
.equ vmentry, 0
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 11/12] ARM: KVM: VGIC initialisation code
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (9 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 10/12] ARM: KVM: VGIC control interface world switch Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-08 18:42 ` [PATCH v5 12/12] ARM: KVM: Add VGIC configuration option Christoffer Dall
2013-01-09 16:26 ` [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Christoffer Dall
12 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
Add the init code for the hypervisor, the virtual machine, and
the virtual CPUs.
An interrupt handler is also wired to allow the VGIC maintenance
interrupts, used to deal with level triggered interrupts and LR
underflows.
A CPU hotplug notifier is registered to disable/enable the interrupt
as requested.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/include/asm/kvm_vgic.h | 11 ++
arch/arm/kvm/arm.c | 15 +++
arch/arm/kvm/vgic.c | 223 +++++++++++++++++++++++++++++++++++++++
3 files changed, 249 insertions(+)
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index 9ff0e52..5e81e28 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -71,6 +71,7 @@ struct vgic_bytemap {
struct vgic_dist {
#ifdef CONFIG_KVM_ARM_VGIC
spinlock_t lock;
+ bool ready;
/* Virtual control interface mapping */
void __iomem *vctrl_base;
@@ -144,6 +145,10 @@ struct kvm_exit_mmio;
#ifdef CONFIG_KVM_ARM_VGIC
int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
+int kvm_vgic_hyp_init(void);
+int kvm_vgic_init(struct kvm *kvm);
+int kvm_vgic_create(struct kvm *kvm);
+int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu);
void kvm_vgic_sync_to_cpu(struct kvm_vcpu *vcpu);
void kvm_vgic_sync_from_cpu(struct kvm_vcpu *vcpu);
int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
@@ -153,6 +158,7 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_exit_mmio *mmio);
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.vctrl_base))
+#define vgic_initialized(k) ((k)->arch.vgic.ready)
#else
static inline int kvm_vgic_hyp_init(void)
@@ -204,6 +210,11 @@ static inline int irqchip_in_kernel(struct kvm *kvm)
{
return 0;
}
+
+static inline bool vgic_initialized(struct kvm *kvm)
+{
+ return true;
+}
#endif
#endif
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 8dd949c..ac72a8f 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -659,6 +659,17 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
if (unlikely(vcpu->arch.target < 0))
return -ENOEXEC;
+ /*
+ * Initialize the VGIC before running a vcpu the first time on
+ * this VM.
+ */
+ if (irqchip_in_kernel(vcpu->kvm) &&
+ unlikely(!vgic_initialized(vcpu->kvm))) {
+ ret = kvm_vgic_init(vcpu->kvm);
+ if (ret)
+ return ret;
+ }
+
if (run->exit_reason == KVM_EXIT_MMIO) {
ret = kvm_handle_mmio_return(vcpu, vcpu->run);
if (ret)
@@ -1066,6 +1077,10 @@ static int init_hyp_mode(void)
if (err)
goto out_free_vfp;
+#ifdef CONFIG_KVM_ARM_VGIC
+ vgic_present = true;
+#endif
+
kvm_info("Hyp mode initialized successfully\n");
return 0;
out_free_vfp:
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 65e5282..083639b 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -16,11 +16,19 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/cpu.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+
#include <asm/kvm_emulate.h>
+#include <asm/hardware/gic.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
/*
* How the whole thing works (courtesy of Christoffer Dall):
@@ -62,6 +70,14 @@
#define VGIC_ADDR_UNDEF (-1)
#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
+/* Physical address of vgic virtual cpu interface */
+static phys_addr_t vgic_vcpu_base;
+
+/* Virtual control interface base address */
+static void __iomem *vgic_vctrl_base;
+
+static struct device_node *vgic_node;
+
#define ACCESS_READ_VALUE (1 << 0)
#define ACCESS_READ_RAZ (0 << 0)
#define ACCESS_READ_MASK(x) ((x) & (1 << 0))
@@ -75,6 +91,9 @@ static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu);
static void vgic_update_state(struct kvm *kvm);
static void vgic_kick_vcpus(struct kvm *kvm);
static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg);
+static u32 vgic_nr_lr;
+
+static unsigned int vgic_maint_irq;
static u32 *vgic_bitmap_get_reg(struct vgic_bitmap *x,
int cpuid, u32 offset)
@@ -1209,6 +1228,210 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
return 0;
}
+static irqreturn_t vgic_maintenance_handler(int irq, void *data)
+{
+ /*
+ * We cannot rely on the vgic maintenance interrupt to be
+ * delivered synchronously. This means we can only use it to
+ * exit the VM, and we perform the handling of EOIed
+ * interrupts on the exit path (see vgic_process_maintenance).
+ */
+ return IRQ_HANDLED;
+}
+
+int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
+{
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int i;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return 0;
+
+ if (vcpu->vcpu_id >= VGIC_MAX_CPUS)
+ return -EBUSY;
+
+ for (i = 0; i < VGIC_NR_IRQS; i++) {
+ if (i < VGIC_NR_PPIS)
+ vgic_bitmap_set_irq_val(&dist->irq_enabled,
+ vcpu->vcpu_id, i, 1);
+ if (i < VGIC_NR_PRIVATE_IRQS)
+ vgic_bitmap_set_irq_val(&dist->irq_cfg,
+ vcpu->vcpu_id, i, VGIC_CFG_EDGE);
+
+ vgic_cpu->vgic_irq_lr_map[i] = LR_EMPTY;
+ }
+
+ /*
+ * By forcing VMCR to zero, the GIC will restore the binary
+ * points to their reset values. Anything else resets to zero
+ * anyway.
+ */
+ vgic_cpu->vgic_vmcr = 0;
+
+ vgic_cpu->nr_lr = vgic_nr_lr;
+ vgic_cpu->vgic_hcr = GICH_HCR_EN; /* Get the show on the road... */
+
+ return 0;
+}
+
+static void vgic_init_maintenance_interrupt(void *info)
+{
+ enable_percpu_irq(vgic_maint_irq, 0);
+}
+
+static int vgic_cpu_notify(struct notifier_block *self,
+ unsigned long action, void *cpu)
+{
+ switch (action) {
+ case CPU_STARTING:
+ case CPU_STARTING_FROZEN:
+ vgic_init_maintenance_interrupt(NULL);
+ break;
+ case CPU_DYING:
+ case CPU_DYING_FROZEN:
+ disable_percpu_irq(vgic_maint_irq);
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block vgic_cpu_nb = {
+ .notifier_call = vgic_cpu_notify,
+};
+
+int kvm_vgic_hyp_init(void)
+{
+ int ret;
+ struct resource vctrl_res;
+ struct resource vcpu_res;
+
+ vgic_node = of_find_compatible_node(NULL, NULL, "arm,cortex-a15-gic");
+ if (!vgic_node) {
+ kvm_err("error: no compatible vgic node in DT\n");
+ return -ENODEV;
+ }
+
+ vgic_maint_irq = irq_of_parse_and_map(vgic_node, 0);
+ if (!vgic_maint_irq) {
+ kvm_err("error getting vgic maintenance irq from DT\n");
+ ret = -ENXIO;
+ goto out;
+ }
+
+ ret = request_percpu_irq(vgic_maint_irq, vgic_maintenance_handler,
+ "vgic", kvm_get_running_vcpus());
+ if (ret) {
+ kvm_err("Cannot register interrupt %d\n", vgic_maint_irq);
+ goto out;
+ }
+
+ ret = register_cpu_notifier(&vgic_cpu_nb);
+ if (ret) {
+ kvm_err("Cannot register vgic CPU notifier\n");
+ goto out_free_irq;
+ }
+
+ ret = of_address_to_resource(vgic_node, 2, &vctrl_res);
+ if (ret) {
+ kvm_err("Cannot obtain VCTRL resource\n");
+ goto out_free_irq;
+ }
+
+ vgic_vctrl_base = of_iomap(vgic_node, 2);
+ if (!vgic_vctrl_base) {
+ kvm_err("Cannot ioremap VCTRL\n");
+ ret = -ENOMEM;
+ goto out_free_irq;
+ }
+
+ vgic_nr_lr = readl_relaxed(vgic_vctrl_base + GICH_VTR);
+ vgic_nr_lr = (vgic_nr_lr & 0x1f) + 1;
+
+ ret = create_hyp_io_mappings(vgic_vctrl_base,
+ vgic_vctrl_base + resource_size(&vctrl_res),
+ vctrl_res.start);
+ if (ret) {
+ kvm_err("Cannot map VCTRL into hyp\n");
+ goto out_unmap;
+ }
+
+ kvm_info("%s@%llx IRQ%d\n", vgic_node->name,
+ vctrl_res.start, vgic_maint_irq);
+ on_each_cpu(vgic_init_maintenance_interrupt, NULL, 1);
+
+ if (of_address_to_resource(vgic_node, 3, &vcpu_res)) {
+ kvm_err("Cannot obtain VCPU resource\n");
+ ret = -ENXIO;
+ goto out_unmap;
+ }
+ vgic_vcpu_base = vcpu_res.start;
+
+ goto out;
+
+out_unmap:
+ iounmap(vgic_vctrl_base);
+out_free_irq:
+ free_percpu_irq(vgic_maint_irq, kvm_get_running_vcpus());
+out:
+ of_node_put(vgic_node);
+ return ret;
+}
+
+int kvm_vgic_init(struct kvm *kvm)
+{
+ int ret = 0, i;
+
+ mutex_lock(&kvm->lock);
+
+ if (vgic_initialized(kvm))
+ goto out;
+
+ if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
+ IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
+ kvm_err("Need to set vgic cpu and dist addresses first\n");
+ ret = -ENXIO;
+ goto out;
+ }
+
+ ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
+ vgic_vcpu_base, KVM_VGIC_V2_CPU_SIZE);
+ if (ret) {
+ kvm_err("Unable to remap VGIC CPU to VCPU\n");
+ goto out;
+ }
+
+ for (i = VGIC_NR_PRIVATE_IRQS; i < VGIC_NR_IRQS; i += 4)
+ vgic_set_target_reg(kvm, 0, i);
+
+ kvm->arch.vgic.ready = true;
+out:
+ mutex_unlock(&kvm->lock);
+ return ret;
+}
+
+int kvm_vgic_create(struct kvm *kvm)
+{
+ int ret = 0;
+
+ mutex_lock(&kvm->lock);
+
+ if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
+ ret = -EEXIST;
+ goto out;
+ }
+
+ spin_lock_init(&kvm->arch.vgic.lock);
+ kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
+ kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
+ kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
+
+out:
+ mutex_unlock(&kvm->lock);
+ return ret;
+}
+
static bool vgic_ioaddr_overlap(struct kvm *kvm)
{
phys_addr_t dist = kvm->arch.vgic.vgic_dist_base;
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 12/12] ARM: KVM: Add VGIC configuration option
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (10 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 11/12] ARM: KVM: VGIC initialisation code Christoffer Dall
@ 2013-01-08 18:42 ` Christoffer Dall
2013-01-09 13:28 ` Sergei Shtylyov
2013-01-09 16:26 ` [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Christoffer Dall
12 siblings, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-08 18:42 UTC (permalink / raw)
To: linux-arm-kernel
From: Marc Zyngier <marc.zyngier@arm.com>
It is now possible to select the VGIC configuration option.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
arch/arm/kvm/Kconfig | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 05227cb..d32e33f 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -51,6 +51,14 @@ config KVM_ARM_MAX_VCPUS
large, so only choose a reasonable number that you expect to
actually use.
+config KVM_ARM_VGIC
+ bool "KVM support for Virtual GIC"
+ depends on KVM_ARM_HOST && OF
+ select HAVE_KVM_IRQCHIP
+ default y
+ ---help---
+ Adds support for a hardware assisted, in-kernel GIC emulation.
+
source drivers/virtio/Kconfig
endif # VIRTUALIZATION
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5 12/12] ARM: KVM: Add VGIC configuration option
2013-01-08 18:42 ` [PATCH v5 12/12] ARM: KVM: Add VGIC configuration option Christoffer Dall
@ 2013-01-09 13:28 ` Sergei Shtylyov
2013-01-09 16:42 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Sergei Shtylyov @ 2013-01-09 13:28 UTC (permalink / raw)
To: linux-arm-kernel
Hello.
On 08-01-2013 22:42, Christoffer Dall wrote:
> From: Marc Zyngier <marc.zyngier@arm.com>
> It is now possible to select the VGIC configuration option.
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
> ---
> arch/arm/kvm/Kconfig | 8 ++++++++
> 1 file changed, 8 insertions(+)
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> index 05227cb..d32e33f 100644
> --- a/arch/arm/kvm/Kconfig
> +++ b/arch/arm/kvm/Kconfig
> @@ -51,6 +51,14 @@ config KVM_ARM_MAX_VCPUS
> large, so only choose a reasonable number that you expect to
> actually use.
>
> +config KVM_ARM_VGIC
> + bool "KVM support for Virtual GIC"
Please indent with tab, as below.
> + depends on KVM_ARM_HOST && OF
> + select HAVE_KVM_IRQCHIP
> + default y
> + ---help---
> + Adds support for a hardware assisted, in-kernel GIC emulation.
> +
WBR, Sergei
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5 12/12] ARM: KVM: Add VGIC configuration option
2013-01-09 13:28 ` Sergei Shtylyov
@ 2013-01-09 16:42 ` Christoffer Dall
0 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 16:42 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 9, 2013 at 8:28 AM, Sergei Shtylyov <sshtylyov@mvista.com> wrote:
> Hello.
>
>
> On 08-01-2013 22:42, Christoffer Dall wrote:
>
>> From: Marc Zyngier <marc.zyngier@arm.com>
>
>
>> It is now possible to select the VGIC configuration option.
>
>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
>> ---
>> arch/arm/kvm/Kconfig | 8 ++++++++
>> 1 file changed, 8 insertions(+)
>
>
>> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
>> index 05227cb..d32e33f 100644
>> --- a/arch/arm/kvm/Kconfig
>> +++ b/arch/arm/kvm/Kconfig
>> @@ -51,6 +51,14 @@ config KVM_ARM_MAX_VCPUS
>> large, so only choose a reasonable number that you expect to
>> actually use.
>>
>> +config KVM_ARM_VGIC
>> + bool "KVM support for Virtual GIC"
>
>
> Please indent with tab, as below.
>
>
fixed both, thanks.
^ permalink raw reply [flat|nested] 79+ messages in thread
* [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-08 18:41 [PATCH v5 00/12] KVM/ARM vGIC support Christoffer Dall
` (11 preceding siblings ...)
2013-01-08 18:42 ` [PATCH v5 12/12] ARM: KVM: Add VGIC configuration option Christoffer Dall
@ 2013-01-09 16:26 ` Christoffer Dall
2013-01-09 16:26 ` [PATCH v5.1 1/2] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl Christoffer Dall
` (2 more replies)
12 siblings, 3 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 16:26 UTC (permalink / raw)
To: linux-arm-kernel
Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
to make it obvious that this is ARM specific in lack of a better generic
interface.
Once we agree on a better interface the KVM/ARM code can also take
advantage of that, but until then we don't want to hold up the KVM/ARM
patches.
A new complete series of these patches including this change can be
pulled from:
git://github.com/virtualopensystems/linux-kvm-arm.git
kvm-arm-v15-devaddr-v2
-Christoffer
---
Christoffer Dall (2):
KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
ARM: KVM: VGIC accept vcpu and dist base addresses from user space
Documentation/virtual/kvm/api.txt | 38 ++++++
arch/arm/include/asm/hardware/gic.h | 25 ++++
arch/arm/include/asm/kvm_host.h | 18 +++
arch/arm/include/asm/kvm_vgic.h | 89 +++++++++++++++
arch/arm/include/uapi/asm/kvm.h | 16 +++
arch/arm/kvm/Makefile | 1
arch/arm/kvm/arm.c | 92 +++++++++++++++-
arch/arm/kvm/interrupts.S | 4 +
arch/arm/kvm/mmio.c | 3 +
arch/arm/kvm/vgic.c | 206 +++++++++++++++++++++++++++++++++++
include/uapi/linux/kvm.h | 8 +
11 files changed, 498 insertions(+), 2 deletions(-)
create mode 100644 arch/arm/include/asm/kvm_vgic.h
create mode 100644 arch/arm/kvm/vgic.c
--
^ permalink raw reply [flat|nested] 79+ messages in thread* [PATCH v5.1 1/2] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
2013-01-09 16:26 ` [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Christoffer Dall
@ 2013-01-09 16:26 ` Christoffer Dall
2013-01-09 16:26 ` [PATCH v5.1 2/2] ARM: KVM: VGIC accept vcpu and dist base addresses from user space Christoffer Dall
2013-01-09 16:48 ` [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Alexander Graf
2 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 16:26 UTC (permalink / raw)
To: linux-arm-kernel
On ARM (and possibly other architectures) some bits are specific to the
model being emulated for the guest and user space needs a way to tell
the kernel about those bits. An example is mmio device base addresses,
where KVM must know the base address for a given device to properly
emulate mmio accesses within a certain address range or directly map a
device with virtualiation extensions into the guest address space.
We try to make this API slightly more generic than for our specific use,
but so far only the VGIC uses this feature.
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
Documentation/virtual/kvm/api.txt | 37 +++++++++++++++++++++++++++++++++++++
arch/arm/include/uapi/asm/kvm.h | 13 +++++++++++++
arch/arm/kvm/arm.c | 23 ++++++++++++++++++++++-
include/uapi/linux/kvm.h | 8 ++++++++
4 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 38066a7a..c2ebd9e 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2206,6 +2206,43 @@ This ioctl returns the guest registers that are supported for the
KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
+4.80 KVM_ARM_SET_DEVICE_ADDR
+
+Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
+Architectures: arm
+Type: vm ioctl
+Parameters: struct kvm_arm_device_address (in)
+Returns: 0 on success, -1 on error
+Errors:
+ ENODEV: The device id is unknown
+ ENXIO: Device not supported on current system
+ EEXIST: Address already set
+ E2BIG: Address outside guest physical address space
+
+struct kvm_arm_device_addr {
+ __u64 id;
+ __u64 addr;
+};
+
+Specify a device address in the guest's physical address space where guests
+can access emulated or directly exposed devices, which the host kernel needs
+to know about. The id field is an architecture specific identifier for a
+specific device.
+
+ARM divides the id field into two parts, a device id and an address type id
+specific to the individual device.
+
+ ?bits: | 63 ... 32 | 31 ... 16 | 15 ... 0 |
+ field: | 0x00000000 | device id | addr type id |
+
+ARM currently only require this when using the in-kernel GIC support for the
+hardware VGIC features, using KVM_ARM_DEVICE_VGIC_V2 as the device id. When
+setting the base address for the guest's mapping of the VGIC virtual CPU
+and distributor interface, the ioctl must be called after calling
+KVM_CREATE_IRQCHIP, but before calling KVM_RUN on any of the VCPUs. Calling
+this ioctl twice for any of the base addresses will return -EEXIST.
+
+
5. The kvm_run structure
------------------------
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 73b9615..09911a7 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -65,6 +65,19 @@ struct kvm_regs {
#define KVM_ARM_TARGET_CORTEX_A15 0
#define KVM_ARM_NUM_TARGETS 1
+/* KVM_SET_DEVICE_ADDRESS ioctl id encoding */
+#define KVM_DEVICE_TYPE_SHIFT 0
+#define KVM_DEVICE_TYPE_MASK (0xffff << KVM_DEVICE_TYPE_SHIFT)
+#define KVM_DEVICE_ID_SHIFT 16
+#define KVM_DEVICE_ID_MASK (0xffff << KVM_DEVICE_ID_SHIFT)
+
+/* Supported device IDs */
+#define KVM_ARM_DEVICE_VGIC_V2 0
+
+/* Supported VGIC address types */
+#define KVM_VGIC_V2_ADDR_TYPE_DIST 0
+#define KVM_VGIC_V2_ADDR_TYPE_CPU 1
+
struct kvm_vcpu_init {
__u32 target;
__u32 features[7];
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index f42d828..ce6e455 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -165,6 +165,8 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_COALESCED_MMIO:
r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break;
+ case KVM_CAP_ARM_SET_DEVICE_ADDR:
+ r = 1;
case KVM_CAP_NR_VCPUS:
r = num_online_cpus();
break;
@@ -805,10 +807,29 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
return -EINVAL;
}
+static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
+ struct kvm_arm_device_addr *dev_addr)
+{
+ return -ENODEV;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
- return -EINVAL;
+ struct kvm *kvm = filp->private_data;
+ void __user *argp = (void __user *)arg;
+
+ switch (ioctl) {
+ case KVM_ARM_SET_DEVICE_ADDR: {
+ struct kvm_arm_device_addr dev_addr;
+
+ if (copy_from_user(&dev_addr, argp, sizeof(dev_addr)))
+ return -EFAULT;
+ return kvm_vm_ioctl_set_device_addr(kvm, &dev_addr);
+ }
+ default:
+ return -EINVAL;
+ }
}
static void cpu_init_hyp_mode(void *vector)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index dc63665..1f68151 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -636,6 +636,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_IRQFD_RESAMPLE 82
#define KVM_CAP_PPC_BOOKE_WATCHDOG 83
#define KVM_CAP_PPC_HTAB_FD 84
+#define KVM_CAP_ARM_SET_DEVICE_ADDR 85
#ifdef KVM_CAP_IRQ_ROUTING
@@ -783,6 +784,11 @@ struct kvm_msi {
__u8 pad[16];
};
+struct kvm_arm_device_addr {
+ __u64 id;
+ __u64 addr;
+};
+
/*
* ioctls for VM fds
*/
@@ -868,6 +874,8 @@ struct kvm_s390_ucas_mapping {
#define KVM_ALLOCATE_RMA _IOR(KVMIO, 0xa9, struct kvm_allocate_rma)
/* Available with KVM_CAP_PPC_HTAB_FD */
#define KVM_PPC_GET_HTAB_FD _IOW(KVMIO, 0xaa, struct kvm_get_htab_fd)
+/* Available with KVM_CAP_SET_DEVICE_ADDR */
+#define KVM_ARM_SET_DEVICE_ADDR _IOW(KVMIO, 0xab, struct kvm_arm_device_addr)
/*
* ioctls for vcpu fds
^ permalink raw reply related [flat|nested] 79+ messages in thread* [PATCH v5.1 2/2] ARM: KVM: VGIC accept vcpu and dist base addresses from user space
2013-01-09 16:26 ` [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Christoffer Dall
2013-01-09 16:26 ` [PATCH v5.1 1/2] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl Christoffer Dall
@ 2013-01-09 16:26 ` Christoffer Dall
2013-01-09 16:48 ` [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Alexander Graf
2 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 16:26 UTC (permalink / raw)
To: linux-arm-kernel
User space defines the model to emulate to a guest and should therefore
decide which addresses are used for both the virtual CPU interface
directly mapped in the guest physical address space and for the emulated
distributor interface, which is mapped in software by the in-kernel VGIC
support.
Signed-off-by: Christoffer Dall <c.dall@virtualopensystems.com>
---
Documentation/virtual/kvm/api.txt | 1 +
arch/arm/include/asm/kvm_vgic.h | 9 +++++
arch/arm/include/uapi/asm/kvm.h | 3 ++
arch/arm/kvm/arm.c | 14 ++++++++
arch/arm/kvm/vgic.c | 62 +++++++++++++++++++++++++++++++++++++
5 files changed, 88 insertions(+), 1 deletion(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index c2ebd9e..497fd48 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2218,6 +2218,7 @@ Errors:
ENXIO: Device not supported on current system
EEXIST: Address already set
E2BIG: Address outside guest physical address space
+ EBUSY: Address overlaps with other device range
struct kvm_arm_device_addr {
__u64 id;
diff --git a/arch/arm/include/asm/kvm_vgic.h b/arch/arm/include/asm/kvm_vgic.h
index fcfd530..270dcd2 100644
--- a/arch/arm/include/asm/kvm_vgic.h
+++ b/arch/arm/include/asm/kvm_vgic.h
@@ -22,6 +22,9 @@
#include <asm/hardware/gic.h>
struct vgic_dist {
+ /* Distributor and vcpu interface mapping in the guest */
+ phys_addr_t vgic_dist_base;
+ phys_addr_t vgic_cpu_base;
};
struct vgic_cpu {
@@ -33,6 +36,7 @@ struct kvm_run;
struct kvm_exit_mmio;
#ifdef CONFIG_KVM_ARM_VGIC
+int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
struct kvm_exit_mmio *mmio);
@@ -42,6 +46,11 @@ static inline int kvm_vgic_hyp_init(void)
return 0;
}
+static inline int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
+{
+ return 0;
+}
+
static inline int kvm_vgic_init(struct kvm *kvm)
{
return 0;
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 09911a7..94e893b 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -78,6 +78,9 @@ struct kvm_regs {
#define KVM_VGIC_V2_ADDR_TYPE_DIST 0
#define KVM_VGIC_V2_ADDR_TYPE_CPU 1
+#define KVM_VGIC_V2_DIST_SIZE 0x1000
+#define KVM_VGIC_V2_CPU_SIZE 0x2000
+
struct kvm_vcpu_init {
__u32 target;
__u32 features[7];
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index fcb0dfc..b4e7571 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -858,7 +858,19 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
struct kvm_arm_device_addr *dev_addr)
{
- return -ENODEV;
+ unsigned long dev_id, type;
+
+ dev_id = (dev_addr->id & KVM_DEVICE_ID_MASK) >> KVM_DEVICE_ID_SHIFT;
+ type = (dev_addr->id & KVM_DEVICE_TYPE_MASK) >> KVM_DEVICE_TYPE_SHIFT;
+
+ switch (dev_id) {
+ case KVM_ARM_DEVICE_VGIC_V2:
+ if (!vgic_present)
+ return -ENXIO;
+ return kvm_vgic_set_addr(kvm, type, dev_addr->addr);
+ default:
+ return -ENODEV;
+ }
}
long kvm_arch_vm_ioctl(struct file *filp,
diff --git a/arch/arm/kvm/vgic.c b/arch/arm/kvm/vgic.c
index 1feee5a..cdb7671 100644
--- a/arch/arm/kvm/vgic.c
+++ b/arch/arm/kvm/vgic.c
@@ -22,6 +22,9 @@
#include <linux/io.h>
#include <asm/kvm_emulate.h>
+#define VGIC_ADDR_UNDEF (-1)
+#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
+
#define ACCESS_READ_VALUE (1 << 0)
#define ACCESS_READ_RAZ (0 << 0)
#define ACCESS_READ_MASK(x) ((x) & (1 << 0))
@@ -142,3 +145,62 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
{
return KVM_EXIT_MMIO;
}
+
+static bool vgic_ioaddr_overlap(struct kvm *kvm)
+{
+ phys_addr_t dist = kvm->arch.vgic.vgic_dist_base;
+ phys_addr_t cpu = kvm->arch.vgic.vgic_cpu_base;
+
+ if (IS_VGIC_ADDR_UNDEF(dist) || IS_VGIC_ADDR_UNDEF(cpu))
+ return 0;
+ if ((dist <= cpu && dist + KVM_VGIC_V2_DIST_SIZE > cpu) ||
+ (cpu <= dist && cpu + KVM_VGIC_V2_CPU_SIZE > dist))
+ return -EBUSY;
+ return 0;
+}
+
+static int vgic_ioaddr_assign(struct kvm *kvm, phys_addr_t *ioaddr,
+ phys_addr_t addr, phys_addr_t size)
+{
+ int ret;
+
+ if (!IS_VGIC_ADDR_UNDEF(*ioaddr))
+ return -EEXIST;
+ if (addr + size < addr)
+ return -EINVAL;
+
+ ret = vgic_ioaddr_overlap(kvm);
+ if (ret)
+ return ret;
+ *ioaddr = addr;
+ return ret;
+}
+
+int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
+{
+ int r = 0;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+
+ if (addr & ~KVM_PHYS_MASK)
+ return -E2BIG;
+
+ if (addr & ~PAGE_MASK)
+ return -EINVAL;
+
+ mutex_lock(&kvm->lock);
+ switch (type) {
+ case KVM_VGIC_V2_ADDR_TYPE_DIST:
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_dist_base,
+ addr, KVM_VGIC_V2_DIST_SIZE);
+ break;
+ case KVM_VGIC_V2_ADDR_TYPE_CPU:
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_cpu_base,
+ addr, KVM_VGIC_V2_CPU_SIZE);
+ break;
+ default:
+ r = -ENODEV;
+ }
+
+ mutex_unlock(&kvm->lock);
+ return r;
+}
^ permalink raw reply related [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 16:26 ` [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Christoffer Dall
2013-01-09 16:26 ` [PATCH v5.1 1/2] KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl Christoffer Dall
2013-01-09 16:26 ` [PATCH v5.1 2/2] ARM: KVM: VGIC accept vcpu and dist base addresses from user space Christoffer Dall
@ 2013-01-09 16:48 ` Alexander Graf
2013-01-09 19:50 ` Scott Wood
2 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 16:48 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 17:26, Christoffer Dall wrote:
> Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
> to make it obvious that this is ARM specific in lack of a better generic
> interface.
>
> Once we agree on a better interface the KVM/ARM code can also take
> advantage of that, but until then we don't want to hold up the KVM/ARM
> patches.
Works for me. Scott, are you happy with this one too? We can start to introduce (and fix ARM) with a generic ioctl in the MPIC patches then.
Alex
>
> A new complete series of these patches including this change can be
> pulled from:
> git://github.com/virtualopensystems/linux-kvm-arm.git
> kvm-arm-v15-devaddr-v2
>
> -Christoffer
>
> ---
>
> Christoffer Dall (2):
> KVM: ARM: Introduce KVM_SET_DEVICE_ADDRESS ioctl
> ARM: KVM: VGIC accept vcpu and dist base addresses from user space
>
>
> Documentation/virtual/kvm/api.txt | 38 ++++++
> arch/arm/include/asm/hardware/gic.h | 25 ++++
> arch/arm/include/asm/kvm_host.h | 18 +++
> arch/arm/include/asm/kvm_vgic.h | 89 +++++++++++++++
> arch/arm/include/uapi/asm/kvm.h | 16 +++
> arch/arm/kvm/Makefile | 1
> arch/arm/kvm/arm.c | 92 +++++++++++++++-
> arch/arm/kvm/interrupts.S | 4 +
> arch/arm/kvm/mmio.c | 3 +
> arch/arm/kvm/vgic.c | 206 +++++++++++++++++++++++++++++++++++
> include/uapi/linux/kvm.h | 8 +
> 11 files changed, 498 insertions(+), 2 deletions(-)
> create mode 100644 arch/arm/include/asm/kvm_vgic.h
> create mode 100644 arch/arm/kvm/vgic.c
>
> --
> _______________________________________________
> kvmarm mailing list
> kvmarm at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 16:48 ` [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS Alexander Graf
@ 2013-01-09 19:50 ` Scott Wood
2013-01-09 20:12 ` Alexander Graf
0 siblings, 1 reply; 79+ messages in thread
From: Scott Wood @ 2013-01-09 19:50 UTC (permalink / raw)
To: linux-arm-kernel
On 01/09/2013 10:48:47 AM, Alexander Graf wrote:
>
> On 09.01.2013, at 17:26, Christoffer Dall wrote:
>
> > Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
> > to make it obvious that this is ARM specific in lack of a better
> generic
> > interface.
> >
> > Once we agree on a better interface the KVM/ARM code can also take
> > advantage of that, but until then we don't want to hold up the
> KVM/ARM
> > patches.
>
> Works for me. Scott, are you happy with this one too?
Not really, given that it will stay around forever even after something
new is introduced.
If you're going to change the name, why not just change it to
KVM_SET_DEVICE_CONFIG? Can we change the name later if nothing else
changes (so it's still binary compatible)?
> We can start to introduce (and fix ARM) with a generic ioctl in the
> MPIC patches then.
The ioctl is already generic, except for its name.
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 19:50 ` Scott Wood
@ 2013-01-09 20:12 ` Alexander Graf
2013-01-09 21:15 ` Scott Wood
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 20:12 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 20:50, Scott Wood wrote:
> On 01/09/2013 10:48:47 AM, Alexander Graf wrote:
>> On 09.01.2013, at 17:26, Christoffer Dall wrote:
>> > Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
>> > to make it obvious that this is ARM specific in lack of a better generic
>> > interface.
>> >
>> > Once we agree on a better interface the KVM/ARM code can also take
>> > advantage of that, but until then we don't want to hold up the KVM/ARM
>> > patches.
>> Works for me. Scott, are you happy with this one too?
>
> Not really, given that it will stay around forever even after something new is introduced.
But only in ARM specific code.
> If you're going to change the name, why not just change it to KVM_SET_DEVICE_CONFIG? Can we change the name later if nothing else changes (so it's still binary compatible)?
Because that again implies that it's generic enough. And to reach that conclusion will take more time than we should spend on this now.
>> We can start to introduce (and fix ARM) with a generic ioctl in the MPIC patches then.
>
> The ioctl is already generic, except for its name.
It's making a few wrong assumptions:
* maximum size of value is u64
* combining device id (variable) with addr type id (const) into a single field. It could just be split into multiple fields
* the id is 100% architecture specific. It shouldn't be. At least the "device id" field should be generic.
I'm not sure if we can come up with more problems in that API when staring at it a bit longer and/or we would actually start using it for more things. So for the sake of not holding up the ARM code, I'm perfectly fine to clutter ARM's ioctl handling code with an ioctl that is already deprecated at its introduction, as long as we don't hold everything else up meanwhile.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 20:12 ` Alexander Graf
@ 2013-01-09 21:15 ` Scott Wood
2013-01-09 21:37 ` Alexander Graf
2013-01-10 22:21 ` Marcelo Tosatti
0 siblings, 2 replies; 79+ messages in thread
From: Scott Wood @ 2013-01-09 21:15 UTC (permalink / raw)
To: linux-arm-kernel
On 01/09/2013 02:12:16 PM, Alexander Graf wrote:
>
> On 09.01.2013, at 20:50, Scott Wood wrote:
>
> > On 01/09/2013 10:48:47 AM, Alexander Graf wrote:
> >> On 09.01.2013, at 17:26, Christoffer Dall wrote:
> >> > Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
> >> > to make it obvious that this is ARM specific in lack of a better
> generic
> >> > interface.
> >> >
> >> > Once we agree on a better interface the KVM/ARM code can also
> take
> >> > advantage of that, but until then we don't want to hold up the
> KVM/ARM
> >> > patches.
> >> Works for me. Scott, are you happy with this one too?
> >
> > Not really, given that it will stay around forever even after
> something new is introduced.
>
> But only in ARM specific code.
...which I'll probably have to deal with when Freescale's
virtualization-capable ARM chips come along. I don't see "it's only in
that other architecture" as "it might as well not exist".
> > If you're going to change the name, why not just change it to
> KVM_SET_DEVICE_CONFIG? Can we change the name later if nothing else
> changes (so it's still binary compatible)?
>
> Because that again implies that it's generic enough. And to reach
> that conclusion will take more time than we should spend on this now.
If the conclusion later on is that it is good enough, can the name be
changed then?
> >> We can start to introduce (and fix ARM) with a generic ioctl in
> the MPIC patches then.
> >
> > The ioctl is already generic, except for its name.
>
> It's making a few wrong assumptions:
>
> * maximum size of value is u64
This is tolerable IMHO.
> * combining device id (variable) with addr type id (const) into a
> single field. It could just be split into multiple fields
I agree, but that could be lived with as well.
I get that there's a tradeoff between getting something in now, versus
waiting until the API is more refined. Tagging it with a particular
ISA seems like an odd way of saying "soon to be deprecated", though.
What happens if we're still squabbling over the perfect replacement API
when we're trying to push PPC MPIC stuff in?
Perhaps the threshold for an API becoming "permanent" should not be
acceptance into the tree, but rather the removal of an "experimental"
tag (including a way of shutting off experimental APIs to make sure
you're not depending on them). Sort of like CONFIG_EXPERIMENTAL,
except actually used for its intended purpose (distributions should
have it *off* by default), and preferably managed at runtime. Sort of
like drivers/staging, except for APIs rather than drivers. Changes at
that point should require more justification than before merging, but
would not have the strict compatibility requirement that
non-experimental APIs have. This would make collaboration and testing
easier on APIs that aren't ready to be permanent.
> * the id is 100% architecture specific. It shouldn't be. At least
> the "device id" field should be generic.
That's a documentation issue that could be changed to have all
architectures adopt what is currently specified for ARM, without
breaking compatibility.
> I'm not sure if we can come up with more problems in that API when
> staring at it a bit longer and/or we would actually start using it
> for more things. So for the sake of not holding up the ARM code, I'm
> perfectly fine to clutter ARM's ioctl handling code with an ioctl
> that is already deprecated at its introduction, as long as we don't
> hold everything else up meanwhile.
I'm not in a position to block it, and if I were I presumably would
have seen this in time for feedback to matter. That's different from
actually being happy. :-)
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 21:15 ` Scott Wood
@ 2013-01-09 21:37 ` Alexander Graf
2013-01-09 22:10 ` Scott Wood
2013-01-10 22:28 ` Marcelo Tosatti
2013-01-10 22:21 ` Marcelo Tosatti
1 sibling, 2 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 21:37 UTC (permalink / raw)
To: linux-arm-kernel
Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
> On 01/09/2013 02:12:16 PM, Alexander Graf wrote:
>> On 09.01.2013, at 20:50, Scott Wood wrote:
>> > On 01/09/2013 10:48:47 AM, Alexander Graf wrote:
>> >> On 09.01.2013, at 17:26, Christoffer Dall wrote:
>> >> > Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
>> >> > to make it obvious that this is ARM specific in lack of a better generic
>> >> > interface.
>> >> >
>> >> > Once we agree on a better interface the KVM/ARM code can also take
>> >> > advantage of that, but until then we don't want to hold up the KVM/ARM
>> >> > patches.
>> >> Works for me. Scott, are you happy with this one too?
>> >
>> > Not really, given that it will stay around forever even after something new is introduced.
>> But only in ARM specific code.
>
> ...which I'll probably have to deal with when Freescale's virtualization-capable ARM chips come along. I don't see "it's only in that other architecture" as "it might as well not exist".
I'm saying it's limiting its scope to a few lines of code in arch an arch specific file and makes everyone aware that we really need to come to a conclusion soon.
>
>> > If you're going to change the name, why not just change it to KVM_SET_DEVICE_CONFIG? Can we change the name later if nothing else changes (so it's still binary compatible)?
>> Because that again implies that it's generic enough. And to reach that conclusion will take more time than we should spend on this now.
>
> If the conclusion later on is that it is good enough, can the name be changed then?
We can add another generic name, yes. Changing it won't work because it'd break compatibility.
>
>> >> We can start to introduce (and fix ARM) with a generic ioctl in the MPIC patches then.
>> >
>> > The ioctl is already generic, except for its name.
>> It's making a few wrong assumptions:
>> * maximum size of value is u64
>
> This is tolerable IMHO.
>
>> * combining device id (variable) with addr type id (const) into a single field. It could just be split into multiple fields
>
> I agree, but that could be lived with as well.
>
> I get that there's a tradeoff between getting something in now, versus waiting until the API is more refined. Tagging it with a particular ISA seems like an odd way of saying "soon to be deprecated", though. What happens if we're still squabbling over the perfect replacement API when we're trying to push PPC MPIC stuff in?
Then we're the ones who have to come up with a good interface.
>
> Perhaps the threshold for an API becoming "permanent" should not be acceptance into the tree, but rather the removal of an "experimental" tag (including a way of shutting off experimental APIs to make sure you're not depending on them). Sort of like CONFIG_EXPERIMENTAL, except actually used for its intended purpose (distributions should have it *off* by default), and preferably managed at runtime. Sort of like drivers/staging, except for APIs rather than drivers. Changes at that point should require more justification than before merging, but would not have the strict compatibility requirement that non-experimental APIs have. This would make collaboration and testing easier on APIs that aren't ready to be permanent.
This tag does exist. It's called "not in Linus' tree" :).
>
>> * the id is 100% architecture specific. It shouldn't be. At least the "device id" field should be generic.
>
> That's a documentation issue that could be changed to have all architectures adopt what is currently specified for ARM, without breaking compatibility.
Depends on how we want to design the final layout. I don't have the impression that we've reached final conclusion on this interface. That's all I'm saying.
>
>> I'm not sure if we can come up with more problems in that API when staring at it a bit longer and/or we would actually start using it for more things. So for the sake of not holding up the ARM code, I'm perfectly fine to clutter ARM's ioctl handling code with an ioctl that is already deprecated at its introduction, as long as we don't hold everything else up meanwhile.
>
> I'm not in a position to block it, and if I were I presumably would have seen this in time for feedback to matter. That's different from actually being happy. :-)
For the ARM specific ioctl it has my ack. I'd say let's go with that and start to work on a good interface ASAP. But we need an ok from Marcelo or Gleb too.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 21:37 ` Alexander Graf
@ 2013-01-09 22:10 ` Scott Wood
2013-01-09 22:26 ` Christoffer Dall
2013-01-09 22:30 ` Alexander Graf
2013-01-10 22:28 ` Marcelo Tosatti
1 sibling, 2 replies; 79+ messages in thread
From: Scott Wood @ 2013-01-09 22:10 UTC (permalink / raw)
To: linux-arm-kernel
On 01/09/2013 03:37:20 PM, Alexander Graf wrote:
>
>
> Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
>
> > I get that there's a tradeoff between getting something in now,
> versus waiting until the API is more refined. Tagging it with a
> particular ISA seems like an odd way of saying "soon to be
> deprecated", though. What happens if we're still squabbling over the
> perfect replacement API when we're trying to push PPC MPIC stuff in?
>
> Then we're the ones who have to come up with a good interface.
How about another bad one, with PPC in the name, and some pleas to
hurry things up? :-)
It's not as if there haven't been last-minute requests for API changes
on the PPC side in the past...
> > Perhaps the threshold for an API becoming "permanent" should not be
> acceptance into the tree, but rather the removal of an "experimental"
> tag (including a way of shutting off experimental APIs to make sure
> you're not depending on them). Sort of like CONFIG_EXPERIMENTAL,
> except actually used for its intended purpose (distributions should
> have it *off* by default), and preferably managed at runtime. Sort
> of like drivers/staging, except for APIs rather than drivers.
> Changes at that point should require more justification than before
> merging, but would not have the strict compatibility requirement that
> non-experimental APIs have. This would make collaboration and
> testing easier on APIs that aren't ready to be permanent.
>
> This tag does exist. It's called "not in Linus' tree" :).
Which makes it a pain for multiple people to work on a new feature,
especially when it spans components such as KVM and QEMU, and means
that it gets less testing before the point of no return.
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 22:10 ` Scott Wood
@ 2013-01-09 22:26 ` Christoffer Dall
2013-01-09 22:34 ` Alexander Graf
2013-01-09 22:30 ` Alexander Graf
1 sibling, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-09 22:26 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 9, 2013 at 5:10 PM, Scott Wood <scottwood@freescale.com> wrote:
> On 01/09/2013 03:37:20 PM, Alexander Graf wrote:
>>
>>
>>
>> Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
>>
>> > I get that there's a tradeoff between getting something in now, versus
>> > waiting until the API is more refined. Tagging it with a particular ISA
>> > seems like an odd way of saying "soon to be deprecated", though. What
>> > happens if we're still squabbling over the perfect replacement API when
>> > we're trying to push PPC MPIC stuff in?
>>
>> Then we're the ones who have to come up with a good interface.
>
>
> How about another bad one, with PPC in the name, and some pleas to hurry
> things up? :-)
>
> It's not as if there haven't been last-minute requests for API changes on
> the PPC side in the past...
>
>
This is getting out of hand.
Do you have another API for PPC, which was send for review and not
commented on several months ago that we can unify right now?
If not, let's go with the ARM name and work on the generic API in the mean time.
The end result will be something along 5 lines in a header files and 3
lines in a switch case that return -EINVAL if the interface is
completely deprecated later on, which is not a big problem.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 22:26 ` Christoffer Dall
@ 2013-01-09 22:34 ` Alexander Graf
2013-01-10 11:15 ` Alexander Graf
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 22:34 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 23:26, Christoffer Dall wrote:
> On Wed, Jan 9, 2013 at 5:10 PM, Scott Wood <scottwood@freescale.com> wrote:
>> On 01/09/2013 03:37:20 PM, Alexander Graf wrote:
>>>
>>>
>>>
>>> Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
>>>
>>>> I get that there's a tradeoff between getting something in now, versus
>>>> waiting until the API is more refined. Tagging it with a particular ISA
>>>> seems like an odd way of saying "soon to be deprecated", though. What
>>>> happens if we're still squabbling over the perfect replacement API when
>>>> we're trying to push PPC MPIC stuff in?
>>>
>>> Then we're the ones who have to come up with a good interface.
>>
>>
>> How about another bad one, with PPC in the name, and some pleas to hurry
>> things up? :-)
>>
>> It's not as if there haven't been last-minute requests for API changes on
>> the PPC side in the past...
>>
>>
>
> This is getting out of hand.
>
> Do you have another API for PPC, which was send for review and not
> commented on several months ago that we can unify right now?
>
> If not, let's go with the ARM name and work on the generic API in the mean time.
>
> The end result will be something along 5 lines in a header files and 3
> lines in a switch case that return -EINVAL if the interface is
> completely deprecated later on, which is not a big problem.
Agreed [1].
So what exactly are we waiting for? Acks from kvm maintainers, right?
Alex
[1] It'd probably end up being an ioctl that converts from one calling convention to another, resulting in maybe 5 lines in the switch case.
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 22:34 ` Alexander Graf
@ 2013-01-10 11:15 ` Alexander Graf
2013-01-10 11:18 ` Gleb Natapov
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-10 11:15 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 23:34, Alexander Graf wrote:
>
> On 09.01.2013, at 23:26, Christoffer Dall wrote:
>
>> On Wed, Jan 9, 2013 at 5:10 PM, Scott Wood <scottwood@freescale.com> wrote:
>>> On 01/09/2013 03:37:20 PM, Alexander Graf wrote:
>>>>
>>>>
>>>>
>>>> Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
>>>>
>>>>> I get that there's a tradeoff between getting something in now, versus
>>>>> waiting until the API is more refined. Tagging it with a particular ISA
>>>>> seems like an odd way of saying "soon to be deprecated", though. What
>>>>> happens if we're still squabbling over the perfect replacement API when
>>>>> we're trying to push PPC MPIC stuff in?
>>>>
>>>> Then we're the ones who have to come up with a good interface.
>>>
>>>
>>> How about another bad one, with PPC in the name, and some pleas to hurry
>>> things up? :-)
>>>
>>> It's not as if there haven't been last-minute requests for API changes on
>>> the PPC side in the past...
>>>
>>>
>>
>> This is getting out of hand.
>>
>> Do you have another API for PPC, which was send for review and not
>> commented on several months ago that we can unify right now?
>>
>> If not, let's go with the ARM name and work on the generic API in the mean time.
>>
>> The end result will be something along 5 lines in a header files and 3
>> lines in a switch case that return -EINVAL if the interface is
>> completely deprecated later on, which is not a big problem.
>
> Agreed [1].
>
> So what exactly are we waiting for? Acks from kvm maintainers, right?
In fact, we should probably CC them :)
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 11:15 ` Alexander Graf
@ 2013-01-10 11:18 ` Gleb Natapov
0 siblings, 0 replies; 79+ messages in thread
From: Gleb Natapov @ 2013-01-10 11:18 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Jan 10, 2013 at 12:15:55PM +0100, Alexander Graf wrote:
>
> On 09.01.2013, at 23:34, Alexander Graf wrote:
>
> >
> > On 09.01.2013, at 23:26, Christoffer Dall wrote:
> >
> >> On Wed, Jan 9, 2013 at 5:10 PM, Scott Wood <scottwood@freescale.com> wrote:
> >>> On 01/09/2013 03:37:20 PM, Alexander Graf wrote:
> >>>>
> >>>>
> >>>>
> >>>> Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
> >>>>
> >>>>> I get that there's a tradeoff between getting something in now, versus
> >>>>> waiting until the API is more refined. Tagging it with a particular ISA
> >>>>> seems like an odd way of saying "soon to be deprecated", though. What
> >>>>> happens if we're still squabbling over the perfect replacement API when
> >>>>> we're trying to push PPC MPIC stuff in?
> >>>>
> >>>> Then we're the ones who have to come up with a good interface.
> >>>
> >>>
> >>> How about another bad one, with PPC in the name, and some pleas to hurry
> >>> things up? :-)
> >>>
> >>> It's not as if there haven't been last-minute requests for API changes on
> >>> the PPC side in the past...
> >>>
> >>>
> >>
> >> This is getting out of hand.
> >>
> >> Do you have another API for PPC, which was send for review and not
> >> commented on several months ago that we can unify right now?
> >>
> >> If not, let's go with the ARM name and work on the generic API in the mean time.
> >>
> >> The end result will be something along 5 lines in a header files and 3
> >> lines in a switch case that return -EINVAL if the interface is
> >> completely deprecated later on, which is not a big problem.
> >
> > Agreed [1].
> >
> > So what exactly are we waiting for? Acks from kvm maintainers, right?
>
> In fact, we should probably CC them :)
>
>
I am looking at them right now :) Give me a couple of days please.
--
Gleb.
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 22:10 ` Scott Wood
2013-01-09 22:26 ` Christoffer Dall
@ 2013-01-09 22:30 ` Alexander Graf
2013-01-10 10:17 ` Peter Maydell
1 sibling, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-09 22:30 UTC (permalink / raw)
To: linux-arm-kernel
On 09.01.2013, at 23:10, Scott Wood wrote:
> On 01/09/2013 03:37:20 PM, Alexander Graf wrote:
>> Am 09.01.2013 um 22:15 schrieb Scott Wood <scottwood@freescale.com>:
>> > I get that there's a tradeoff between getting something in now, versus waiting until the API is more refined. Tagging it with a particular ISA seems like an odd way of saying "soon to be deprecated", though. What happens if we're still squabbling over the perfect replacement API when we're trying to push PPC MPIC stuff in?
>> Then we're the ones who have to come up with a good interface.
>
> How about another bad one, with PPC in the name, and some pleas to hurry things up? :-)
Then I'd be the maintainer of that one and tell you whatever I think would be reasonable given the circumstances :).
> It's not as if there haven't been last-minute requests for API changes on the PPC side in the past...
I don't remember a full PPC target merge ever relying on an API change like this.
In fact, in this particular case one could merge all of the patches except for the particular ioctl implementation and just give the respective addresses default values until there is an API to set them, similar to how we did things with PVR in the beginning.
But this is something that's always up to the respective maintainers and I'm not going to interfere with how they do their job :).
>
>> > Perhaps the threshold for an API becoming "permanent" should not be acceptance into the tree, but rather the removal of an "experimental" tag (including a way of shutting off experimental APIs to make sure you're not depending on them). Sort of like CONFIG_EXPERIMENTAL, except actually used for its intended purpose (distributions should have it *off* by default), and preferably managed at runtime. Sort of like drivers/staging, except for APIs rather than drivers. Changes at that point should require more justification than before merging, but would not have the strict compatibility requirement that non-experimental APIs have. This would make collaboration and testing easier on APIs that aren't ready to be permanent.
>> This tag does exist. It's called "not in Linus' tree" :).
>
> Which makes it a pain for multiple people to work on a new feature, especially when it spans components such as KVM and QEMU, and means that it gets less testing before the point of no return.
I agree, but we're not going to solve this one now :). In fact, it'd be even harder to solve (with questionable results) than the actual ioctl in question.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 22:30 ` Alexander Graf
@ 2013-01-10 10:17 ` Peter Maydell
2013-01-10 11:06 ` Alexander Graf
2013-01-10 11:53 ` Marc Zyngier
0 siblings, 2 replies; 79+ messages in thread
From: Peter Maydell @ 2013-01-10 10:17 UTC (permalink / raw)
To: linux-arm-kernel
On 9 January 2013 22:30, Alexander Graf <agraf@suse.de> wrote:
> In fact, in this particular case one could merge all of the patches except for the
> particular ioctl implementation and just give the respective addresses default
> values until there is an API to set them, similar to how we did things with PVR
> in the beginning.
That's what we started with and we got review comments saying "yuck,
don't hard code this"...
-- PMM
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 10:17 ` Peter Maydell
@ 2013-01-10 11:06 ` Alexander Graf
2013-01-10 11:53 ` Marc Zyngier
1 sibling, 0 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-10 11:06 UTC (permalink / raw)
To: linux-arm-kernel
Am 10.01.2013 um 11:17 schrieb Peter Maydell <peter.maydell@linaro.org>:
> On 9 January 2013 22:30, Alexander Graf <agraf@suse.de> wrote:
>> In fact, in this particular case one could merge all of the patches except for the
>> particular ioctl implementation and just give the respective addresses default
>> values until there is an API to set them, similar to how we did things with PVR
>> in the beginning.
>
> That's what we started with and we got review comments saying "yuck,
> don't hard code this"...
And you shouldn't in the long run. But as an intermediate step, it's the best solution IMHO.
Either way, let's just finally get something in that doesn't clutter the global namespace :).
Alex
>
> -- PMM
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 10:17 ` Peter Maydell
2013-01-10 11:06 ` Alexander Graf
@ 2013-01-10 11:53 ` Marc Zyngier
2013-01-10 11:57 ` Alexander Graf
1 sibling, 1 reply; 79+ messages in thread
From: Marc Zyngier @ 2013-01-10 11:53 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, 10 Jan 2013 10:17:09 +0000, Peter Maydell
<peter.maydell@linaro.org> wrote:
> On 9 January 2013 22:30, Alexander Graf <agraf@suse.de> wrote:
>> In fact, in this particular case one could merge all of the patches
>> except for the
>> particular ioctl implementation and just give the respective addresses
>> default
>> values until there is an API to set them, similar to how we did things
>> with PVR
>> in the beginning.
>
> That's what we started with and we got review comments saying "yuck,
> don't hard code this"...
Even worse: hard-coding things breaks kvmtool (we do not emulate a
vexpress and have a rather different memory map) and arm64 (the
requirements on GIC mappings are different).
So hard coding addresses is already unacceptable.
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 11:53 ` Marc Zyngier
@ 2013-01-10 11:57 ` Alexander Graf
0 siblings, 0 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-10 11:57 UTC (permalink / raw)
To: linux-arm-kernel
On 10.01.2013, at 12:53, Marc Zyngier wrote:
> On Thu, 10 Jan 2013 10:17:09 +0000, Peter Maydell
> <peter.maydell@linaro.org> wrote:
>> On 9 January 2013 22:30, Alexander Graf <agraf@suse.de> wrote:
>>> In fact, in this particular case one could merge all of the patches
>>> except for the
>>> particular ioctl implementation and just give the respective addresses
>>> default
>>> values until there is an API to set them, similar to how we did things
>>> with PVR
>>> in the beginning.
>>
>> That's what we started with and we got review comments saying "yuck,
>> don't hard code this"...
>
> Even worse: hard-coding things breaks kvmtool (we do not emulate a
> vexpress and have a rather different memory map) and arm64 (the
> requirements on GIC mappings are different).
>
> So hard coding addresses is already unacceptable.
I see. The current approach with an intermediate ARM specific ioctl is certainly good with me then. And who knows - with a bit of luck we might be able to get a unified ioctl model for this before the code hits Linus' tree (read: APIs are really stable).
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 21:37 ` Alexander Graf
2013-01-09 22:10 ` Scott Wood
@ 2013-01-10 22:28 ` Marcelo Tosatti
2013-01-10 22:40 ` Scott Wood
2013-01-11 19:17 ` Alexander Graf
1 sibling, 2 replies; 79+ messages in thread
From: Marcelo Tosatti @ 2013-01-10 22:28 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 09, 2013 at 10:37:20PM +0100, Alexander Graf wrote:
>
> >
> >> >> We can start to introduce (and fix ARM) with a generic ioctl in the MPIC patches then.
> >> >
> >> > The ioctl is already generic, except for its name.
> >> It's making a few wrong assumptions:
> >> * maximum size of value is u64
> >
> > This is tolerable IMHO.
> >
> >> * combining device id (variable) with addr type id (const) into a single field. It could just be split into multiple fields
> >
> > I agree, but that could be lived with as well.
> >
> > I get that there's a tradeoff between getting something in now, versus waiting until the API is more refined. Tagging it with a particular ISA seems like an odd way of saying "soon to be deprecated", though. What happens if we're still squabbling over the perfect replacement API when we're trying to push PPC MPIC stuff in?
As mentioned, i fail to see the benefit in sharing 0.0x% of the code (the
interface), while the remaining code is not shared.
> Then we're the ones who have to come up with a good interface.
Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
> >
> > Perhaps the threshold for an API becoming "permanent" should not be acceptance into the tree, but rather the removal of an "experimental" tag (including a way of shutting off experimental APIs to make sure you're not depending on them). Sort of like CONFIG_EXPERIMENTAL, except actually used for its intended purpose (distributions should have it *off* by default), and preferably managed at runtime. Sort of like drivers/staging, except for APIs rather than drivers. Changes at that point should require more justification than before merging, but would not have the strict compatibility requirement that non-experimental APIs have. This would make collaboration and testing easier on APIs that aren't ready to be permanent.
>
> This tag does exist. It's called "not in Linus' tree" :).
>
> >
> >> * the id is 100% architecture specific. It shouldn't be. At least the "device id" field should be generic.
> >
> > That's a documentation issue that could be changed to have all architectures adopt what is currently specified for ARM, without breaking compatibility.
>
> Depends on how we want to design the final layout. I don't have the impression that we've reached final conclusion on this interface. That's all I'm saying.
>
> >
> >> I'm not sure if we can come up with more problems in that API when staring at it a bit longer and/or we would actually start using it for more things. So for the sake of not holding up the ARM code, I'm perfectly fine to clutter ARM's ioctl handling code with an ioctl that is already deprecated at its introduction, as long as we don't hold everything else up meanwhile.
> >
> > I'm not in a position to block it, and if I were I presumably would have seen this in time for feedback to matter. That's different from actually being happy. :-)
>
> For the ARM specific ioctl it has my ack. I'd say let's go with that and start to work on a good interface ASAP. But we need an ok from Marcelo or Gleb too.
>
>
> Alex
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 22:28 ` Marcelo Tosatti
@ 2013-01-10 22:40 ` Scott Wood
2013-01-11 0:35 ` Marcelo Tosatti
2013-01-11 19:17 ` Alexander Graf
1 sibling, 1 reply; 79+ messages in thread
From: Scott Wood @ 2013-01-10 22:40 UTC (permalink / raw)
To: linux-arm-kernel
On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
> On Wed, Jan 09, 2013 at 10:37:20PM +0100, Alexander Graf wrote:
> >
> > >
> > >> >> We can start to introduce (and fix ARM) with a generic ioctl
> in the MPIC patches then.
> > >> >
> > >> > The ioctl is already generic, except for its name.
> > >> It's making a few wrong assumptions:
> > >> * maximum size of value is u64
> > >
> > > This is tolerable IMHO.
> > >
> > >> * combining device id (variable) with addr type id (const) into
> a single field. It could just be split into multiple fields
> > >
> > > I agree, but that could be lived with as well.
> > >
> > > I get that there's a tradeoff between getting something in now,
> versus waiting until the API is more refined. Tagging it with a
> particular ISA seems like an odd way of saying "soon to be
> deprecated", though. What happens if we're still squabbling over the
> perfect replacement API when we're trying to push PPC MPIC stuff in?
>
> As mentioned, i fail to see the benefit in sharing 0.0x% of the code
> (the
> interface), while the remaining code is not shared.
Pointlessly making things architecture-specific just makes more work
for people who later need the functionality on another architecture.
It might not be much in this case (unless a particular device ends up
being used on multiple architectures), but the principle still
applies. It's also more work for tools like strace, and could get in
the way of the userspace caller using common code.
> > Then we're the ones who have to come up with a good interface.
>
> Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
Besides the above, and my original complaint that it shouldn't be
specific to addresses?
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 22:40 ` Scott Wood
@ 2013-01-11 0:35 ` Marcelo Tosatti
2013-01-11 1:10 ` Scott Wood
0 siblings, 1 reply; 79+ messages in thread
From: Marcelo Tosatti @ 2013-01-11 0:35 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Jan 10, 2013 at 04:40:12PM -0600, Scott Wood wrote:
> On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
> >On Wed, Jan 09, 2013 at 10:37:20PM +0100, Alexander Graf wrote:
> >>
> >> >
> >> >> >> We can start to introduce (and fix ARM) with a generic
> >ioctl in the MPIC patches then.
> >> >> >
> >> >> > The ioctl is already generic, except for its name.
> >> >> It's making a few wrong assumptions:
> >> >> * maximum size of value is u64
> >> >
> >> > This is tolerable IMHO.
> >> >
> >> >> * combining device id (variable) with addr type id (const)
> >into a single field. It could just be split into multiple fields
> >> >
> >> > I agree, but that could be lived with as well.
> >> >
> >> > I get that there's a tradeoff between getting something in
> >now, versus waiting until the API is more refined. Tagging it
> >with a particular ISA seems like an odd way of saying "soon to be
> >deprecated", though. What happens if we're still squabbling over
> >the perfect replacement API when we're trying to push PPC MPIC
> >stuff in?
> >
> >As mentioned, i fail to see the benefit in sharing 0.0x% of the
> >code (the
> >interface), while the remaining code is not shared.
>
> Pointlessly making things architecture-specific just makes more work
> for people who later need the functionality on another architecture.
> It might not be much in this case (unless a particular device ends
> up being used on multiple architectures), but the principle still
> applies. It's also more work for tools like strace, and could get
> in the way of the userspace caller using common code.
>
> >> Then we're the ones who have to come up with a good interface.
> >
> >Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
>
> Besides the above, and my original complaint that it shouldn't be
> specific to addresses?
>
> -Scott
I did not really grasp that ('shouldnt be specific to addresses'), but
anyway.
OK, can you write down your proposed improvements to the interface?
In case you have something ready, otherwise there is time pressure
to merge the ARM port.
That is, if you have interest/energy to spend in a possibly reusable
interface, as long as that does not delay integration of the ARM code,
i don't think the ARM people will mind that.
As mentioned in the thread, they (ARM) require configurable device
address.
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 0:35 ` Marcelo Tosatti
@ 2013-01-11 1:10 ` Scott Wood
2013-01-11 7:26 ` Christoffer Dall
2013-01-11 15:42 ` Alexander Graf
0 siblings, 2 replies; 79+ messages in thread
From: Scott Wood @ 2013-01-11 1:10 UTC (permalink / raw)
To: linux-arm-kernel
On 01/10/2013 06:35:02 PM, Marcelo Tosatti wrote:
> On Thu, Jan 10, 2013 at 04:40:12PM -0600, Scott Wood wrote:
> > On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
> > >Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to
> that?
> >
> > Besides the above, and my original complaint that it shouldn't be
> > specific to addresses?
> >
> > -Scott
>
> I did not really grasp that ('shouldnt be specific to addresses'), but
> anyway.
A device may have other configuration parameters that need to be set,
besides addresses. PPC MPIC will require information about the vendor
and version, for example.
> OK, can you write down your proposed improvements to the interface?
> In case you have something ready, otherwise there is time pressure
> to merge the ARM port.
My original request was just to change the name to something like
KVM_SET_DEVICE_CONFIG or KVM_SET_DEVICE_ATTR, and not make the id
encoding architecture-specific (preferably, separate into a "device id"
field and an "attribute id" field rather than using bitfields). Actual
values for device id could be architecture-specific (or there could be a
global enumeration), and attribute id values would be device-specific.
Alex suggested that an ideal interface might accept values larger than
64
bits, though I think it's good enough -- there are currently no proposed
uses that need more than 64 bits for a single attribute (unlike
ONE_REG),
and if it is needed, such configuration could be split up between
multiple attributes, or the attribute could specify that "value" be a
userspace pointer to the actual data (as with ONE_REG).
Here's a writeup (the ARM details would go under ARM/vGIC-specific
documentation):
4.80 KVM_SET_DEVICE_ATTR
Capability: KVM_CAP_SET_DEVICE_ATTR
Type: vm ioctl
Parameters: struct kvm_device_attr (in)
Returns: 0 on success, -1 on error
Errors:
ENODEV: The device id is unknown
ENXIO: Device not supported on current system
Other errors may be returned by specific devices and attributes.
struct kvm_device_attr {
__u32 device;
__u32 attr;
__u64 value;
};
Specify an attribute of a device emulated or directly exposed by the
kernel, which the host kernel needs to know about. The device field is
an
architecture-specific identifier for a specific device. The attr field
is a device-specific identifier for a specific attribute. Individual
attributes may have particular requirements for when they can and cannot
be set.
> That is, if you have interest/energy to spend in a possibly reusable
> interface, as long as that does not delay integration of the ARM code,
> i don't think the ARM people will mind that.
The impression I've been given is that just about any change will delay
the integration at this point. If that's the case, and everyone's OK
with having an interface that is deprecated on arrival, then fine.
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 1:10 ` Scott Wood
@ 2013-01-11 7:26 ` Christoffer Dall
2013-01-11 18:39 ` Marcelo Tosatti
2013-01-11 15:42 ` Alexander Graf
1 sibling, 1 reply; 79+ messages in thread
From: Christoffer Dall @ 2013-01-11 7:26 UTC (permalink / raw)
To: linux-arm-kernel
On 10/01/2013, at 20.10, Scott Wood <scottwood@freescale.com> wrote:
> On 01/10/2013 06:35:02 PM, Marcelo Tosatti wrote:
>> On Thu, Jan 10, 2013 at 04:40:12PM -0600, Scott Wood wrote:
>> > On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
>> > >Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
>> >
>> > Besides the above, and my original complaint that it shouldn't be
>> > specific to addresses?
>> >
>> > -Scott
>> I did not really grasp that ('shouldnt be specific to addresses'), but
>> anyway.
>
> A device may have other configuration parameters that need to be set,
> besides addresses. PPC MPIC will require information about the vendor
> and version, for example.
>
>> OK, can you write down your proposed improvements to the interface?
>> In case you have something ready, otherwise there is time pressure
>> to merge the ARM port.
>
> My original request was just to change the name to something like
> KVM_SET_DEVICE_CONFIG or KVM_SET_DEVICE_ATTR, and not make the id
> encoding architecture-specific (preferably, separate into a "device id"
> field and an "attribute id" field rather than using bitfields). Actual
> values for device id could be architecture-specific (or there could be a
> global enumeration), and attribute id values would be device-specific.
>
> Alex suggested that an ideal interface might accept values larger than 64
> bits, though I think it's good enough -- there are currently no proposed
> uses that need more than 64 bits for a single attribute (unlike ONE_REG),
> and if it is needed, such configuration could be split up between
> multiple attributes, or the attribute could specify that "value" be a
> userspace pointer to the actual data (as with ONE_REG).
>
> Here's a writeup (the ARM details would go under ARM/vGIC-specific
> documentation):
>
> 4.80 KVM_SET_DEVICE_ATTR
>
> Capability: KVM_CAP_SET_DEVICE_ATTR
> Type: vm ioctl
> Parameters: struct kvm_device_attr (in)
> Returns: 0 on success, -1 on error
> Errors:
> ENODEV: The device id is unknown
> ENXIO: Device not supported on current system
> Other errors may be returned by specific devices and attributes.
>
> struct kvm_device_attr {
> __u32 device;
> __u32 attr;
> __u64 value;
> };
>
> Specify an attribute of a device emulated or directly exposed by the
> kernel, which the host kernel needs to know about. The device field is an
> architecture-specific identifier for a specific device. The attr field
> is a device-specific identifier for a specific attribute. Individual
> attributes may have particular requirements for when they can and cannot
> be set.
>
>> That is, if you have interest/energy to spend in a possibly reusable
>> interface, as long as that does not delay integration of the ARM code,
>> i don't think the ARM people will mind that.
>
> The impression I've been given is that just about any change will delay
> the integration at this point. If that's the case, and everyone's OK
> with having an interface that is deprecated on arrival, then fine.
That is not entirely the case, but there wasn't event agreement on this revised API, and we didn't want to wait for weeks until a decision was made. Alex suggested we use a DEV_REG API similar to the ONE_REG API, but I am quite strongly against having such an API for this specific purpose as it is too semantically distant to what we do on ARM. (Having a DEV_REG API for other purposes might be fine though).
So I have no problem with your suggestion, although we could consider having a size and addr field in the struct instead and be slightly more extensible. But I don't feel strongly about it.
If we can agree on Scott's suggestion or with my modification (Alex, Gleb, Marcelo ?) then I'll change the KVM/ARM patches to use this and resend them right away. But we have to agree now!
If not, I really think we should keep things as they are now, as we're really discussing idealistic scenarios here, and the ARM patches have been out of tree for long enough. As Marcelo pointed out, the benefits of the perfect API are really minimal.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 7:26 ` Christoffer Dall
@ 2013-01-11 18:39 ` Marcelo Tosatti
2013-01-11 19:11 ` Alexander Graf
0 siblings, 1 reply; 79+ messages in thread
From: Marcelo Tosatti @ 2013-01-11 18:39 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Jan 11, 2013 at 02:26:51AM -0500, Christoffer Dall wrote:
> On 10/01/2013, at 20.10, Scott Wood <scottwood@freescale.com> wrote:
>
> > On 01/10/2013 06:35:02 PM, Marcelo Tosatti wrote:
> >> On Thu, Jan 10, 2013 at 04:40:12PM -0600, Scott Wood wrote:
> >> > On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
> >> > >Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
> >> >
> >> > Besides the above, and my original complaint that it shouldn't be
> >> > specific to addresses?
> >> >
> >> > -Scott
> >> I did not really grasp that ('shouldnt be specific to addresses'), but
> >> anyway.
> >
> > A device may have other configuration parameters that need to be set,
> > besides addresses. PPC MPIC will require information about the vendor
> > and version, for example.
> >
> >> OK, can you write down your proposed improvements to the interface?
> >> In case you have something ready, otherwise there is time pressure
> >> to merge the ARM port.
> >
> > My original request was just to change the name to something like
> > KVM_SET_DEVICE_CONFIG or KVM_SET_DEVICE_ATTR, and not make the id
> > encoding architecture-specific (preferably, separate into a "device id"
> > field and an "attribute id" field rather than using bitfields). Actual
> > values for device id could be architecture-specific (or there could be a
> > global enumeration), and attribute id values would be device-specific.
> >
> > Alex suggested that an ideal interface might accept values larger than 64
> > bits, though I think it's good enough -- there are currently no proposed
> > uses that need more than 64 bits for a single attribute (unlike ONE_REG),
> > and if it is needed, such configuration could be split up between
> > multiple attributes, or the attribute could specify that "value" be a
> > userspace pointer to the actual data (as with ONE_REG).
> >
> > Here's a writeup (the ARM details would go under ARM/vGIC-specific
> > documentation):
> >
> > 4.80 KVM_SET_DEVICE_ATTR
> >
> > Capability: KVM_CAP_SET_DEVICE_ATTR
> > Type: vm ioctl
> > Parameters: struct kvm_device_attr (in)
> > Returns: 0 on success, -1 on error
> > Errors:
> > ENODEV: The device id is unknown
> > ENXIO: Device not supported on current system
> > Other errors may be returned by specific devices and attributes.
> >
> > struct kvm_device_attr {
> > __u32 device;
> > __u32 attr;
> > __u64 value;
> > };
> >
> > Specify an attribute of a device emulated or directly exposed by the
> > kernel, which the host kernel needs to know about. The device field is an
> > architecture-specific identifier for a specific device. The attr field
> > is a device-specific identifier for a specific attribute. Individual
> > attributes may have particular requirements for when they can and cannot
> > be set.
> >
> >> That is, if you have interest/energy to spend in a possibly reusable
> >> interface, as long as that does not delay integration of the ARM code,
> >> i don't think the ARM people will mind that.
> >
> > The impression I've been given is that just about any change will delay
> > the integration at this point. If that's the case, and everyone's OK
> > with having an interface that is deprecated on arrival, then fine.
>
>
> That is not entirely the case, but there wasn't event agreement on this revised API, and we didn't want to wait for weeks until a decision was made. Alex suggested we use a DEV_REG API similar to the ONE_REG API, but I am quite strongly against having such an API for this specific purpose as it is too semantically distant to what we do on ARM. (Having a DEV_REG API for other purposes might be fine though).
>
> So I have no problem with your suggestion, although we could consider having a size and addr field in the struct instead and be slightly more extensible. But I don't feel strongly about it.
>
> If we can agree on Scott's suggestion or with my modification (Alex, Gleb, Marcelo ?) then I'll change the KVM/ARM patches to use this and resend them right away. But we have to agree now!
>
> If not, I really think we should keep things as they are now, as we're really discussing idealistic scenarios here, and the ARM patches have been out of tree for long enough. As Marcelo pointed out, the benefits of the perfect API are really minimal.
>
> -Christoffer--
Can you make KVM_SET_ARM_DEVICE address extensible?
Add some reserved space and a flags field.
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 18:39 ` Marcelo Tosatti
@ 2013-01-11 19:11 ` Alexander Graf
2013-01-11 19:18 ` Marcelo Tosatti
0 siblings, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-11 19:11 UTC (permalink / raw)
To: linux-arm-kernel
On 11.01.2013, at 19:39, Marcelo Tosatti wrote:
> On Fri, Jan 11, 2013 at 02:26:51AM -0500, Christoffer Dall wrote:
>> On 10/01/2013, at 20.10, Scott Wood <scottwood@freescale.com> wrote:
>>
>>> On 01/10/2013 06:35:02 PM, Marcelo Tosatti wrote:
>>>> On Thu, Jan 10, 2013 at 04:40:12PM -0600, Scott Wood wrote:
>>>>> On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
>>>>>> Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
>>>>>
>>>>> Besides the above, and my original complaint that it shouldn't be
>>>>> specific to addresses?
>>>>>
>>>>> -Scott
>>>> I did not really grasp that ('shouldnt be specific to addresses'), but
>>>> anyway.
>>>
>>> A device may have other configuration parameters that need to be set,
>>> besides addresses. PPC MPIC will require information about the vendor
>>> and version, for example.
>>>
>>>> OK, can you write down your proposed improvements to the interface?
>>>> In case you have something ready, otherwise there is time pressure
>>>> to merge the ARM port.
>>>
>>> My original request was just to change the name to something like
>>> KVM_SET_DEVICE_CONFIG or KVM_SET_DEVICE_ATTR, and not make the id
>>> encoding architecture-specific (preferably, separate into a "device id"
>>> field and an "attribute id" field rather than using bitfields). Actual
>>> values for device id could be architecture-specific (or there could be a
>>> global enumeration), and attribute id values would be device-specific.
>>>
>>> Alex suggested that an ideal interface might accept values larger than 64
>>> bits, though I think it's good enough -- there are currently no proposed
>>> uses that need more than 64 bits for a single attribute (unlike ONE_REG),
>>> and if it is needed, such configuration could be split up between
>>> multiple attributes, or the attribute could specify that "value" be a
>>> userspace pointer to the actual data (as with ONE_REG).
>>>
>>> Here's a writeup (the ARM details would go under ARM/vGIC-specific
>>> documentation):
>>>
>>> 4.80 KVM_SET_DEVICE_ATTR
>>>
>>> Capability: KVM_CAP_SET_DEVICE_ATTR
>>> Type: vm ioctl
>>> Parameters: struct kvm_device_attr (in)
>>> Returns: 0 on success, -1 on error
>>> Errors:
>>> ENODEV: The device id is unknown
>>> ENXIO: Device not supported on current system
>>> Other errors may be returned by specific devices and attributes.
>>>
>>> struct kvm_device_attr {
>>> __u32 device;
>>> __u32 attr;
>>> __u64 value;
>>> };
>>>
>>> Specify an attribute of a device emulated or directly exposed by the
>>> kernel, which the host kernel needs to know about. The device field is an
>>> architecture-specific identifier for a specific device. The attr field
>>> is a device-specific identifier for a specific attribute. Individual
>>> attributes may have particular requirements for when they can and cannot
>>> be set.
>>>
>>>> That is, if you have interest/energy to spend in a possibly reusable
>>>> interface, as long as that does not delay integration of the ARM code,
>>>> i don't think the ARM people will mind that.
>>>
>>> The impression I've been given is that just about any change will delay
>>> the integration at this point. If that's the case, and everyone's OK
>>> with having an interface that is deprecated on arrival, then fine.
>>
>>
>> That is not entirely the case, but there wasn't event agreement on this revised API, and we didn't want to wait for weeks until a decision was made. Alex suggested we use a DEV_REG API similar to the ONE_REG API, but I am quite strongly against having such an API for this specific purpose as it is too semantically distant to what we do on ARM. (Having a DEV_REG API for other purposes might be fine though).
>>
>> So I have no problem with your suggestion, although we could consider having a size and addr field in the struct instead and be slightly more extensible. But I don't feel strongly about it.
>>
>> If we can agree on Scott's suggestion or with my modification (Alex, Gleb, Marcelo ?) then I'll change the KVM/ARM patches to use this and resend them right away. But we have to agree now!
>>
>> If not, I really think we should keep things as they are now, as we're really discussing idealistic scenarios here, and the ARM patches have been out of tree for long enough. As Marcelo pointed out, the benefits of the perfect API are really minimal.
>>
>> -Christoffer--
>
> Can you make KVM_SET_ARM_DEVICE address extensible?
> Add some reserved space and a flags field.
Can't we do this for the new ioctl that we all would agree on rather than the interim one that's only a short term solution for a greater problem?
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 19:11 ` Alexander Graf
@ 2013-01-11 19:18 ` Marcelo Tosatti
2013-01-11 19:33 ` Christoffer Dall
0 siblings, 1 reply; 79+ messages in thread
From: Marcelo Tosatti @ 2013-01-11 19:18 UTC (permalink / raw)
To: linux-arm-kernel
> > Can you make KVM_SET_ARM_DEVICE address extensible?
> > Add some reserved space and a flags field.
>
> Can't we do this for the new ioctl that we all would agree on rather than the interim one that's only a short term solution for a greater problem?
>
>
> Alex
OK.
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 19:18 ` Marcelo Tosatti
@ 2013-01-11 19:33 ` Christoffer Dall
0 siblings, 0 replies; 79+ messages in thread
From: Christoffer Dall @ 2013-01-11 19:33 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Jan 11, 2013 at 2:18 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
>> > Can you make KVM_SET_ARM_DEVICE address extensible?
>> > Add some reserved space and a flags field.
>>
>> Can't we do this for the new ioctl that we all would agree on rather than the interim one that's only a short term solution for a greater problem?
>>
>>
>> Alex
>
> OK.
>
Good. It's decided - the KVM/ARM patches remain in their current state
with an ARM specific ioctl, and will transition to something generic
post merge.
-Christoffer
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 1:10 ` Scott Wood
2013-01-11 7:26 ` Christoffer Dall
@ 2013-01-11 15:42 ` Alexander Graf
2013-01-11 20:11 ` Scott Wood
1 sibling, 1 reply; 79+ messages in thread
From: Alexander Graf @ 2013-01-11 15:42 UTC (permalink / raw)
To: linux-arm-kernel
On 11.01.2013, at 02:10, Scott Wood wrote:
> On 01/10/2013 06:35:02 PM, Marcelo Tosatti wrote:
>> On Thu, Jan 10, 2013 at 04:40:12PM -0600, Scott Wood wrote:
>> > On 01/10/2013 04:28:01 PM, Marcelo Tosatti wrote:
>> > >Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
>> >
>> > Besides the above, and my original complaint that it shouldn't be
>> > specific to addresses?
>> >
>> > -Scott
>> I did not really grasp that ('shouldnt be specific to addresses'), but
>> anyway.
>
> A device may have other configuration parameters that need to be set,
> besides addresses. PPC MPIC will require information about the vendor
> and version, for example.
>
>> OK, can you write down your proposed improvements to the interface?
>> In case you have something ready, otherwise there is time pressure
>> to merge the ARM port.
>
> My original request was just to change the name to something like
> KVM_SET_DEVICE_CONFIG or KVM_SET_DEVICE_ATTR, and not make the id
> encoding architecture-specific (preferably, separate into a "device id"
> field and an "attribute id" field rather than using bitfields). Actual
> values for device id could be architecture-specific (or there could be a
> global enumeration), and attribute id values would be device-specific.
>
> Alex suggested that an ideal interface might accept values larger than 64
> bits, though I think it's good enough -- there are currently no proposed
> uses that need more than 64 bits for a single attribute (unlike ONE_REG),
> and if it is needed, such configuration could be split up between
> multiple attributes, or the attribute could specify that "value" be a
> userspace pointer to the actual data (as with ONE_REG).
>
> Here's a writeup (the ARM details would go under ARM/vGIC-specific
> documentation):
>
> 4.80 KVM_SET_DEVICE_ATTR
>
> Capability: KVM_CAP_SET_DEVICE_ATTR
> Type: vm ioctl
> Parameters: struct kvm_device_attr (in)
> Returns: 0 on success, -1 on error
> Errors:
> ENODEV: The device id is unknown
> ENXIO: Device not supported on current system
> Other errors may be returned by specific devices and attributes.
>
> struct kvm_device_attr {
> __u32 device;
This needs some semantic specification. Is device a constant value? Is it the return value of CREATE_IRQCHIP?
> __u32 attr;
> __u64 value;
> };
>
> Specify an attribute of a device emulated or directly exposed by the
> kernel, which the host kernel needs to know about. The device field is an
> architecture-specific identifier for a specific device. The attr field
> is a device-specific identifier for a specific attribute. Individual
> attributes may have particular requirements for when they can and cannot
> be set.
>
>> That is, if you have interest/energy to spend in a possibly reusable
>> interface, as long as that does not delay integration of the ARM code,
>> i don't think the ARM people will mind that.
>
> The impression I've been given is that just about any change will delay
> the integration at this point. If that's the case, and everyone's OK
> with having an interface that is deprecated on arrival, then fine.
To me it's perfectly fine to merge the patches with the arm specific interface and then push an interface like the above in in the next merge window.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 15:42 ` Alexander Graf
@ 2013-01-11 20:11 ` Scott Wood
2013-01-11 20:26 ` Alexander Graf
0 siblings, 1 reply; 79+ messages in thread
From: Scott Wood @ 2013-01-11 20:11 UTC (permalink / raw)
To: linux-arm-kernel
On 01/11/2013 09:42:55 AM, Alexander Graf wrote:
>
> On 11.01.2013, at 02:10, Scott Wood wrote:
>
> > struct kvm_device_attr {
> > __u32 device;
>
> This needs some semantic specification. Is device a constant value?
> Is it the return value of CREATE_IRQCHIP?
As proposed, it's up to the architecture to provide that
specification. In theory this could be used for things other than IRQ
chips. We could still say that device creation functions return a
valid device ID (if the device has any attributes), as well as have
other architecture-specific ways of describing device IDs (static
enumeration). Or we could have non-architecture-specific static
enumeration. Or just require that all devices be explicitly created by
something that returns the ID.
Do you have a preferred approach?
-Scott
^ permalink raw reply [flat|nested] 79+ messages in thread* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-11 20:11 ` Scott Wood
@ 2013-01-11 20:26 ` Alexander Graf
0 siblings, 0 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-11 20:26 UTC (permalink / raw)
To: linux-arm-kernel
Am 11.01.2013 um 21:11 schrieb Scott Wood <scottwood@freescale.com>:
> On 01/11/2013 09:42:55 AM, Alexander Graf wrote:
>> On 11.01.2013, at 02:10, Scott Wood wrote:
>> > struct kvm_device_attr {
>> > __u32 device;
>> This needs some semantic specification. Is device a constant value? Is it the return value of CREATE_IRQCHIP?
>
> As proposed, it's up to the architecture to provide that specification. In theory this could be used for things other than IRQ chips. We could still say that device creation functions return a valid device ID (if the device has any attributes), as well as have other architecture-specific ways of describing device IDs (static enumeration). Or we could have non-architecture-specific static enumeration. Or just require that all devices be explicitly created by something that returns the ID.
>
> Do you have a preferred approach?
I am a fan of dynamically generated ids that get returned from the create functions (like open() and an fd).
I'm not sure if that would always work here though. It would mean that we explicitly have to create per-cpu interrupt controllers. Or append their state onto vcpu state.
Alex
>
> -Scott
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-10 22:28 ` Marcelo Tosatti
2013-01-10 22:40 ` Scott Wood
@ 2013-01-11 19:17 ` Alexander Graf
1 sibling, 0 replies; 79+ messages in thread
From: Alexander Graf @ 2013-01-11 19:17 UTC (permalink / raw)
To: linux-arm-kernel
On 10.01.2013, at 23:28, Marcelo Tosatti wrote:
> On Wed, Jan 09, 2013 at 10:37:20PM +0100, Alexander Graf wrote:
>>
>>>
>>>>>> We can start to introduce (and fix ARM) with a generic ioctl in the MPIC patches then.
>>>>>
>>>>> The ioctl is already generic, except for its name.
>>>> It's making a few wrong assumptions:
>>>> * maximum size of value is u64
>>>
>>> This is tolerable IMHO.
>>>
>>>> * combining device id (variable) with addr type id (const) into a single field. It could just be split into multiple fields
>>>
>>> I agree, but that could be lived with as well.
>>>
>>> I get that there's a tradeoff between getting something in now, versus waiting until the API is more refined. Tagging it with a particular ISA seems like an odd way of saying "soon to be deprecated", though. What happens if we're still squabbling over the perfect replacement API when we're trying to push PPC MPIC stuff in?
>
> As mentioned, i fail to see the benefit in sharing 0.0x% of the code (the
> interface), while the remaining code is not shared.
Important code sharing happens in user space.
>
>> Then we're the ones who have to come up with a good interface.
>
> Or just have KVM_SET_PPC_DEVICE_ADDRESS. Is there a downside to that?
Yes. Harder to read code and a lack of commonality. Also it wouldn't cut it for us, as we need to set more than just addresses. But most of all I really hate to take a generic problem and solve it only for myself.
Btw, x86 has the same issue too, it just only somehow got away with not caring too much about it. The LAPIC has a version register that Mac OS X guests read. If the version is too old, it doesn't boot. So some time ago we just changed the version register.
The real, backwards compatible change however would've been to have user space tell us which version to emulate and set it accordingly. The way we did it basically broke backwards compatibility. We only got away with it because the register is rarely read.
Alex
^ permalink raw reply [flat|nested] 79+ messages in thread
* [kvmarm] [PATCH v5.1 0/2] KVM: ARM: Rename KVM_SET_DEVICE_ADDRESS
2013-01-09 21:15 ` Scott Wood
2013-01-09 21:37 ` Alexander Graf
@ 2013-01-10 22:21 ` Marcelo Tosatti
1 sibling, 0 replies; 79+ messages in thread
From: Marcelo Tosatti @ 2013-01-10 22:21 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Jan 09, 2013 at 03:15:43PM -0600, Scott Wood wrote:
> On 01/09/2013 02:12:16 PM, Alexander Graf wrote:
> >
> >On 09.01.2013, at 20:50, Scott Wood wrote:
> >
> >> On 01/09/2013 10:48:47 AM, Alexander Graf wrote:
> >>> On 09.01.2013, at 17:26, Christoffer Dall wrote:
> >>> > Renames the KVM_SET_DEVICE_ADDRESS to KVM_ARM_SET_DEVICE_ADDR
> >>> > to make it obvious that this is ARM specific in lack of a
> >better generic
> >>> > interface.
> >>> >
> >>> > Once we agree on a better interface the KVM/ARM code can also
> >take
> >>> > advantage of that, but until then we don't want to hold up
> >the KVM/ARM
> >>> > patches.
> >>> Works for me. Scott, are you happy with this one too?
> >>
> >> Not really, given that it will stay around forever even after
> >something new is introduced.
> >
> >But only in ARM specific code.
>
> ...which I'll probably have to deal with when Freescale's
> virtualization-capable ARM chips come along. I don't see "it's only
> in that other architecture" as "it might as well not exist".
Doesnt it make sense to make it extensible by adding some reserved
space and a flags field? (to the ioctl) (independently of anything
else).
So that, say, supporting new ARM processors does not require adding
a new ioctl?
Having a single ioctl does not increase code sharing significantly
because large percentage of the code both in QEMU and the kernel are not
shared (well perhaps except the interface).
> >> If you're going to change the name, why not just change it to
> >KVM_SET_DEVICE_CONFIG? Can we change the name later if nothing
> >else changes (so it's still binary compatible)?
> >
> >Because that again implies that it's generic enough. And to reach
> >that conclusion will take more time than we should spend on this
> >now.
>
> If the conclusion later on is that it is good enough, can the name
> be changed then?
>
> >>> We can start to introduce (and fix ARM) with a generic ioctl in
> >the MPIC patches then.
> >>
> >> The ioctl is already generic, except for its name.
> >
> >It's making a few wrong assumptions:
> >
> > * maximum size of value is u64
>
> This is tolerable IMHO.
>
> > * combining device id (variable) with addr type id (const) into
> >a single field. It could just be split into multiple fields
>
> I agree, but that could be lived with as well.
>
> I get that there's a tradeoff between getting something in now,
> versus waiting until the API is more refined. Tagging it with a
> particular ISA seems like an odd way of saying "soon to be
> deprecated", though. What happens if we're still squabbling over
> the perfect replacement API when we're trying to push PPC MPIC stuff
> in?
>
> Perhaps the threshold for an API becoming "permanent" should not be
> acceptance into the tree, but rather the removal of an
> "experimental" tag (including a way of shutting off experimental
> APIs to make sure you're not depending on them). Sort of like
> CONFIG_EXPERIMENTAL, except actually used for its intended purpose
> (distributions should have it *off* by default), and preferably
> managed at runtime. Sort of like drivers/staging, except for APIs
> rather than drivers. Changes at that point should require more
> justification than before merging, but would not have the strict
> compatibility requirement that non-experimental APIs have. This
> would make collaboration and testing easier on APIs that aren't
> ready to be permanent.
>
> > * the id is 100% architecture specific. It shouldn't be. At
> >least the "device id" field should be generic.
>
> That's a documentation issue that could be changed to have all
> architectures adopt what is currently specified for ARM, without
> breaking compatibility.
>
> >I'm not sure if we can come up with more problems in that API when
> >staring at it a bit longer and/or we would actually start using it
> >for more things. So for the sake of not holding up the ARM code,
> >I'm perfectly fine to clutter ARM's ioctl handling code with an
> >ioctl that is already deprecated at its introduction, as long as
> >we don't hold everything else up meanwhile.
>
> I'm not in a position to block it, and if I were I presumably would
> have seen this in time for feedback to matter. That's different
> from actually being happy. :-)
>
> -Scott
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 79+ messages in thread