Re: [PATCH v5 03/14] KVM: ARM: Initial skeleton to compile KVM support

Gleb Natapov
2013-01-15T13:32:27Z

On Mon, Jan 14, 2013 at 05:17:31PM -0500, Christoffer Dall wrote:
> On Mon, Jan 14, 2013 at 1:49 PM, Gleb Natapov <gleb@redhat.com> wrote:
> > A couple of general question about ABI. If they were already answered
> > just refer me to the previous discussion.
> >
> > On Tue, Jan 08, 2013 at 01:38:55PM -0500, Christoffer Dall wrote:
> >> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> >> index a4df553..4237c27 100644
> >> --- a/Documentation/virtual/kvm/api.txt
> >> +++ b/Documentation/virtual/kvm/api.txt
> >> @@ -293,7 +293,7 @@ kvm_run' (see below).
> >> 4.11 KVM_GET_REGS
> >>
> >> Capability: basic
> >> -Architectures: all
> >> +Architectures: all except ARM
> >> Type: vcpu ioctl
> >> Parameters: struct kvm_regs (out)
> >> Returns: 0 on success, -1 on error
> >> @@ -314,7 +314,7 @@ struct kvm_regs {
> >> 4.12 KVM_SET_REGS
> >>
> >> Capability: basic
> >> -Architectures: all
> >> +Architectures: all except ARM
> >> Type: vcpu ioctl
> >> Parameters: struct kvm_regs (in)
> >> Returns: 0 on success, -1 on error
> >> @@ -600,7 +600,7 @@ struct kvm_fpu {
> >> 4.24 KVM_CREATE_IRQCHIP
> > Why KVM_GET_REGS/KVM_SET_REGS are not usable for arm?
> >
>
> We use the ONE_REG API instead and we don't want to support two
> separate APIs to user space.
>
I suppose fetching all registers is not anywhere on a fast path in
userspace :)

> >>
> >> Capability: KVM_CAP_IRQCHIP
> >> -Architectures: x86, ia64
> >> +Architectures: x86, ia64, ARM
> >> Type: vm ioctl
> >> Parameters: none
> >> Returns: 0 on success, -1 on error
> >> @@ -608,7 +608,8 @@ Returns: 0 on success, -1 on error
> >> Creates an interrupt controller model in the kernel. On x86, creates a virtual
> >> ioapic, a virtual PIC (two PICs, nested), and sets up future vcpus to have a
> >> local APIC. IRQ routing for GSIs 0-15 is set to both PIC and IOAPIC; GSI 16-23
> >> -only go to the IOAPIC. On ia64, a IOSAPIC is created.
> >> +only go to the IOAPIC. On ia64, a IOSAPIC is created. On ARM, a GIC is
> >> +created.
> >>
> >>
> >> 4.25 KVM_IRQ_LINE
> >> @@ -1775,6 +1776,14 @@ registers, find a list below:
> >> PPC | KVM_REG_PPC_VPA_DTL | 128
> >> PPC | KVM_REG_PPC_EPCR | 32
> >>
> >> +ARM registers are mapped using the lower 32 bits. The upper 16 of that
> >> +is the register group type, or coprocessor number:
> >> +
> >> +ARM core registers have the following id bit patterns:
> >> + 0x4002 0000 0010 <index into the kvm_regs struct:16>
> >> +
> >> +
> >> +
> >> 4.69 KVM_GET_ONE_REG
> >>
> >> Capability: KVM_CAP_ONE_REG
> >> @@ -2127,6 +2136,46 @@ written, then `n_invalid' invalid entries, invalidating any previously
> >> valid entries found.
> >>
> >>
> >> +4.77 KVM_ARM_VCPU_INIT
> >> +
> >> +Capability: basic
> >> +Architectures: arm
> >> +Type: vcpu ioctl
> >> +Parameters: struct struct kvm_vcpu_init (in)
> >> +Returns: 0 on success; -1 on error
> >> +Errors:
> >> + EINVAL: the target is unknown, or the combination of features is invalid.
> >> + ENOENT: a features bit specified is unknown.
> >> +
> >> +This tells KVM what type of CPU to present to the guest, and what
> >> +optional features it should have. This will cause a reset of the cpu
> >> +registers to their initial values. If this is not called, KVM_RUN will
> >> +return ENOEXEC for that vcpu.
> >> +
> > Can different vcpus of the same VM be of different type?
> >
>
> In the future yes. For example, if we ever want to virtualize a
> Big.Little system.
>
> >> +Note that because some registers reflect machine topology, all vcpus
> >> +should be created before this ioctl is invoked.
> > How cpu hot plug suppose to work?
> >
>
> Those CPUs would be added from the beginning, but not powered on, and
> would be powered on later on, I suppose. See
> https://lists.cs.columbia.edu/pipermail/kvmarm/2013-January/004617.html.
>
When we suggested similar "hot plug" for x86, people started screaming
how they suppose to know when they create a VM how much vcpus they will
need in the future. In short people who are asking for hot plug (on x86
at least) do not like such solution.

>
> >> +
> >> +
> >> +4.78 KVM_GET_REG_LIST
> >> +
> >> +Capability: basic
> >> +Architectures: arm
> >> +Type: vcpu ioctl
> >> +Parameters: struct kvm_reg_list (in/out)
> >> +Returns: 0 on success; -1 on error
> >> +Errors:
> >> + E2BIG: the reg index list is too big to fit in the array specified by
> >> + the user (the number required will be written into n).
> >> +
> >> +struct kvm_reg_list {
> >> + __u64 n; /* number of registers in reg[] */
> >> + __u64 reg[0];
> >> +};
> >> +
> >> +This ioctl returns the guest registers that are supported for the
> >> +KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
> >> +
> >> +
> > Doesn't userspace know what registers are supported by each CPU type?
> >
> It would know about core registers, but there is a huge space of
> co-processors, and we don't emulate all of them or support
> getting/setting all of them yet. Surely this is something that will
> change over time and we want user space to be able to discover the
> available registers for backwards compatibility, migration, etc.
>
> -Christoffer

--
Gleb.