Re: Enable more than 255 VCPU support without irq remapping function in the guest

["Radim Krčmář" <rkrcmar@redhat.com>]

2016-04-26 18:17+0200, Jan Kiszka:
> On 2016-04-26 18:14, Lan, Tianyu wrote:
>> Hi All:
>> 
>> Recently I am working on extending max vcpu to more than 256 on the both
>> KVM/Xen. For some HPC cases, it needs many vcpus. The job requires to
>> use X2APIC in the guest which supports 32-bit APIC id. Linux kernel
>> requires irq remapping function during enabling X2APIC when max APIC id
>> is more than 255(More detail please see try_to_enable_x2apic()).

Our of curiosity, how many VCPUs are you aiming at?

>> The irq remapping function helps to deliver irq to cpu 255~. IOAPIC just
>> supports 8-bit target APIC id field and only can deliver irq to
>> cpu 0~255.
>> 
>> So far both KVM/Xen doesn't enable irq remapping function. If enable the
>> function, it seems a huge job which need to rework IO-APIC, local APIC,
>> MSI parts and add virtual VTD support in the KVM.
>> 
>> Other quick way to enable more than 256 VCPUs is to eliminate the
>> dependency between irq remapping and X2APIC in the guest linux kernel.
>> So far I can boot the guest after removing the dependency.
>> The side effect I thought is that irq only can deliver to 0~255 vcpus
>> but 256 vcpus seem enough to balance irq requests in the guest. In the
>> most cases, there are fewer devices in the guest.
>> 
>> I wonder whether it's feasible. There maybe some other side effects I
>> didn't think of. Very appreciate for your comments.
> 
> Radim is working on the KVM side already, Peter is currently driving the
> VT-d interrupt emulation topic in QEMU. It's in reach, I would say. :)

+ Igor extends QEMU to support more than 255 in internal structures and
ACPI.  What remains mostly untracked is Seabios/OVMF.

> PS: Please no PV mess, at least without good reasons.

Seconded.

(If we designed all related devices as virtware, then it would not be
 that bad, but slightly modifying and putting hardware drivers into
 situations that cannot happen in hardware, not even in the spec, and
 then juggling the KVM side to make them work, is a road to hell.)