From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Subject: Re: Enable more than 255 VCPU support without irq remapping function in the guest Date: Tue, 26 Apr 2016 18:17:11 +0200 Message-ID: <571F9487.5090009@siemens.com> References: <571F93CA.40200@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit To: "Lan, Tianyu" , pbonzini@redhat.com, kvm@vger.kernel.org, yang.zhang.wz@gmail.com, tglx@linutronix.de, gleb@redhat.com, mst@redhat.com, x86@kernel.org, =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Peter Xu Return-path: Received: from david.siemens.de ([192.35.17.14]:55222 "EHLO david.siemens.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752212AbcDZQR3 (ORCPT ); Tue, 26 Apr 2016 12:17:29 -0400 In-Reply-To: <571F93CA.40200@intel.com> Sender: kvm-owner@vger.kernel.org List-ID: On 2016-04-26 18:14, Lan, Tianyu wrote: > Hi All: > > Recently I am working on extending max vcpu to more than 256 on the both > KVM/Xen. For some HPC cases, it needs many vcpus. The job requires to > use X2APIC in the guest which supports 32-bit APIC id. Linux kernel > requires irq remapping function during enabling X2APIC when max APIC id > is more than 255(More detail please see try_to_enable_x2apic()). > > The irq remapping function helps to deliver irq to cpu 255~. IOAPIC just > supports 8-bit target APIC id field and only can deliver irq to > cpu 0~255. > > So far both KVM/Xen doesn't enable irq remapping function. If enable the > function, it seems a huge job which need to rework IO-APIC, local APIC, > MSI parts and add virtual VTD support in the KVM. > > Other quick way to enable more than 256 VCPUs is to eliminate the > dependency between irq remapping and X2APIC in the guest linux kernel. > So far I can boot the guest after removing the dependency. > The side effect I thought is that irq only can deliver to 0~255 vcpus > but 256 vcpus seem enough to balance irq requests in the guest. In the > most cases, there are fewer devices in the guest. > > I wonder whether it's feasible. There maybe some other side effects I > didn't think of. Very appreciate for your comments. Radim is working on the KVM side already, Peter is currently driving the VT-d interrupt emulation topic in QEMU. It's in reach, I would say. :) Jan PS: Please no PV mess, at least without good reasons. -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux