From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gleb Natapov Subject: Re: [PATCH v4] enable x2APIC without interrupt remapping under KVM Date: Tue, 30 Jun 2009 10:54:57 +0300 Message-ID: <20090630075457.GH20289@redhat.com> References: <20090630064515.GG20289@redhat.com> <86802c440906300018p8c5156dy3e8d84b8c263797e@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: linux-kernel@vger.kernel.org, Suresh Siddha , Sheng Yang , "kvm@vger.kernel.org" , "avi@redhat.com" To: Yinghai Lu Return-path: Received: from mx2.redhat.com ([66.187.237.31]:60938 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750755AbZF3Hy6 convert rfc822-to-8bit (ORCPT ); Tue, 30 Jun 2009 03:54:58 -0400 Content-Disposition: inline In-Reply-To: <86802c440906300018p8c5156dy3e8d84b8c263797e@mail.gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On Tue, Jun 30, 2009 at 12:18:19AM -0700, Yinghai Lu wrote: > On Mon, Jun 29, 2009 at 11:45 PM, Gleb Natapov wrote= : > > KVM would like to provide x2APIC interface to a guest without emula= ting > > interrupt remapping device. The reason KVM prefers guest to use x2A= PIC > > is that x2APIC interface is better virtualizable and provides bette= r > > performance than mmio xAPIC interface: > > > > - msr exits are faster than mmio (no page table walk, emulation) > > - no need to read back ICR to look at the busy bit > > - one 64 bit ICR write instead of two 32 bit writes > > - shared code with the Hyper-V paravirt interface > > > > Included patch changes x2APIC enabling logic to enable it even if I= R > > initialization failed, but kernel runs under KVM and no apic id is > > greater than 255 (if there is one spec requires BIOS to move to x2a= pic > > mode before starting an OS). > > > > Signed-off-by: Gleb Natapov > > diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/api= c.c > > index 8c7c042..351d55a 100644 > > --- a/arch/x86/kernel/apic/apic.c > > +++ b/arch/x86/kernel/apic/apic.c > > @@ -49,6 +49,7 @@ > > =9A#include > > =9A#include > > =9A#include > > +#include > > > > =9Aunsigned int num_processors; > > > > @@ -1363,52 +1364,75 @@ void enable_x2apic(void) > > =9A} > > =9A#endif /* CONFIG_X86_X2APIC */ > > > > -void __init enable_IR_x2apic(void) > > +int __init enable_IR(void) > > =9A{ > > =9A#ifdef CONFIG_INTR_REMAP > > =9A =9A =9A =9Aint ret; > > - =9A =9A =9A unsigned long flags; > > - =9A =9A =9A struct IO_APIC_route_entry **ioapic_entries =3D NULL; > > > > =9A =9A =9A =9Aret =3D dmar_table_init(); > > =9A =9A =9A =9Aif (ret) { > > =9A =9A =9A =9A =9A =9A =9A =9Apr_debug("dmar_table_init() failed w= ith %d:\n", ret); > > - =9A =9A =9A =9A =9A =9A =9A goto ir_failed; > > + =9A =9A =9A =9A =9A =9A =9A return 0; > > =9A =9A =9A =9A} > > > > =9A =9A =9A =9Aif (!intr_remapping_supported()) { > > =9A =9A =9A =9A =9A =9A =9A =9Apr_debug("intr-remapping not support= ed\n"); > > - =9A =9A =9A =9A =9A =9A =9A goto ir_failed; > > + =9A =9A =9A =9A =9A =9A =9A return 0; > > =9A =9A =9A =9A} > > > > - > > =9A =9A =9A =9Aif (!x2apic_preenabled && skip_ioapic_setup) { > > =9A =9A =9A =9A =9A =9A =9A =9Apr_info("Skipped enabling intr-remap= because of skipping " > > =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A"io-apic setup\n"); > > - =9A =9A =9A =9A =9A =9A =9A return; > > + =9A =9A =9A =9A =9A =9A =9A return 0; > > =9A =9A =9A =9A} > > > > + =9A =9A =9A if (enable_intr_remapping(x2apic_supported())) > > + =9A =9A =9A =9A =9A =9A =9A return 0; > > + > > + =9A =9A =9A pr_info("Enabled Interrupt-remapping\n"); > > + > > + =9A =9A =9A return 1; > > + > > +#endif > > + =9A =9A =9A return 0; > > +} > > + > > +void __init enable_IR_x2apic(void) > > +{ > > + =9A =9A =9A unsigned long flags; > > + =9A =9A =9A struct IO_APIC_route_entry **ioapic_entries =3D NULL; > > + =9A =9A =9A int ret, x2apic_enabled =3D 0; > > + > > =9A =9A =9A =9Aioapic_entries =3D alloc_ioapic_entries(); > > =9A =9A =9A =9Aif (!ioapic_entries) { > > - =9A =9A =9A =9A =9A =9A =9A pr_info("Allocate ioapic_entries fail= ed: %d\n", ret); > > - =9A =9A =9A =9A =9A =9A =9A goto end; > > + =9A =9A =9A =9A =9A =9A =9A =9Apr_info("Allocate ioapic_entries f= ailed\n"); > > + =9A =9A =9A =9A =9A =9A =9A =9Agoto out; > > =9A =9A =9A =9A} > > > > =9A =9A =9A =9Aret =3D save_IO_APIC_setup(ioapic_entries); > > =9A =9A =9A =9Aif (ret) { > > =9A =9A =9A =9A =9A =9A =9A =9Apr_info("Saving IO-APIC state failed= : %d\n", ret); > > - =9A =9A =9A =9A =9A =9A =9A goto end; > > + =9A =9A =9A =9A =9A =9A =9A goto out; > > =9A =9A =9A =9A} > > > > =9A =9A =9A =9Alocal_irq_save(flags); > > - =9A =9A =9A mask_IO_APIC_setup(ioapic_entries); > > =9A =9A =9A =9Amask_8259A(); > > + =9A =9A =9A mask_IO_APIC_setup(ioapic_entries); > > > > - =9A =9A =9A ret =3D enable_intr_remapping(x2apic_supported()); > > - =9A =9A =9A if (ret) > > - =9A =9A =9A =9A =9A =9A =9A goto end_restore; > > + =9A =9A =9A ret =3D enable_IR(); > > + =9A =9A =9A if (!ret) { > > + =9A =9A =9A =9A =9A =9A =9A /* IR is required if x2apic is enable= d by BIOS even > > + =9A =9A =9A =9A =9A =9A =9A =9A* when running in kvm since this i= ndicates present > > + =9A =9A =9A =9A =9A =9A =9A =9A* of APIC ID > 255 */ > > + =9A =9A =9A =9A =9A =9A =9A if (x2apic_preenabled || !kvm_para_av= ailable()) > > + =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A goto nox2apic; > > + =9A =9A =9A =9A =9A =9A =9A else > > + =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A /* without IR all CPU= s can be addressed by IOAPIC/MSI > > + =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A* only in physical= mode */ > > + =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A =9A x2apic_phys =3D 1; > > + =9A =9A =9A } >=20 > how about kexec second kernel in KVM ? >=20 > x2apic_preenabled will be set in second kernel. >=20 Yes, bummer. But the similar problem exist now and without KVM. After running x2apic enabled kernel you can't kexec kernel without x2apic support. Shouldn't apic be reset to its initial state on reboot? -- Gleb.