From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39D69C4332F for ; Wed, 1 Nov 2023 01:06:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.626046.975955 (Exim 4.92) (envelope-from ) id 1qxzgI-0004id-PO; Wed, 01 Nov 2023 01:06:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 626046.975955; Wed, 01 Nov 2023 01:06:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qxzgI-0004iW-M8; Wed, 01 Nov 2023 01:06:02 +0000 Received: by outflank-mailman (input) for mailman id 626046; Wed, 01 Nov 2023 01:06:01 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qxzgH-0004iQ-ID for xen-devel@lists.xenproject.org; Wed, 01 Nov 2023 01:06:01 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d075067c-7852-11ee-98d6-6d05b1d4d9a1; Wed, 01 Nov 2023 02:05:59 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7]) by mailhost.m5p.com (8.17.1/8.15.2) with ESMTPS id 3A115N3Y025419 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 31 Oct 2023 21:05:29 -0400 (EDT) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.17.1/8.15.2/Submit) id 3A115K8c025418; Tue, 31 Oct 2023 18:05:20 -0700 (PDT) (envelope-from ehem) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d075067c-7852-11ee-98d6-6d05b1d4d9a1 Date: Tue, 31 Oct 2023 18:05:20 -0700 From: Elliott Mitchell To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Cc: Neowutran , xen-devel@lists.xenproject.org, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: Re: [PATCH] x86/x2apic: introduce a mixed physical/cluster mode Message-ID: References: <20231024135150.49232-1-roger.pau@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Mon, Oct 30, 2023 at 04:27:22PM +0100, Roger Pau Monné wrote: > On Mon, Oct 30, 2023 at 07:50:27AM -0700, Elliott Mitchell wrote: > > On Tue, Oct 24, 2023 at 03:51:50PM +0200, Roger Pau Monne wrote: > > > diff --git a/xen/arch/x86/genapic/x2apic.c b/xen/arch/x86/genapic/x2apic.c > > > index 707deef98c27..15632cc7332e 100644 > > > --- a/xen/arch/x86/genapic/x2apic.c > > > +++ b/xen/arch/x86/genapic/x2apic.c > > > @@ -220,38 +239,56 @@ static struct notifier_block x2apic_cpu_nfb = { > > > static int8_t __initdata x2apic_phys = -1; > > > boolean_param("x2apic_phys", x2apic_phys); > > > > > > +enum { > > > + unset, physical, cluster, mixed > > > +} static __initdata x2apic_mode = unset; > > > + > > > +static int __init parse_x2apic_mode(const char *s) > > > +{ > > > + if ( !cmdline_strcmp(s, "physical") ) > > > + x2apic_mode = physical; > > > + else if ( !cmdline_strcmp(s, "cluster") ) > > > + x2apic_mode = cluster; > > > + else if ( !cmdline_strcmp(s, "mixed") ) > > > + x2apic_mode = mixed; > > > + else > > > + return EINVAL; > > > + > > > + return 0; > > > +} > > > +custom_param("x2apic-mode", parse_x2apic_mode); > > > + > > > const struct genapic *__init apic_x2apic_probe(void) > > > { > > > - if ( x2apic_phys < 0 ) > > > + /* x2apic-mode option has preference over x2apic_phys. */ > > > + if ( x2apic_phys >= 0 && x2apic_mode == unset ) > > > + x2apic_mode = x2apic_phys ? physical : cluster; > > > + > > > + if ( x2apic_mode == unset ) > > > { > > > - /* > > > - * Force physical mode if there's no (full) interrupt remapping support: > > > - * The ID in clustered mode requires a 32 bit destination field due to > > > - * the usage of the high 16 bits to hold the cluster ID. > > > - */ > > > - x2apic_phys = iommu_intremap != iommu_intremap_full || > > > - (acpi_gbl_FADT.flags & ACPI_FADT_APIC_PHYSICAL) || > > > - IS_ENABLED(CONFIG_X2APIC_PHYSICAL); > > > - } > > > - else if ( !x2apic_phys ) > > > - switch ( iommu_intremap ) > > > + if ( acpi_gbl_FADT.flags & ACPI_FADT_APIC_PHYSICAL ) > > > { > > > > Could this explain the issues with recent AMD processors/motherboards? > > > > Mainly the firmware had been setting this flag, but Xen was previously > > ignoring it? > > No, not unless you pass {no-}x2apic_phys={false,0} on the Xen command > line to force logical (clustered) destination mode. > > > As such Xen had been attempting to use cluster mode on an > > x2APIC where that mode was broken for physical interrupts? > > No, not realy, x2apic_phys was already forced to true if > acpi_gbl_FADT.flags & ACPI_FADT_APIC_PHYSICAL is set on the FADT (I > just delete that line in this same chunk and move it here). I think I inverted the case in my speculation. Perhaps AMD's firmware SHOULD have been setting ACPI_FADT_APIC_PHYSICAL, but failed to? Given the rough launch with AMD's latest, I could readily see there being a goof like this. Question will then be, will mixed mode work on these processors? -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445