From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BF031FDA99 for ; Mon, 13 Jan 2025 10:04:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736762644; cv=none; b=iKwTr6joLxiwKJQB96DSUnRfd13I2wSlZ+MSbMhSJ1jUSWGZqH4auGo2dUGDwv4txhCVIeZbzWpEskj0PAPaejChhyPHTF+tntS3RyR+XbdgntYpR9xP53/PvMqLN3XBzv/Ydoe5Drrm6vSor7rZu61ZJl5bsMc8dpvfSy3sX3I= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736762644; c=relaxed/simple; bh=ZOfgHpa8ypvnRMVqtmC+m7Qff2xgsmf54CGkKpjdoWQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kWVW26qH0+RI+LodJ0TVLm9fDCxI+h6r8hMup9FUJEzH2rHHTA9qJBUv5j5QiBDkWRvN2tsoAkZlitPZEMWZxQ6PsnHlzGhLcIwCIaxs9CUFqGfIIaOJ0gRoNh9cEqqtLuzyQII1LrmSA+UqHf6dJWKXECwVqI08Fz8+d4peFjo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=citrix.com; spf=pass smtp.mailfrom=cloud.com; dkim=pass (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b=Q7YsBhbS; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=citrix.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="Q7YsBhbS" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-aaec61d0f65so800327966b.1 for ; Mon, 13 Jan 2025 02:04:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1736762641; x=1737367441; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=NAozgVCwe5MTkDq2tRs81RaOUpNYbkuEcwNQ118VObQ=; b=Q7YsBhbSuc8KZkQ4Dn7p2GSutbBZaRv/ZqVbUw5Pia4cGlYn9EBIj+oZkekT3QGM34 crPhq5hlN8aUftGAeP4XkBeTloHK/FDA4ytmRHy6fB9XNTFn/h1LVWnkkPunvzJZCIrV C9FdaWk0SYsbeY6Y0cA3AyFvGINP2uT3PTIhw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736762641; x=1737367441; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NAozgVCwe5MTkDq2tRs81RaOUpNYbkuEcwNQ118VObQ=; b=N+DpHmeWNVv/JAu0lytwdW5NkE6OhAEr55ioZQZinO5/vV+WsEG0/tm3vS47fMipfj mUR5INp8N8GVXehV72lROMT6Fe0tphkS78JUH50LjiqxyXIru/NXqJEAORtDrPANZHkX KmjD+so/NCf1F3t/5B89p+0DJ2x4iJp3Qa8gMFxCtPsO+vRqcxBwI081rPOBpX6ab5ay MXBqEZOGE5NBPiKctsRR7urmhmugYEP2kKf+BPspEsWZyQige8GbgsIA1hqmoGxmbwd8 DMcw+wD0Lugb/zaRrOIzeNifz/eF//glrAKNbcS38GR0BG/gzdAM80GYQCZjaysSinBe m76A== X-Gm-Message-State: AOJu0YzNlDCFcEy9sDo+IFRjdGTp9zfeyk02BJM/oJ/Y90cdm1no+Obz WDssRWPVU0nL8AJ7hG9dR5IsNghhbgBSe4S5eesxOWylTPxVN11gz5ArdNxe5yM= X-Gm-Gg: ASbGncsdtcO6NxJtvPCbDHvT9Q7us24SO2v5KtYHJc4gIfhtR+RZR+Ehv7ioTaGtGEX 2lb/lc8XQp4HclcFt5k2Stlofr3LfY1s6Uh8e41BVdc0uVb4u7cdY/c9p8SVb/I/+pjq3adMOK/ I7AI2u9FggyDLFLLeE+r+aaLg3A3YP8r2fpPRl5wXo2lSO+wJ0bGm2YGQ8reoYFR7hBtzVBxouw BNDekoGTMRGLuNVFJNOu0LbmKgj8QMud+/sNxdXpIC982qRUjUomV7AL1+loQ== X-Google-Smtp-Source: AGHT+IHN6cLAzIHvJBapqtn9TCnnP24m5RjiZPdKHg1bO2RsIgvurTxekJiYNBk5q8SmCTsIwdMt3A== X-Received: by 2002:a17:907:60cf:b0:aa6:88f5:5fef with SMTP id a640c23a62f3a-ab2ab6fb426mr1905306566b.32.1736762641198; Mon, 13 Jan 2025 02:04:01 -0800 (PST) Received: from localhost ([84.78.159.3]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ab2c90d9b19sm474988566b.73.2025.01.13.02.03.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jan 2025 02:03:59 -0800 (PST) Date: Mon, 13 Jan 2025 11:03:58 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Bjorn Helgaas Cc: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, linux-pci@vger.kernel.org, Nirmal Patel , Jonathan Derrick , Lorenzo Pieralisi , Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= , Manivannan Sadhasivam , Rob Herring , Bjorn Helgaas Subject: Re: [PATCH 2/3] vmd: disable MSI remapping bypass under Xen Message-ID: References: <20250110140152.27624-3-roger.pau@citrix.com> <20250110222525.GA318386@bhelgaas> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20250110222525.GA318386@bhelgaas> On Fri, Jan 10, 2025 at 04:25:25PM -0600, Bjorn Helgaas wrote: > Match historical subject line style for prefix and capitalization: > > PCI: vmd: Set devices to D0 before enabling PM L1 Substates > PCI: vmd: Add DID 8086:B06F and 8086:B60B for Intel client SKUs > PCI: vmd: Fix indentation issue in vmd_shutdown() > > On Fri, Jan 10, 2025 at 03:01:49PM +0100, Roger Pau Monne wrote: > > MSI remapping bypass (directly configuring MSI entries for devices on the VMD > > bus) won't work under Xen, as Xen is not aware of devices in such bus, and > > hence cannot configure the entries using the pIRQ interface in the PV case, and > > in the PVH case traps won't be setup for MSI entries for such devices. > > > > Until Xen is aware of devices in the VMD bus prevent the > > VMD_FEAT_CAN_BYPASS_MSI_REMAP capability from being used when running as any > > kind of Xen guest. > > Wrap to fit in 75 columns. Hm, OK, but isn't the limit 80 columns according to the kernel coding style (Documentation/process/coding-style.rst)? I don't mind adjusting, but if you are going to ask every submitter to limit to 75 columns then the coding style document should be updated to reflect that. > Can you include a hint about *why* Xen is not aware of devices below > VMD? That will help to know whether it's a permanent unfixable > situation or something that could be done eventually. Xen would need to be made aware of the devices exposed behind the VMD bridge, so it can manage them. For example Xen is the entity that controls the local APICs, and hence interrupts must be configured by Xen. Xen needs knowledge about the devices behind the VMD bridge, and how to access those devices PCI config space to at least configure MSI or MSI-X capabilities. It could possibly be exposed similarly to how Xen currently deals with ECAM areas. None of this is present at the moment, could always be added later and Linux be made aware that the limitation no longer applies. That would require changes in both Xen and Linux to propagate the VMD information into Xen. > > Signed-off-by: Roger Pau Monné > > --- > > drivers/pci/controller/vmd.c | 9 +++++++++ > > 1 file changed, 9 insertions(+) > > > > diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c > > index 264a180403a0..d9b7510ace29 100644 > > --- a/drivers/pci/controller/vmd.c > > +++ b/drivers/pci/controller/vmd.c > > @@ -965,6 +965,15 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id) > > struct vmd_dev *vmd; > > int err; > > > > + if (xen_domain()) > > + /* > > + * Xen doesn't have knowledge about devices in the VMD bus. > > Also here. Would you be OK with something like: "Xen doesn't have knowledge about devices in the VMD bus because the config space of devices behind the VMD bridge is not known to Xen, and hence Xen cannot discover or configure them in any way. Bypass of MSI remapping won't work in that case as direct write by Linux to the MSI entries won't result in functional interrupts, as it's Xen the entity that manages the local APIC and must configure interrupts." Thanks, Roger.