pci: acpi: Query on ACPI Device Tree Representation and Enumeration for Xilinx FPGA PCIe Endpoint functions

pci: acpi: Query on ACPI Device Tree Representation and Enumeration for Xilinx FPGA PCIe Endpoint functions

[Muni Sekhar]

Hi all,

I am currently working on a project involving a Xilinx FPGA connected
to an x86 CPU via a PCIe root port. The Xilinx FPGA functions as a
PCIe endpoint with single function capability and is programmed to
emulate the Soundwire Master controller. It can be dynamically
reprogrammed to emulate other interfaces as needed. Essentially, the
FPGA emulates an interface and connects to the CPU via the PCIe bus.

Given this setup, the BIOS does not have prior knowledge of the
function implemented in the Xilinx FPGA PCIe endpoint. I have a couple
of questions regarding this configuration:

Is it possible to define an ACPI Device Tree representation for this
type of hardware setup?
Can we achieve ACPI-based device enumeration with this configuration?

I would greatly appreciate any guidance or references to documentation
that could help us achieve this.

Thank you for your time and assistance.


-- 
Thanks,
Sekhar

Re: pci: acpi: Query on ACPI Device Tree Representation and Enumeration for Xilinx FPGA PCIe Endpoint functions

[Bjorn Helgaas]

On Thu, Feb 27, 2025 at 07:25:32PM +0530, Muni Sekhar wrote:
> Hi all,
> 
> I am currently working on a project involving a Xilinx FPGA connected
> to an x86 CPU via a PCIe root port. The Xilinx FPGA functions as a
> PCIe endpoint with single function capability and is programmed to
> emulate the Soundwire Master controller. It can be dynamically
> reprogrammed to emulate other interfaces as needed. Essentially, the
> FPGA emulates an interface and connects to the CPU via the PCIe bus.
> 
> Given this setup, the BIOS does not have prior knowledge of the
> function implemented in the Xilinx FPGA PCIe endpoint. I have a couple
> of questions regarding this configuration:
> 
> Is it possible to define an ACPI Device Tree representation for this
> type of hardware setup?
> Can we achieve ACPI-based device enumeration with this configuration?

If the FPGA is programmed before BIOS enumerates PCI devices, the FPGA
would look just like any other PCI device, and BIOS would be able to
read the Vendor ID and Device ID and would be able to size and program
the BARs.

So I assume the FPGA is not programmed before BIOS enumeration, the
FPGA doesn't respond at all when BIOS or Linux reads the Vendor ID,
and you want to program the FPGA later and make Linux enumerate to
find it.

From Linux's point of view, this is basically a hot-add of a PCI
device.  If the Root Port supports hotplug and you have pciehp enabled
(CONFIG_HOTPLUG_PCI_PCIE=y) and if the FPGA comes out of reset and
brings up the PCIe link after being programmed, it all might "just
work."  You can also force a complete re-enumeration by writing a
non-zero value to /sys/bus/pci/rescan.

I'm not sure why you would need ACPI or a device tree to be involved.
ACPI and device tree are ways to tell the OS about devices that do not
have a native enumeration protocol.  PCI devices (like the programmed
FPGA) do support native enumeration, so generally we don't need ACPI
or device tree descriptions of them.  PCI host bridges have a
CPU-specific bus on the upstream side and a PCI bus on the downstream
side, so they are not themselves PCI devices, and we do need ACPI or
device tree descriptions for them.

If you have something that doesn't work like you expect, can you post
a complete dmesg log and any user commands you're using to program the
FPGA?

Bjorn

Re: pci: acpi: Query on ACPI Device Tree Representation and Enumeration for Xilinx FPGA PCIe Endpoint functions

[Muni Sekhar]

On Thu, Feb 27, 2025 at 9:34 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Thu, Feb 27, 2025 at 07:25:32PM +0530, Muni Sekhar wrote:
> > Hi all,
> >
> > I am currently working on a project involving a Xilinx FPGA connected
> > to an x86 CPU via a PCIe root port. The Xilinx FPGA functions as a
> > PCIe endpoint with single function capability and is programmed to
> > emulate the Soundwire Master controller. It can be dynamically
> > reprogrammed to emulate other interfaces as needed. Essentially, the
> > FPGA emulates an interface and connects to the CPU via the PCIe bus.
> >
> > Given this setup, the BIOS does not have prior knowledge of the
> > function implemented in the Xilinx FPGA PCIe endpoint. I have a couple
> > of questions regarding this configuration:
> >
> > Is it possible to define an ACPI Device Tree representation for this
> > type of hardware setup?
> > Can we achieve ACPI-based device enumeration with this configuration?
>
> If the FPGA is programmed before BIOS enumerates PCI devices, the FPGA
> would look just like any other PCI device, and BIOS would be able to
> read the Vendor ID and Device ID and would be able to size and program
> the BARs.
Yes, the FPGA is programmed with this Soundwire IP before the BIOS
enumerates PCI devices.
We need to port the Soundwire driver
(https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soundwire/qcom.c)
 to the x86 platform.

Since x86 platforms typically do not use device trees, and the
Soundwire IP is implemented in the FPGA, how can we emulate device
tree functionality or use a different mechanism to pass hardware
configuration to the driver? Specifically, how can we handle the
following API calls on an x86 platform?

   ret = of_property_read_u32(np, "qcom,din-ports", &val);
   ret = of_property_read_u32(np, "qcom,dout-ports", &val);
   ret = of_property_read_u8_array(np, "qcom,ports-offset1", off1, nports);

static const struct of_device_id qcom_swrm_of_match[] = {
{ .compatible = "qcom,soundwire-v1.3.0", .data = &swrm_v1_3_data },
{ .compatible = "qcom,soundwire-v1.5.1", .data = &swrm_v1_5_data },
{ .compatible = "qcom,soundwire-v1.6.0", .data = &swrm_v1_6_data },
{ .compatible = "qcom,soundwire-v1.7.0", .data = &swrm_v1_5_data },
{ .compatible = "qcom,soundwire-v2.0.0", .data = &swrm_v2_0_data },
{/* sentinel */},
};

Basically, how can we define ACPI tables for functions implemented in
an FPGA that connects to the system via PCI?


>
> So I assume the FPGA is not programmed before BIOS enumeration, the
> FPGA doesn't respond at all when BIOS or Linux reads the Vendor ID,
> and you want to program the FPGA later and make Linux enumerate to
> find it.
>
> From Linux's point of view, this is basically a hot-add of a PCI
> device.  If the Root Port supports hotplug and you have pciehp enabled
> (CONFIG_HOTPLUG_PCI_PCIE=y) and if the FPGA comes out of reset and
> brings up the PCIe link after being programmed, it all might "just
> work."  You can also force a complete re-enumeration by writing a
> non-zero value to /sys/bus/pci/rescan.
>
> I'm not sure why you would need ACPI or a device tree to be involved.
> ACPI and device tree are ways to tell the OS about devices that do not
> have a native enumeration protocol.  PCI devices (like the programmed
> FPGA) do support native enumeration, so generally we don't need ACPI
> or device tree descriptions of them.  PCI host bridges have a
> CPU-specific bus on the upstream side and a PCI bus on the downstream
> side, so they are not themselves PCI devices, and we do need ACPI or
> device tree descriptions for them.
>
> If you have something that doesn't work like you expect, can you post
> a complete dmesg log and any user commands you're using to program the
> FPGA?
>
> Bjorn



--
Thanks,
Sekhar

Re: pci: acpi: Query on ACPI Device Tree Representation and Enumeration for Xilinx FPGA PCIe Endpoint functions

[Bjorn Helgaas]

[+cc SoundWire folks]

On Fri, Feb 28, 2025 at 08:19:44PM +0530, Muni Sekhar wrote:
> On Thu, Feb 27, 2025 at 9:34 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > On Thu, Feb 27, 2025 at 07:25:32PM +0530, Muni Sekhar wrote:
> > > I am currently working on a project involving a Xilinx FPGA connected
> > > to an x86 CPU via a PCIe root port. The Xilinx FPGA functions as a
> > > PCIe endpoint with single function capability and is programmed to
> > > emulate the Soundwire Master controller. It can be dynamically
> > > reprogrammed to emulate other interfaces as needed. Essentially, the
> > > FPGA emulates an interface and connects to the CPU via the PCIe bus.
> > >
> > > Given this setup, the BIOS does not have prior knowledge of the
> > > function implemented in the Xilinx FPGA PCIe endpoint. I have a couple
> > > of questions regarding this configuration:
> > >
> > > Is it possible to define an ACPI Device Tree representation for this
> > > type of hardware setup?
> > > Can we achieve ACPI-based device enumeration with this configuration?
> >
> > If the FPGA is programmed before BIOS enumerates PCI devices, the FPGA
> > would look just like any other PCI device, and BIOS would be able to
> > read the Vendor ID and Device ID and would be able to size and program
> > the BARs.
>
> Yes, the FPGA is programmed with this Soundwire IP before the BIOS
> enumerates PCI devices.
> We need to port the Soundwire driver
> (https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soundwire/qcom.c)
>  to the x86 platform.
> 
> Since x86 platforms typically do not use device trees, and the
> Soundwire IP is implemented in the FPGA, how can we emulate device
> tree functionality or use a different mechanism to pass hardware
> configuration to the driver? Specifically, how can we handle the
> following API calls on an x86 platform?
> 
>    ret = of_property_read_u32(np, "qcom,din-ports", &val);
>    ret = of_property_read_u32(np, "qcom,dout-ports", &val);
>    ret = of_property_read_u8_array(np, "qcom,ports-offset1", off1, nports);
> 
> static const struct of_device_id qcom_swrm_of_match[] = {
> { .compatible = "qcom,soundwire-v1.3.0", .data = &swrm_v1_3_data },
> { .compatible = "qcom,soundwire-v1.5.1", .data = &swrm_v1_5_data },
> { .compatible = "qcom,soundwire-v1.6.0", .data = &swrm_v1_6_data },
> { .compatible = "qcom,soundwire-v1.7.0", .data = &swrm_v1_5_data },
> { .compatible = "qcom,soundwire-v2.0.0", .data = &swrm_v2_0_data },
> {/* sentinel */},
> };
> 
> Basically, how can we define ACPI tables for functions implemented in
> an FPGA that connects to the system via PCI?

Seems like a generic problem for PCI sound devices, and I don't know
how drivers deal with it.  It looks like all the SoundWire drivers
are platform drivers (not PCI drivers), so there's nothing there to
look at.

Maybe the sound folks have ideas.

Bjorn

Re: pci: acpi: Query on ACPI Device Tree Representation and Enumeration for Xilinx FPGA PCIe Endpoint functions

[Vinod Koul]

On 28-02-25, 10:32, Bjorn Helgaas wrote:
> [+cc SoundWire folks]
> 
> On Fri, Feb 28, 2025 at 08:19:44PM +0530, Muni Sekhar wrote:
> > On Thu, Feb 27, 2025 at 9:34 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > On Thu, Feb 27, 2025 at 07:25:32PM +0530, Muni Sekhar wrote:
> > > > I am currently working on a project involving a Xilinx FPGA connected
> > > > to an x86 CPU via a PCIe root port. The Xilinx FPGA functions as a
> > > > PCIe endpoint with single function capability and is programmed to
> > > > emulate the Soundwire Master controller. It can be dynamically
> > > > reprogrammed to emulate other interfaces as needed. Essentially, the
> > > > FPGA emulates an interface and connects to the CPU via the PCIe bus.
> > > >
> > > > Given this setup, the BIOS does not have prior knowledge of the
> > > > function implemented in the Xilinx FPGA PCIe endpoint. I have a couple
> > > > of questions regarding this configuration:
> > > >
> > > > Is it possible to define an ACPI Device Tree representation for this
> > > > type of hardware setup?
> > > > Can we achieve ACPI-based device enumeration with this configuration?
> > >
> > > If the FPGA is programmed before BIOS enumerates PCI devices, the FPGA
> > > would look just like any other PCI device, and BIOS would be able to
> > > read the Vendor ID and Device ID and would be able to size and program
> > > the BARs.
> >
> > Yes, the FPGA is programmed with this Soundwire IP before the BIOS
> > enumerates PCI devices.
> > We need to port the Soundwire driver
> > (https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soundwire/qcom.c)
> >  to the x86 platform.
> > 
> > Since x86 platforms typically do not use device trees, and the
> > Soundwire IP is implemented in the FPGA, how can we emulate device
> > tree functionality or use a different mechanism to pass hardware
> > configuration to the driver? Specifically, how can we handle the
> > following API calls on an x86 platform?
> > 
> >    ret = of_property_read_u32(np, "qcom,din-ports", &val);
> >    ret = of_property_read_u32(np, "qcom,dout-ports", &val);
> >    ret = of_property_read_u8_array(np, "qcom,ports-offset1", off1, nports);
> > 
> > static const struct of_device_id qcom_swrm_of_match[] = {
> > { .compatible = "qcom,soundwire-v1.3.0", .data = &swrm_v1_3_data },
> > { .compatible = "qcom,soundwire-v1.5.1", .data = &swrm_v1_5_data },
> > { .compatible = "qcom,soundwire-v1.6.0", .data = &swrm_v1_6_data },
> > { .compatible = "qcom,soundwire-v1.7.0", .data = &swrm_v1_5_data },
> > { .compatible = "qcom,soundwire-v2.0.0", .data = &swrm_v2_0_data },
> > {/* sentinel */},
> > };
> > 
> > Basically, how can we define ACPI tables for functions implemented in
> > an FPGA that connects to the system via PCI?
> 
> Seems like a generic problem for PCI sound devices, and I don't know
> how drivers deal with it.  It looks like all the SoundWire drivers
> are platform drivers (not PCI drivers), so there's nothing there to
> look at.
> 
> Maybe the sound folks have ideas.

X86-Intel devices are PCI devices but then they are compound devices
with DSP and SoundWire links. Please check the DisCo spec for SoundWire,
that already defines the ACPI tables and properties which both Intel and
AMD drivers already use

That should be your start point...

-- 
~Vinod