From: Lizhi Hou <lizhi.hou@amd.com>
To: David Zhang <yidong.zhang@amd.com>, <quic_jhugo@quicinc.com>,
<karol.wachowski@linux.intel.com>, <max.zhen@amd.com>,
<ogabbay@kernel.org>, <dri-devel@lists.freedesktop.org>,
<linux-kernel@vger.kernel.org>
Cc: <sonal.santan@amd.com>, <mario.limonciello@amd.com>,
Wendy Liang <wendy.liang@amd.com>
Subject: Re: [PATCH V0 12/21] accel/amdxdna: Decouple AIE4 doorbell and MSI-X notification transport hooks
Date: Mon, 28 Sep 2026 13:16:30 -0700 [thread overview]
Message-ID: <ebd56ccf-e7df-beb2-57ff-6fdf34b29a97@amd.com> (raw)
In-Reply-To: <20260926013448.3840921-13-yidong.zhang@amd.com>
On 9/25/26 18:34, David Zhang wrote:
> Separate PCI-specific doorbell and interrupt notification handling from
> the transport-neutral context code:
> - Move MSI-X ISR and registration out of aie4_ctx.c into transport hooks
> aie4_request_notification() and aie4_free_notification() in aie4_pci.c.
> - Add transport hooks aie4_doorbell_setup() and aie4_doorbell_ring() to
> validate the doorbell offset against the mapped doorbell BAR and ring
> the hardware doorbell.
> - Map the doorbell BAR (BAR 2) via pcim_iomap() in aie4m_pcidev_init()
> and record ndev->doorbell_base. The doorbells are used exclusively by
> kernel submit driver on VF and classic devices. PF devices only perform
> management functions, and never host hardware contexts, thus never use
> the doorbells.
>
> Co-developed-by: Wendy Liang <wendy.liang@amd.com>
> Signed-off-by: Wendy Liang <wendy.liang@amd.com>
> Signed-off-by: David Zhang <yidong.zhang@amd.com>
> ---
> drivers/accel/amdxdna/aie4_ctx.c | 29 +++--------
> drivers/accel/amdxdna/aie4_pci.c | 79 +++++++++++++++++++++++++++++
> drivers/accel/amdxdna/aie4_pci.h | 18 +++++++
> drivers/accel/amdxdna/amdxdna_ctx.h | 2 +
> 4 files changed, 107 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/accel/amdxdna/aie4_ctx.c b/drivers/accel/amdxdna/aie4_ctx.c
> index 90da4e8c4f54..fba4ef25ffbe 100644
> --- a/drivers/accel/amdxdna/aie4_ctx.c
> +++ b/drivers/accel/amdxdna/aie4_ctx.c
> @@ -21,18 +21,9 @@
> #include "amdxdna_mailbox_helper.h"
> #include "amdxdna_pci_drv.h"
>
> -static irqreturn_t cert_comp_isr(int irq, void *p)
> -{
> - struct cert_comp *cert_comp = p;
> -
> - wake_up_all(&cert_comp->waitq);
> - return IRQ_HANDLED;
> -}
> -
> static struct cert_comp *aie4_lookup_cert_comp(struct amdxdna_dev_hdl *ndev, u32 msix_idx)
> {
> struct amdxdna_dev *xdna = ndev->aie.xdna;
> - struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
> struct cert_comp *cert_comp;
> int ret;
>
> @@ -50,32 +41,27 @@ static struct cert_comp *aie4_lookup_cert_comp(struct amdxdna_dev_hdl *ndev, u32
>
> cert_comp->ndev = ndev;
> cert_comp->msix_idx = msix_idx;
> + cert_comp->irq = -ENOENT;
> init_waitqueue_head(&cert_comp->waitq);
> kref_init(&cert_comp->kref);
>
> - ret = pci_irq_vector(pdev, cert_comp->msix_idx);
> - if (ret < 0) {
> - XDNA_ERR(xdna, "MSI-X idx %u is invalid, ret:%d", msix_idx, ret);
> - goto free_cert_comp;
> - }
> - cert_comp->irq = ret;
> -
> - ret = request_irq(cert_comp->irq, cert_comp_isr, 0, "xdna_hsa", cert_comp);
> + /* Transport-specific: PCI wires an MSI-X irq, platform an IPI callback. */
> + ret = aie4_request_notification(cert_comp);
> if (ret) {
> - XDNA_ERR(xdna, "request irq %d failed %d", cert_comp->irq, ret);
> + XDNA_ERR(xdna, "request notification for msix idx %u failed %d", msix_idx, ret);
> goto free_cert_comp;
> }
>
> ret = xa_err(xa_store(&ndev->cert_comp_xa, msix_idx, cert_comp, GFP_KERNEL));
> if (ret) {
> - XDNA_ERR(xdna, "store cert_comp for msix idx %d failed %d", msix_idx, ret);
> + XDNA_ERR(xdna, "store cert_comp for msix idx %u failed %d", msix_idx, ret);
> goto free_irq;
> }
>
> return cert_comp;
>
> free_irq:
> - free_irq(cert_comp->irq, cert_comp);
> + aie4_free_notification(cert_comp);
> free_cert_comp:
> kfree(cert_comp);
> return NULL;
> @@ -89,7 +75,7 @@ static void cert_comp_release(struct kref *kref)
> drm_WARN_ON(&ndev->aie.xdna->ddev, !mutex_is_locked(&ndev->cert_comp_lock));
>
> xa_erase(&ndev->cert_comp_xa, cert_comp->msix_idx);
> - free_irq(cert_comp->irq, cert_comp);
> + aie4_free_notification(cert_comp);
> kfree(cert_comp);
> }
>
> @@ -99,6 +85,7 @@ static void aie4_put_cert_comp(struct cert_comp *cert_comp)
>
> ndev = cert_comp->ndev;
> guard(mutex)(&ndev->cert_comp_lock);
> +
> kref_put(&cert_comp->kref, cert_comp_release);
> }
>
> diff --git a/drivers/accel/amdxdna/aie4_pci.c b/drivers/accel/amdxdna/aie4_pci.c
> index 1e2b120c4972..7b36bd001b64 100644
> --- a/drivers/accel/amdxdna/aie4_pci.c
> +++ b/drivers/accel/amdxdna/aie4_pci.c
> @@ -14,6 +14,7 @@
>
> #include "aie.h"
> #include "aie4_msg_priv.h"
> +#include "amdxdna_ctx.h"
> #include "aie4_pci.h"
> #include "amdxdna_mailbox.h"
> #include "amdxdna_mailbox_helper.h"
> @@ -110,6 +111,82 @@ static void aie4_mailbox_fini(struct amdxdna_dev_hdl *ndev)
> ndev->mbox = NULL;
> }
>
> +static irqreturn_t cert_comp_isr(int irq, void *p)
> +{
> + struct cert_comp *cert_comp = p;
> +
> + wake_up_all(&cert_comp->waitq);
> + return IRQ_HANDLED;
> +}
> +
> +/*
> + * Transport hook: wire the per-cert completion notification. PCI maps the
> + * firmware-provided MSI-X index to a Linux irq and registers cert_comp_isr;
> + * the platform build registers an IPI mailbox callback instead.
> + */
> +int aie4_request_notification(struct cert_comp *comp)
> +{
> + struct pci_dev *pdev = to_pci_dev(comp->ndev->aie.xdna->ddev.dev);
> + int ret;
> +
> + ret = pci_irq_vector(pdev, comp->msix_idx);
> + if (ret < 0)
> + return ret;
> + comp->irq = ret;
> +
> + ret = request_irq(comp->irq, cert_comp_isr, 0, "xdna_hsa", comp);
> + if (ret) {
> + comp->irq = -ENOENT;
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +/* Transport hook: tear down the completion notification wired by the hook above. */
> +void aie4_free_notification(struct cert_comp *comp)
> +{
> + if (comp->irq >= 0)
> + free_irq(comp->irq, comp);
> +}
> +
> +/*
> + * Transport hook: take what this transport needs from the create-context
> + * response. PCI validates the firmware-provided doorbell offset against the
> + * mapped doorbell BAR and stores this context's kick target.
> + */
> +int aie4_doorbell_setup(struct amdxdna_hwctx *hwctx,
> + const struct aie4_msg_create_hw_context_resp *resp)
> +{
> + struct amdxdna_dev *xdna = hwctx->client->xdna;
> + struct amdxdna_dev_hdl *ndev = xdna->dev_handle;
> + struct amdxdna_hwctx_priv *priv = hwctx->priv;
> + struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
> + u64 db_off = (u64)ndev->priv->doorbell_off + resp->doorbell_offset;
> +
> + /*
> + * doorbell_base is a pcim_iomap() of the whole doorbell BAR. The offset
> + * comes from firmware (or, on a VF, the PF/hypervisor); reject one that
> + * would place the u32 doorbell write past the mapped BAR before
> + * aie4_doorbell_ring() ever dereferences priv->doorbell_addr.
> + */
> + if (db_off + sizeof(u32) >
> + pci_resource_len(pdev, xdna->dev_info->doorbell_bar)) {
> + XDNA_ERR(xdna, "doorbell offset 0x%llx out of BAR", db_off);
> + return -EINVAL;
> + }
> +
> + priv->doorbell_addr = ndev->doorbell_base + ndev->priv->doorbell_off +
> + resp->doorbell_offset;
> + return 0;
> +}
> +
> +/* Transport hook: ring this context's doorbell (kick CERT). */
> +void aie4_doorbell_ring(struct amdxdna_hwctx *hwctx)
> +{
> + writel(0, hwctx->priv->doorbell_addr);
> +}
> +
> static int aie4_irq_init(struct amdxdna_dev *xdna)
> {
> struct pci_dev *pdev = to_pci_dev(xdna->ddev.dev);
> @@ -634,6 +711,7 @@ static int aie4m_pcidev_init(struct amdxdna_dev *xdna)
> set_bit(SMU_REG_BAR(ndev, i), &bars);
> set_bit(xdna->dev_info->mbox_bar, &bars);
> set_bit(xdna->dev_info->sram_bar, &bars);
> + set_bit(xdna->dev_info->doorbell_bar, &bars);
>
> for (i = 0; i < PCI_NUM_RESOURCES; i++) {
> if (!test_bit(i, &bars))
> @@ -647,6 +725,7 @@ static int aie4m_pcidev_init(struct amdxdna_dev *xdna)
>
> ndev->mbox_base = tbl[xdna->dev_info->mbox_bar];
> ndev->rbuf_base = tbl[xdna->dev_info->sram_bar];
> + ndev->doorbell_base = tbl[xdna->dev_info->doorbell_bar];
>
> pci_set_master(pdev);
>
> diff --git a/drivers/accel/amdxdna/aie4_pci.h b/drivers/accel/amdxdna/aie4_pci.h
> index 063cedfe3c9d..c6e7f6a80f69 100644
> --- a/drivers/accel/amdxdna/aie4_pci.h
> +++ b/drivers/accel/amdxdna/aie4_pci.h
> @@ -32,6 +32,8 @@ struct amdxdna_hwctx_priv {
>
> struct cert_comp *cert_comp;
> u32 hw_ctx_id;
> +
> + void __iomem *doorbell_addr;
> };
>
> struct amdxdna_dev_priv {
> @@ -54,6 +56,7 @@ struct amdxdna_dev_hdl {
> const struct amdxdna_dev_priv *priv;
> void __iomem *mbox_base;
> void __iomem *rbuf_base;
> + void __iomem *doorbell_base;
>
> struct mailbox *mbox;
> u32 partition_id;
> @@ -114,6 +117,21 @@ int aie4_cmd_wait(struct amdxdna_hwctx *hwctx, u64 seq, u32 timeout);
> /* aie4_pci.c */
> int aie4_restore_power_mode(struct amdxdna_dev_hdl *ndev);
>
> +/*
> + * Transport hooks: one definition per build (aie4_pci.c for PCI; a future
> + * OF/platform transport provides its own), selected at compile time. aie4_ctx.c is
> + * transport-neutral and reaches the doorbell kick and the completion interrupt
> + * only through these. The cert_comp object itself (allocation/xarray/kref/
> + * waitq) is firmware-driven and stays neutral in aie4_ctx.c; only the notification
> + * wiring (PCI MSI-X vs platform IPI callback) is transport-specific.
> + */
> +struct aie4_msg_create_hw_context_resp;
> +int aie4_doorbell_setup(struct amdxdna_hwctx *hwctx,
> + const struct aie4_msg_create_hw_context_resp *resp);
> +void aie4_doorbell_ring(struct amdxdna_hwctx *hwctx);
> +int aie4_request_notification(struct cert_comp *comp);
> +void aie4_free_notification(struct cert_comp *comp);
> +
> /* aie4_sriov.c */
> #if IS_ENABLED(CONFIG_PCI_IOV)
> int aie4_sriov_configure(struct amdxdna_dev *xdna, int num_vfs);
> diff --git a/drivers/accel/amdxdna/amdxdna_ctx.h b/drivers/accel/amdxdna/amdxdna_ctx.h
> index 6e78bab8a02c..1529e7507fed 100644
> --- a/drivers/accel/amdxdna/amdxdna_ctx.h
> +++ b/drivers/accel/amdxdna/amdxdna_ctx.h
> @@ -6,9 +6,11 @@
> #ifndef _AMDXDNA_CTX_H_
> #define _AMDXDNA_CTX_H_
>
> +#include <drm/gpu_scheduler.h>
> #include <linux/bitfield.h>
>
> #include "amdxdna_gem.h"
> +#include "drm/amdxdna_accel.h"
#include <drm/amdxdna_accel.h>
Lizhi
>
> struct amdxdna_hwctx_priv;
>
next prev parent reply other threads:[~2026-09-28 20:16 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-26 1:34 [PATCH V0 00/21] accel/amdxdna: Kernel submission and PM for AIE4 David Zhang
2026-09-26 1:34 ` [PATCH V0 01/21] accel/amdxdna: Rename NPU3 firmware files David Zhang
2026-09-26 1:34 ` [PATCH V0 02/21] accel/amdxdna: Remove mmap for doorbell David Zhang
2026-09-26 1:34 ` [PATCH V0 03/21] accel/amdxdna: Add CERT firmware version support David Zhang
2026-09-26 1:34 ` [PATCH V0 04/21] accel/amdxdna: Upgrade firmware version to 6.0 David Zhang
2026-09-28 17:08 ` Lizhi Hou
2026-09-26 1:34 ` [PATCH V0 05/21] accel/amdxdna: Add NPU3 classic device support David Zhang
2026-09-26 1:34 ` [PATCH V0 06/21] accel/amdxdna: Add AIE version query to aie4_get_info David Zhang
2026-09-26 1:34 ` [PATCH V0 07/21] accel/amdxdna: Add get and set power_mode for AIE4 David Zhang
2026-09-28 18:39 ` Lizhi Hou
2026-09-26 1:34 ` [PATCH V0 08/21] accel/amdxdna: Restore power mode override on AIE4 hardware start David Zhang
2026-09-28 18:48 ` Lizhi Hou
2026-09-26 1:34 ` [PATCH V0 09/21] accel/amdxdna: Add clock, DPM frequency, and resource info queries for AIE4 David Zhang
2026-09-26 1:34 ` [PATCH V0 10/21] accel/amdxdna: Add context switch hysteresis with debugfs control David Zhang
2026-09-26 1:34 ` [PATCH V0 11/21] accel/amdxdna: Refactor AIE4 hardware initialization sequence David Zhang
2026-09-26 1:34 ` [PATCH V0 12/21] accel/amdxdna: Decouple AIE4 doorbell and MSI-X notification transport hooks David Zhang
2026-09-28 20:16 ` Lizhi Hou [this message]
2026-09-26 1:34 ` [PATCH V0 13/21] accel/amdxdna: Implement AIE4 kernel queue lifecycle and memory layout David Zhang
2026-09-26 1:34 ` [PATCH V0 14/21] accel/amdxdna: Fix fence timeline name and context allocation David Zhang
2026-09-28 21:06 ` Lizhi Hou
2026-09-30 0:20 ` Zhang, Yidong (David)
2026-09-26 1:34 ` [PATCH V0 15/21] accel/amdxdna: Prepare for AIE4 command submission David Zhang
2026-09-26 1:34 ` [PATCH V0 16/21] accel/amdxdna: Implement AIE4 command packet building and submission David Zhang
2026-09-26 1:34 ` [PATCH V0 17/21] accel/amdxdna: Finalize runtime PM before acquiring dev_lock on removal David Zhang
2026-09-26 1:34 ` [PATCH V0 18/21] accel/amdxdna: Implement AIE4 suspend and resume David Zhang
2026-09-26 1:34 ` [PATCH V0 19/21] accel/amdxdna: Link SR-IOV VFs for power management sequencing David Zhang
2026-09-26 1:34 ` [PATCH V0 20/21] accel/amdxdna: Implement runtime suspend and resume support David Zhang
2026-09-26 1:34 ` [PATCH V0 21/21] accel/amdxdna: Add stub hwctx_config for AIE4 David Zhang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ebd56ccf-e7df-beb2-57ff-6fdf34b29a97@amd.com \
--to=lizhi.hou@amd.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=karol.wachowski@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mario.limonciello@amd.com \
--cc=max.zhen@amd.com \
--cc=ogabbay@kernel.org \
--cc=quic_jhugo@quicinc.com \
--cc=sonal.santan@amd.com \
--cc=wendy.liang@amd.com \
--cc=yidong.zhang@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.