From: "Michał Winiarski" <michal.winiarski@intel.com>
To: "Piórkowski, Piotr" <piotr.piorkowski@intel.com>
Cc: <intel-xe@lists.freedesktop.org>, <kvm@vger.kernel.org>,
Michal Wajdeczko <michal.wajdeczko@intel.com>
Subject: Re: [PATCH v2 1/2] drm/xe/pf: Add FLR_PREPARE state to VF control flow
Date: Mon, 23 Mar 2026 17:40:59 +0100 [thread overview]
Message-ID: <acFs3bwEkiynDkJZ@nostramo> (raw)
In-Reply-To: <20260309152449.910636-2-piotr.piorkowski@intel.com>
On Mon, Mar 09, 2026 at 04:24:48PM +0100, Piórkowski, Piotr wrote:
> From: Piotr Piórkowski <piotr.piorkowski@intel.com>
>
> Our xe-vfio-pci component relies on the confirmation from the PF
> that VF FLR processing has finished, but due to the notification
> latency on the HW/FW side, PF might be unaware yet of the already
> triggered VF FLR.
>
> Update VF state machine with new FLR_PREPARE state that indicate
> imminent VF FLR notification and treat that as a begin of the FLR
> sequence. Also introduce function that xe-vfio-pci should call to
> guarantee correct synchronization.
>
> v2: move PREPARE into WIP, update commit msg (Michal)
>
> Signed-off-by: Piotr Piórkowski <piotr.piorkowski@intel.com>
> Co-developed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
> Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Michał Winiarski <michal.winiarski@intel.com>
Thanks,
-Michał
> ---
> drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c | 78 +++++++++++++++----
> drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h | 1 +
> .../gpu/drm/xe/xe_gt_sriov_pf_control_types.h | 2 +
> drivers/gpu/drm/xe/xe_sriov_pf_control.c | 24 ++++++
> drivers/gpu/drm/xe/xe_sriov_pf_control.h | 1 +
> drivers/gpu/drm/xe/xe_sriov_vfio.c | 1 +
> include/drm/intel/xe_sriov_vfio.h | 11 +++
> 7 files changed, 102 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c b/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c
> index 5cb705c7ee7a..058585f063a9 100644
> --- a/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c
> +++ b/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.c
> @@ -171,6 +171,7 @@ static const char *control_bit_to_string(enum xe_gt_sriov_control_bits bit)
> case XE_GT_SRIOV_STATE_##_X: return #_X
> CASE2STR(WIP);
> CASE2STR(FLR_WIP);
> + CASE2STR(FLR_PREPARE);
> CASE2STR(FLR_SEND_START);
> CASE2STR(FLR_WAIT_GUC);
> CASE2STR(FLR_GUC_DONE);
> @@ -1486,11 +1487,15 @@ int xe_gt_sriov_pf_control_stop_vf(struct xe_gt *gt, unsigned int vfid)
> * The VF FLR state machine looks like::
> *
> * (READY,PAUSED,STOPPED)<------------<--------------o
> - * | \
> - * flr \
> - * | \
> - * ....V..........................FLR_WIP........... \
> - * : \ : \
> + * | | \
> + * flr prepare \
> + * | | \
> + * ....V.............V............FLR_WIP........... \
> + * : | | : \
> + * : | FLR_PREPARE : |
> + * : | / : |
> + * : \ flr : |
> + * : \ / : |
> * : \ o----<----busy : |
> * : \ / / : |
> * : FLR_SEND_START---failed----->-----------o--->(FLR_FAILED)<---o
> @@ -1539,20 +1544,28 @@ static void pf_enter_vf_flr_send_start(struct xe_gt *gt, unsigned int vfid)
> pf_queue_vf(gt, vfid);
> }
>
> -static void pf_enter_vf_flr_wip(struct xe_gt *gt, unsigned int vfid)
> +static bool pf_exit_vf_flr_prepare(struct xe_gt *gt, unsigned int vfid)
> {
> - if (!pf_enter_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_WIP)) {
> - xe_gt_sriov_dbg(gt, "VF%u FLR is already in progress\n", vfid);
> - return;
> - }
> + if (!pf_exit_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_PREPARE))
> + return false;
>
> - pf_enter_vf_wip(gt, vfid);
> pf_enter_vf_flr_send_start(gt, vfid);
> + return true;
> +}
> +
> +static bool pf_enter_vf_flr_wip(struct xe_gt *gt, unsigned int vfid)
> +{
> + if (!pf_enter_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_WIP))
> + return false;
> +
> + pf_enter_vf_wip(gt, vfid);
> + return true;
> }
>
> static void pf_exit_vf_flr_wip(struct xe_gt *gt, unsigned int vfid)
> {
> if (pf_exit_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_WIP)) {
> + pf_escape_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_PREPARE);
> pf_escape_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_SEND_FINISH);
> pf_escape_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_RESET_MMIO);
> pf_escape_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_RESET_DATA);
> @@ -1760,21 +1773,54 @@ static void pf_enter_vf_flr_guc_done(struct xe_gt *gt, unsigned int vfid)
> }
>
> /**
> - * xe_gt_sriov_pf_control_trigger_flr - Start a VF FLR sequence.
> + * xe_gt_sriov_pf_control_prepare_flr() - Notify PF that VF FLR request was issued.
> * @gt: the &xe_gt
> * @vfid: the VF identifier
> *
> + * This is an optional early notification path used to mark pending FLR before
> + * the GuC notifies the PF with a FLR event.
> + *
> * This function is for PF only.
> *
> * Return: 0 on success or a negative error code on failure.
> */
> -int xe_gt_sriov_pf_control_trigger_flr(struct xe_gt *gt, unsigned int vfid)
> +int xe_gt_sriov_pf_control_prepare_flr(struct xe_gt *gt, unsigned int vfid)
> {
> - pf_enter_vf_flr_wip(gt, vfid);
> + if (!pf_enter_vf_flr_wip(gt, vfid))
> + return -EALREADY;
>
> + pf_enter_vf_state(gt, vfid, XE_GT_SRIOV_STATE_FLR_PREPARE);
> return 0;
> }
>
> +static int pf_begin_vf_flr(struct xe_gt *gt, unsigned int vfid)
> +{
> + if (pf_enter_vf_flr_wip(gt, vfid)) {
> + pf_enter_vf_flr_send_start(gt, vfid);
> + return 0;
> + }
> +
> + if (pf_exit_vf_flr_prepare(gt, vfid))
> + return 0;
> +
> + xe_gt_sriov_dbg(gt, "VF%u FLR is already in progress\n", vfid);
> + return -EALREADY;
> +}
> +
> +/**
> + * xe_gt_sriov_pf_control_trigger_flr - Start a VF FLR sequence.
> + * @gt: the &xe_gt
> + * @vfid: the VF identifier
> + *
> + * This function is for PF only.
> + *
> + * Return: 0 on success or a negative error code on failure.
> + */
> +int xe_gt_sriov_pf_control_trigger_flr(struct xe_gt *gt, unsigned int vfid)
> +{
> + return pf_begin_vf_flr(gt, vfid);
> +}
> +
> /**
> * xe_gt_sriov_pf_control_sync_flr() - Synchronize on the VF FLR checkpoint.
> * @gt: the &xe_gt
> @@ -1879,9 +1925,9 @@ static void pf_handle_vf_flr(struct xe_gt *gt, u32 vfid)
>
> if (needs_dispatch_flr(xe)) {
> for_each_gt(gtit, xe, gtid)
> - pf_enter_vf_flr_wip(gtit, vfid);
> + pf_begin_vf_flr(gtit, vfid);
> } else {
> - pf_enter_vf_flr_wip(gt, vfid);
> + pf_begin_vf_flr(gt, vfid);
> }
> }
>
> diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h b/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h
> index c36c8767f3ad..23182a5c5fb8 100644
> --- a/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h
> +++ b/drivers/gpu/drm/xe/xe_gt_sriov_pf_control.h
> @@ -27,6 +27,7 @@ int xe_gt_sriov_pf_control_process_restore_data(struct xe_gt *gt, unsigned int v
> int xe_gt_sriov_pf_control_trigger_restore_vf(struct xe_gt *gt, unsigned int vfid);
> int xe_gt_sriov_pf_control_finish_restore_vf(struct xe_gt *gt, unsigned int vfid);
> int xe_gt_sriov_pf_control_stop_vf(struct xe_gt *gt, unsigned int vfid);
> +int xe_gt_sriov_pf_control_prepare_flr(struct xe_gt *gt, unsigned int vfid);
> int xe_gt_sriov_pf_control_trigger_flr(struct xe_gt *gt, unsigned int vfid);
> int xe_gt_sriov_pf_control_sync_flr(struct xe_gt *gt, unsigned int vfid, bool sync);
> int xe_gt_sriov_pf_control_wait_flr(struct xe_gt *gt, unsigned int vfid);
> diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_pf_control_types.h b/drivers/gpu/drm/xe/xe_gt_sriov_pf_control_types.h
> index 6027ba05a7f2..e78c59e08adf 100644
> --- a/drivers/gpu/drm/xe/xe_gt_sriov_pf_control_types.h
> +++ b/drivers/gpu/drm/xe/xe_gt_sriov_pf_control_types.h
> @@ -15,6 +15,7 @@
> *
> * @XE_GT_SRIOV_STATE_WIP: indicates that some operations are in progress.
> * @XE_GT_SRIOV_STATE_FLR_WIP: indicates that a VF FLR is in progress.
> + * @XE_GT_SRIOV_STATE_FLR_PREPARE: indicates that the PF received early VF FLR prepare notification.
> * @XE_GT_SRIOV_STATE_FLR_SEND_START: indicates that the PF wants to send a FLR START command.
> * @XE_GT_SRIOV_STATE_FLR_WAIT_GUC: indicates that the PF awaits for a response from the GuC.
> * @XE_GT_SRIOV_STATE_FLR_GUC_DONE: indicates that the PF has received a response from the GuC.
> @@ -56,6 +57,7 @@ enum xe_gt_sriov_control_bits {
> XE_GT_SRIOV_STATE_WIP = 1,
>
> XE_GT_SRIOV_STATE_FLR_WIP,
> + XE_GT_SRIOV_STATE_FLR_PREPARE,
> XE_GT_SRIOV_STATE_FLR_SEND_START,
> XE_GT_SRIOV_STATE_FLR_WAIT_GUC,
> XE_GT_SRIOV_STATE_FLR_GUC_DONE,
> diff --git a/drivers/gpu/drm/xe/xe_sriov_pf_control.c b/drivers/gpu/drm/xe/xe_sriov_pf_control.c
> index ed4b9820b06e..15b4341d7f12 100644
> --- a/drivers/gpu/drm/xe/xe_sriov_pf_control.c
> +++ b/drivers/gpu/drm/xe/xe_sriov_pf_control.c
> @@ -123,6 +123,30 @@ int xe_sriov_pf_control_reset_vf(struct xe_device *xe, unsigned int vfid)
> return result;
> }
>
> +/**
> + * xe_sriov_pf_control_prepare_flr() - Notify PF that VF FLR prepare has started.
> + * @xe: the &xe_device
> + * @vfid: the VF identifier
> + *
> + * This function is for PF only.
> + *
> + * Return: 0 on success or a negative error code on failure.
> + */
> +int xe_sriov_pf_control_prepare_flr(struct xe_device *xe, unsigned int vfid)
> +{
> + struct xe_gt *gt;
> + unsigned int id;
> + int result = 0;
> + int err;
> +
> + for_each_gt(gt, xe, id) {
> + err = xe_gt_sriov_pf_control_prepare_flr(gt, vfid);
> + result = result ? -EUCLEAN : err;
> + }
> +
> + return result;
> +}
> +
> /**
> * xe_sriov_pf_control_wait_flr() - Wait for a VF reset (FLR) to complete.
> * @xe: the &xe_device
> diff --git a/drivers/gpu/drm/xe/xe_sriov_pf_control.h b/drivers/gpu/drm/xe/xe_sriov_pf_control.h
> index ef9f219b2109..74981a67db88 100644
> --- a/drivers/gpu/drm/xe/xe_sriov_pf_control.h
> +++ b/drivers/gpu/drm/xe/xe_sriov_pf_control.h
> @@ -12,6 +12,7 @@ int xe_sriov_pf_control_pause_vf(struct xe_device *xe, unsigned int vfid);
> int xe_sriov_pf_control_resume_vf(struct xe_device *xe, unsigned int vfid);
> int xe_sriov_pf_control_stop_vf(struct xe_device *xe, unsigned int vfid);
> int xe_sriov_pf_control_reset_vf(struct xe_device *xe, unsigned int vfid);
> +int xe_sriov_pf_control_prepare_flr(struct xe_device *xe, unsigned int vfid);
> int xe_sriov_pf_control_wait_flr(struct xe_device *xe, unsigned int vfid);
> int xe_sriov_pf_control_sync_flr(struct xe_device *xe, unsigned int vfid);
> int xe_sriov_pf_control_trigger_save_vf(struct xe_device *xe, unsigned int vfid);
> diff --git a/drivers/gpu/drm/xe/xe_sriov_vfio.c b/drivers/gpu/drm/xe/xe_sriov_vfio.c
> index 3da81af97b8b..00f96b0976d1 100644
> --- a/drivers/gpu/drm/xe/xe_sriov_vfio.c
> +++ b/drivers/gpu/drm/xe/xe_sriov_vfio.c
> @@ -42,6 +42,7 @@ _type xe_sriov_vfio_##_func(struct xe_device *xe, unsigned int vfid) \
> EXPORT_SYMBOL_FOR_MODULES(xe_sriov_vfio_##_func, "xe-vfio-pci")
>
> DEFINE_XE_SRIOV_VFIO_FUNCTION(int, wait_flr_done, control_wait_flr);
> +DEFINE_XE_SRIOV_VFIO_FUNCTION(int, flr_prepare, control_prepare_flr);
> DEFINE_XE_SRIOV_VFIO_FUNCTION(int, suspend_device, control_pause_vf);
> DEFINE_XE_SRIOV_VFIO_FUNCTION(int, resume_device, control_resume_vf);
> DEFINE_XE_SRIOV_VFIO_FUNCTION(int, stop_copy_enter, control_trigger_save_vf);
> diff --git a/include/drm/intel/xe_sriov_vfio.h b/include/drm/intel/xe_sriov_vfio.h
> index e9814e8149fd..27c224a70e6f 100644
> --- a/include/drm/intel/xe_sriov_vfio.h
> +++ b/include/drm/intel/xe_sriov_vfio.h
> @@ -27,6 +27,17 @@ struct xe_device *xe_sriov_vfio_get_pf(struct pci_dev *pdev);
> */
> bool xe_sriov_vfio_migration_supported(struct xe_device *xe);
>
> +/**
> + * xe_sriov_vfio_flr_prepare() - Notify PF that VF FLR prepare has started.
> + * @xe: the PF &xe_device obtained by calling xe_sriov_vfio_get_pf()
> + * @vfid: the VF identifier (can't be 0)
> + *
> + * This function marks VF FLR as pending before PF receives GuC FLR event.
> + *
> + * Return: 0 on success or a negative error code on failure.
> + */
> +int xe_sriov_vfio_flr_prepare(struct xe_device *xe, unsigned int vfid);
> +
> /**
> * xe_sriov_vfio_wait_flr_done() - Wait for VF FLR completion.
> * @xe: the PF &xe_device obtained by calling xe_sriov_vfio_get_pf()
> --
> 2.34.1
>
next prev parent reply other threads:[~2026-03-23 16:41 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-09 15:24 [PATCH v2 0/2] Improve VF FLR synchronization for Xe VFIO Piórkowski, Piotr
2026-03-09 15:24 ` [PATCH v2 1/2] drm/xe/pf: Add FLR_PREPARE state to VF control flow Piórkowski, Piotr
2026-03-23 16:40 ` Michał Winiarski [this message]
2026-03-09 15:24 ` [PATCH v2 2/2] vfio/xe: Notify PF about VF FLR in reset_prepare Piórkowski, Piotr
2026-03-17 14:43 ` Alex Williamson
2026-03-23 16:38 ` Michał Winiarski
2026-03-09 15:34 ` ✓ CI.KUnit: success for Improve VF FLR synchronization for Xe VFIO (rev2) Patchwork
2026-03-09 16:37 ` ✓ Xe.CI.BAT: " Patchwork
2026-03-09 20:16 ` ✗ Xe.CI.FULL: failure " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=acFs3bwEkiynDkJZ@nostramo \
--to=michal.winiarski@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=kvm@vger.kernel.org \
--cc=michal.wajdeczko@intel.com \
--cc=piotr.piorkowski@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox