All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yahui Cao <yahui.cao@intel.com>
To: intel-wired-lan@lists.osuosl.org
Cc: kevin.tian@intel.com, yishaih@nvidia.com, brett.creeley@amd.com,
	kvm@vger.kernel.org, sridhar.samudrala@intel.com,
	edumazet@google.com, shameerali.kolothum.thodi@huawei.com,
	alex.williamson@redhat.com, madhu.chittim@intel.com,
	jgg@nvidia.com, netdev@vger.kernel.org, kuba@kernel.org,
	pabeni@redhat.com, davem@davemloft.net
Subject: [Intel-wired-lan] [PATCH iwl-next v4 10/12] ice: Add device suspend function for migration
Date: Tue, 21 Nov 2023 02:51:09 +0000	[thread overview]
Message-ID: <20231121025111.257597-11-yahui.cao@intel.com> (raw)
In-Reply-To: <20231121025111.257597-1-yahui.cao@intel.com>

From: Lingyu Liu <lingyu.liu@intel.com>

Device suspend handler is called by vfio driver before saving device
state. Typical operation includes stopping TX/RX queue.

Signed-off-by: Lingyu Liu <lingyu.liu@intel.com>
Signed-off-by: Yahui Cao <yahui.cao@intel.com>
---
 .../net/ethernet/intel/ice/ice_migration.c    | 69 +++++++++++++++++++
 include/linux/net/intel/ice_migration.h       |  6 ++
 2 files changed, 75 insertions(+)

diff --git a/drivers/net/ethernet/intel/ice/ice_migration.c b/drivers/net/ethernet/intel/ice/ice_migration.c
index 082ae2b79f60..a11cd0d3ad3d 100644
--- a/drivers/net/ethernet/intel/ice/ice_migration.c
+++ b/drivers/net/ethernet/intel/ice/ice_migration.c
@@ -2,6 +2,8 @@
 /* Copyright (C) 2018-2023 Intel Corporation */
 
 #include "ice.h"
+#include "ice_lib.h"
+#include "ice_fltr.h"
 #include "ice_base.h"
 #include "ice_txrx_lib.h"
 
@@ -274,6 +276,73 @@ u32 ice_migration_supported_caps(void)
 	return VIRTCHNL_VF_MIGRATION_SUPPORT_FEATURE;
 }
 
+/**
+ * ice_migration_suspend_dev - suspend device
+ * @pf: pointer to PF of migration device
+ * @vf_id: VF index of migration device
+ *
+ * Return 0 for success, negative for error
+ */
+int ice_migration_suspend_dev(struct ice_pf *pf, int vf_id)
+{
+	struct device *dev = ice_pf_to_dev(pf);
+	struct ice_vsi *vsi;
+	struct ice_vf *vf;
+	int ret;
+
+	vf = ice_get_vf_by_id(pf, vf_id);
+	if (!vf) {
+		dev_err(dev, "Unable to locate VF from VF ID%d\n", vf_id);
+		return -EINVAL;
+	}
+
+	if (!test_bit(ICE_VF_STATE_QS_ENA, vf->vf_states)) {
+		ret = 0;
+		goto out_put_vf;
+	}
+
+	if (vf->virtchnl_msg_num > VIRTCHNL_MSG_MAX) {
+		dev_err(dev, "SR-IOV live migration disabled on VF %d. Migration buffer exceeded\n",
+			vf->vf_id);
+		ret = -EIO;
+		goto out_put_vf;
+	}
+
+	vsi = ice_get_vf_vsi(vf);
+	if (!vsi) {
+		dev_err(dev, "VF %d VSI is NULL\n", vf->vf_id);
+		ret = -EINVAL;
+		goto out_put_vf;
+	}
+
+	/* Prevent VSI from queuing incoming packets by removing all filters */
+	ice_fltr_remove_all(vsi);
+
+	/* MAC based filter rule is disabled at this point. Set MAC to zero
+	 * to keep consistency with VF mac address info shown by ip link
+	 */
+	eth_zero_addr(vf->hw_lan_addr);
+	eth_zero_addr(vf->dev_lan_addr);
+
+	ret = ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, vf->vf_id);
+	if (ret) {
+		dev_err(dev, "VF %d failed to stop tx rings\n", vf->vf_id);
+		ret = -EIO;
+		goto out_put_vf;
+	}
+	ret = ice_vsi_stop_all_rx_rings(vsi);
+	if (ret) {
+		dev_err(dev, "VF %d failed to stop rx rings\n", vf->vf_id);
+		ret = -EIO;
+		goto out_put_vf;
+	}
+
+out_put_vf:
+	ice_put_vf(vf);
+	return ret;
+}
+EXPORT_SYMBOL(ice_migration_suspend_dev);
+
 /**
  * ice_migration_save_rx_head - save rx head into device state buffer
  * @vf: pointer to VF structure
diff --git a/include/linux/net/intel/ice_migration.h b/include/linux/net/intel/ice_migration.h
index a142b78283a8..47f46dca07ae 100644
--- a/include/linux/net/intel/ice_migration.h
+++ b/include/linux/net/intel/ice_migration.h
@@ -14,6 +14,7 @@ int ice_migration_save_devstate(struct ice_pf *pf, int vf_id,
 				u8 *buf, u64 buf_sz);
 int ice_migration_load_devstate(struct ice_pf *pf, int vf_id,
 				const u8 *buf, u64 buf_sz);
+int ice_migration_suspend_dev(struct ice_pf *pf, int vf_id);
 #else
 static inline struct ice_pf *ice_migration_get_pf(struct pci_dev *pdev)
 {
@@ -37,6 +38,11 @@ static inline int ice_migration_load_devstate(struct ice_pf *pf, int vf_id,
 {
 	return 0;
 }
+
+static inline int ice_migration_suspend_dev(struct ice_pf *pf, int vf_id)
+{
+	return 0;
+}
 #endif /* CONFIG_ICE_VFIO_PCI */
 
 #endif /* _ICE_MIGRATION_H_ */
-- 
2.34.1

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

WARNING: multiple messages have this Message-ID (diff)
From: Yahui Cao <yahui.cao@intel.com>
To: intel-wired-lan@lists.osuosl.org
Cc: kvm@vger.kernel.org, netdev@vger.kernel.org,
	lingyu.liu@intel.com, kevin.tian@intel.com,
	madhu.chittim@intel.com, sridhar.samudrala@intel.com,
	alex.williamson@redhat.com, jgg@nvidia.com, yishaih@nvidia.com,
	shameerali.kolothum.thodi@huawei.com, brett.creeley@amd.com,
	davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
	pabeni@redhat.com
Subject: [PATCH iwl-next v4 10/12] ice: Add device suspend function for migration
Date: Tue, 21 Nov 2023 02:51:09 +0000	[thread overview]
Message-ID: <20231121025111.257597-11-yahui.cao@intel.com> (raw)
In-Reply-To: <20231121025111.257597-1-yahui.cao@intel.com>

From: Lingyu Liu <lingyu.liu@intel.com>

Device suspend handler is called by vfio driver before saving device
state. Typical operation includes stopping TX/RX queue.

Signed-off-by: Lingyu Liu <lingyu.liu@intel.com>
Signed-off-by: Yahui Cao <yahui.cao@intel.com>
---
 .../net/ethernet/intel/ice/ice_migration.c    | 69 +++++++++++++++++++
 include/linux/net/intel/ice_migration.h       |  6 ++
 2 files changed, 75 insertions(+)

diff --git a/drivers/net/ethernet/intel/ice/ice_migration.c b/drivers/net/ethernet/intel/ice/ice_migration.c
index 082ae2b79f60..a11cd0d3ad3d 100644
--- a/drivers/net/ethernet/intel/ice/ice_migration.c
+++ b/drivers/net/ethernet/intel/ice/ice_migration.c
@@ -2,6 +2,8 @@
 /* Copyright (C) 2018-2023 Intel Corporation */
 
 #include "ice.h"
+#include "ice_lib.h"
+#include "ice_fltr.h"
 #include "ice_base.h"
 #include "ice_txrx_lib.h"
 
@@ -274,6 +276,73 @@ u32 ice_migration_supported_caps(void)
 	return VIRTCHNL_VF_MIGRATION_SUPPORT_FEATURE;
 }
 
+/**
+ * ice_migration_suspend_dev - suspend device
+ * @pf: pointer to PF of migration device
+ * @vf_id: VF index of migration device
+ *
+ * Return 0 for success, negative for error
+ */
+int ice_migration_suspend_dev(struct ice_pf *pf, int vf_id)
+{
+	struct device *dev = ice_pf_to_dev(pf);
+	struct ice_vsi *vsi;
+	struct ice_vf *vf;
+	int ret;
+
+	vf = ice_get_vf_by_id(pf, vf_id);
+	if (!vf) {
+		dev_err(dev, "Unable to locate VF from VF ID%d\n", vf_id);
+		return -EINVAL;
+	}
+
+	if (!test_bit(ICE_VF_STATE_QS_ENA, vf->vf_states)) {
+		ret = 0;
+		goto out_put_vf;
+	}
+
+	if (vf->virtchnl_msg_num > VIRTCHNL_MSG_MAX) {
+		dev_err(dev, "SR-IOV live migration disabled on VF %d. Migration buffer exceeded\n",
+			vf->vf_id);
+		ret = -EIO;
+		goto out_put_vf;
+	}
+
+	vsi = ice_get_vf_vsi(vf);
+	if (!vsi) {
+		dev_err(dev, "VF %d VSI is NULL\n", vf->vf_id);
+		ret = -EINVAL;
+		goto out_put_vf;
+	}
+
+	/* Prevent VSI from queuing incoming packets by removing all filters */
+	ice_fltr_remove_all(vsi);
+
+	/* MAC based filter rule is disabled at this point. Set MAC to zero
+	 * to keep consistency with VF mac address info shown by ip link
+	 */
+	eth_zero_addr(vf->hw_lan_addr);
+	eth_zero_addr(vf->dev_lan_addr);
+
+	ret = ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, vf->vf_id);
+	if (ret) {
+		dev_err(dev, "VF %d failed to stop tx rings\n", vf->vf_id);
+		ret = -EIO;
+		goto out_put_vf;
+	}
+	ret = ice_vsi_stop_all_rx_rings(vsi);
+	if (ret) {
+		dev_err(dev, "VF %d failed to stop rx rings\n", vf->vf_id);
+		ret = -EIO;
+		goto out_put_vf;
+	}
+
+out_put_vf:
+	ice_put_vf(vf);
+	return ret;
+}
+EXPORT_SYMBOL(ice_migration_suspend_dev);
+
 /**
  * ice_migration_save_rx_head - save rx head into device state buffer
  * @vf: pointer to VF structure
diff --git a/include/linux/net/intel/ice_migration.h b/include/linux/net/intel/ice_migration.h
index a142b78283a8..47f46dca07ae 100644
--- a/include/linux/net/intel/ice_migration.h
+++ b/include/linux/net/intel/ice_migration.h
@@ -14,6 +14,7 @@ int ice_migration_save_devstate(struct ice_pf *pf, int vf_id,
 				u8 *buf, u64 buf_sz);
 int ice_migration_load_devstate(struct ice_pf *pf, int vf_id,
 				const u8 *buf, u64 buf_sz);
+int ice_migration_suspend_dev(struct ice_pf *pf, int vf_id);
 #else
 static inline struct ice_pf *ice_migration_get_pf(struct pci_dev *pdev)
 {
@@ -37,6 +38,11 @@ static inline int ice_migration_load_devstate(struct ice_pf *pf, int vf_id,
 {
 	return 0;
 }
+
+static inline int ice_migration_suspend_dev(struct ice_pf *pf, int vf_id)
+{
+	return 0;
+}
 #endif /* CONFIG_ICE_VFIO_PCI */
 
 #endif /* _ICE_MIGRATION_H_ */
-- 
2.34.1


  parent reply	other threads:[~2023-11-21  2:50 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-21  2:50 [Intel-wired-lan] [PATCH iwl-next v4 00/12] Add E800 live migration driver Yahui Cao
2023-11-21  2:50 ` Yahui Cao
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 01/12] ice: Add function to get RX queue context Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-08 22:01   ` [Intel-wired-lan] " Brett Creeley
2023-12-08 22:01     ` Brett Creeley
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 02/12] ice: Add function to get and set TX " Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-08 22:14   ` [Intel-wired-lan] " Brett Creeley
2023-12-08 22:14     ` Brett Creeley
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 03/12] ice: Introduce VF state ICE_VF_STATE_REPLAYING_VC for migration Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-08 22:28   ` [Intel-wired-lan] " Brett Creeley
2023-12-08 22:28     ` Brett Creeley
2024-02-12 23:07     ` [Intel-wired-lan] " Jacob Keller
2024-02-12 23:07       ` Jacob Keller
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 04/12] ice: Add fundamental migration init and exit function Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 05/12] ice: Log virtual channel messages in PF Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-11-29 17:12   ` [Intel-wired-lan] " Simon Horman
2023-11-29 17:12     ` Simon Horman
2023-12-01  8:27     ` [Intel-wired-lan] " Cao, Yahui
2023-12-01  8:27       ` Cao, Yahui
2023-12-07  7:33   ` [Intel-wired-lan] " Tian, Kevin
2023-12-07  7:33     ` Tian, Kevin
2023-12-08  1:53   ` [Intel-wired-lan] " Brett Creeley
2023-12-08  1:53     ` Brett Creeley
2024-01-18 22:14     ` [Intel-wired-lan] " Jacob Keller
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 06/12] ice: Add device state save/load function for migration Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-07  7:39   ` [Intel-wired-lan] " Tian, Kevin
2023-12-07  7:39     ` Tian, Kevin
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 07/12] ice: Fix VSI id in virtual channel message " Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-07  7:42   ` [Intel-wired-lan] " Tian, Kevin
2023-12-07  7:42     ` Tian, Kevin
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 08/12] ice: Save and load RX Queue head Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-07  7:55   ` [Intel-wired-lan] " Tian, Kevin
2023-12-07  7:55     ` Tian, Kevin
2023-12-07 14:46     ` [Intel-wired-lan] " Jason Gunthorpe
2023-12-07 14:46       ` Jason Gunthorpe
2023-12-08  2:53       ` [Intel-wired-lan] " Tian, Kevin
2023-12-08  2:53         ` Tian, Kevin
2024-01-18 22:17         ` [Intel-wired-lan] " Jacob Keller
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 09/12] ice: Save and load TX " Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-07  8:22   ` [Intel-wired-lan] " Tian, Kevin
2023-12-07  8:22     ` Tian, Kevin
2023-12-07 14:48     ` [Intel-wired-lan] " Jason Gunthorpe
2023-12-07 14:48       ` Jason Gunthorpe
2023-11-21  2:51 ` Yahui Cao [this message]
2023-11-21  2:51   ` [PATCH iwl-next v4 10/12] ice: Add device suspend function for migration Yahui Cao
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 11/12] ice: Save and load mmio registers Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-11-21  2:51 ` [Intel-wired-lan] [PATCH iwl-next v4 12/12] vfio/ice: Implement vfio_pci driver for E800 devices Yahui Cao
2023-11-21  2:51   ` Yahui Cao
2023-12-07 22:43   ` [Intel-wired-lan] " Alex Williamson
2023-12-07 22:43     ` Alex Williamson
2023-12-08  3:42     ` [Intel-wired-lan] " Tian, Kevin
2023-12-08  3:42       ` Tian, Kevin
2023-12-08  3:42       ` [Intel-wired-lan] " Tian, Kevin
2023-12-08  3:42         ` Tian, Kevin
2023-12-04 11:18 ` [Intel-wired-lan] [PATCH iwl-next v4 00/12] Add E800 live migration driver Cao, Yahui
2023-12-04 11:18   ` Cao, Yahui
2024-01-18 22:09 ` [Intel-wired-lan] " Jacob Keller
2024-01-18 22:09   ` Jacob Keller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231121025111.257597-11-yahui.cao@intel.com \
    --to=yahui.cao@intel.com \
    --cc=alex.williamson@redhat.com \
    --cc=brett.creeley@amd.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=jgg@nvidia.com \
    --cc=kevin.tian@intel.com \
    --cc=kuba@kernel.org \
    --cc=kvm@vger.kernel.org \
    --cc=madhu.chittim@intel.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=shameerali.kolothum.thodi@huawei.com \
    --cc=sridhar.samudrala@intel.com \
    --cc=yishaih@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.