[PATCH v9 0/5] Support system sleep with offloaded usb transfers

[PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Guan-Yu Lin]

Wesley Cheng and Mathias Nyman's USB offload design enables a co-processor
to handle some USB transfers, potentially allowing the main system to
sleep and save power. However, Linux's power management system halts the
USB host controller when the main system isn't managing any USB transfers.
To address this, the proposal modifies the system to recognize offloaded
USB transfers and manage power accordingly.

This involves two key steps:
1. Transfer Status Tracking: Propose xhci_sideband_get and
xhci_sideband_put to track USB transfers on the co-processor, ensuring the
system is aware of any ongoing activity.
2. Power Management Adjustment:  Modifications to the USB driver stack
(dwc3 controller driver, xhci host controller driver, and USB device
drivers) allow the system to sleep without disrupting co-processor managed
USB transfers. This involves adding conditional checks to bypass some
power management operations.

patches depends on series "Introduce QC USB SND audio offloading support" 
https://lore.kernel.org/lkml/20241213235403.4109199-1-quic_wcheng@quicinc.com/T/

changelog
----------
Changes in v9:
- Remove unnecessary white space change.

Changes in v8:
- Change the runtime pm api to correct the error handling flow.
- Not flushing endpoints of actively offloaded USB devices to avoid
  possible USB transfer conflicts.

Changes in v7:
- Remove counting mechanism in struct usb_hcd. The USB device's offload
  status will be solely recorded in each related struct usb_device.
- Utilizes `needs_remote_wakeup` attribute in struct usb_interface to
  control the suspend flow of USB interfaces and associated USB endpoints.
  This addresses the need to support interrupt transfers generated by
  offloaded USB devices while the system is suspended.
- Block any offload_usage change during USB device suspend period.

Changes in v6:
- Fix build errors when CONFIG_USB_XHCI_SIDEBAND is disabled.
- Explicitly specify the data structure of the drvdata refereced in
  dwc3_suspend(), dwc3_resume().
- Move the initialization of counters to the patches introducing them.

Changes in v5:
- Walk through the USB children in usb_sideband_check() to determine the
  sideband activity under the specific USB device. 
- Replace atomic_t by refcount_t.
- Reduce logs by using dev_dbg & remove __func__.

Changes in v4:
- Isolate the feature into USB driver stack.
- Integrate with series "Introduce QC USB SND audio offloading support"

Changes in v3:
- Integrate the feature with the pm core framework.

Changes in v2:
- Cosmetics changes on coding style.

[v3] PM / core: conditionally skip system pm in device/driver model
[v2] usb: host: enable suspend-to-RAM control in userspace
[v1] [RFC] usb: host: Allow userspace to control usb suspend flows
---

Guan-Yu Lin (5):
  usb: dwc3: separate dev_pm_ops for each pm_event
  usb: xhci-plat: separate dev_pm_ops for each pm_event
  usb: add apis for offload usage tracking
  xhci: sideband: add api to trace sideband usage
  usb: host: enable USB offload during system sleep

 drivers/usb/core/driver.c         | 131 +++++++++++++++++++++++++++++-
 drivers/usb/core/endpoint.c       |   8 --
 drivers/usb/core/usb.c            |   4 +
 drivers/usb/dwc3/core.c           | 105 +++++++++++++++++++++++-
 drivers/usb/dwc3/core.h           |   1 +
 drivers/usb/host/xhci-plat.c      |  42 +++++++++-
 drivers/usb/host/xhci-sideband.c  |  82 +++++++++++++++++++
 include/linux/usb.h               |  27 +++++-
 include/linux/usb/hcd.h           |   7 ++
 include/linux/usb/xhci-sideband.h |   6 ++
 sound/usb/qcom/qc_audio_offload.c |   3 +
 11 files changed, 398 insertions(+), 18 deletions(-)

-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v9 1/5] usb: dwc3: separate dev_pm_ops for each pm_event

[Guan-Yu Lin]

Separate dev_pm_ops for different power events such as suspend, thaw,
and hibernation. This is crucial when dwc3 driver needs to adapt its
behavior based on different power state changes.

Signed-off-by: Guan-Yu Lin <guanyulin@google.com>
---
 drivers/usb/dwc3/core.c | 77 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 76 insertions(+), 1 deletion(-)

diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index 6c960ff30c92..0735881d4650 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -2632,6 +2632,76 @@ static int dwc3_resume(struct device *dev)
 	return ret;
 }
 
+static int dwc3_freeze(struct device *dev)
+{
+	struct dwc3	*dwc = dev_get_drvdata(dev);
+	int		ret;
+
+	ret = dwc3_suspend_common(dwc, PMSG_FREEZE);
+	if (ret)
+		return ret;
+
+	pinctrl_pm_select_sleep_state(dev);
+
+	return 0;
+}
+
+static int dwc3_thaw(struct device *dev)
+{
+	struct dwc3	*dwc = dev_get_drvdata(dev);
+	int		ret;
+
+	pinctrl_pm_select_default_state(dev);
+
+	pm_runtime_disable(dev);
+	pm_runtime_set_active(dev);
+
+	ret = dwc3_resume_common(dwc, PMSG_THAW);
+	if (ret) {
+		pm_runtime_set_suspended(dev);
+		return ret;
+	}
+
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
+static int dwc3_poweroff(struct device *dev)
+{
+	struct dwc3	*dwc = dev_get_drvdata(dev);
+	int		ret;
+
+	ret = dwc3_suspend_common(dwc, PMSG_HIBERNATE);
+	if (ret)
+		return ret;
+
+	pinctrl_pm_select_sleep_state(dev);
+
+	return 0;
+}
+
+static int dwc3_restore(struct device *dev)
+{
+	struct dwc3	*dwc = dev_get_drvdata(dev);
+	int		ret;
+
+	pinctrl_pm_select_default_state(dev);
+
+	pm_runtime_disable(dev);
+	pm_runtime_set_active(dev);
+
+	ret = dwc3_resume_common(dwc, PMSG_RESTORE);
+	if (ret) {
+		pm_runtime_set_suspended(dev);
+		return ret;
+	}
+
+	pm_runtime_enable(dev);
+
+	return 0;
+}
+
 static void dwc3_complete(struct device *dev)
 {
 	struct dwc3	*dwc = dev_get_drvdata(dev);
@@ -2649,7 +2719,12 @@ static void dwc3_complete(struct device *dev)
 #endif /* CONFIG_PM_SLEEP */
 
 static const struct dev_pm_ops dwc3_dev_pm_ops = {
-	SET_SYSTEM_SLEEP_PM_OPS(dwc3_suspend, dwc3_resume)
+	.suspend = pm_sleep_ptr(dwc3_suspend),
+	.resume = pm_sleep_ptr(dwc3_resume),
+	.freeze = pm_sleep_ptr(dwc3_freeze),
+	.thaw = pm_sleep_ptr(dwc3_thaw),
+	.poweroff = pm_sleep_ptr(dwc3_poweroff),
+	.restore = pm_sleep_ptr(dwc3_restore),
 	.complete = dwc3_complete,
 
 	/*
-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v9 2/5] usb: xhci-plat: separate dev_pm_ops for each pm_event

[Guan-Yu Lin]

Separate dev_pm_ops for different power events such as suspend, thaw,
and hibernation. This is crucial when xhci-plat driver needs to adapt
its behavior based on different power state changes.

Signed-off-by: Guan-Yu Lin <guanyulin@google.com>
---
 drivers/usb/host/xhci-plat.c | 28 ++++++++++++++++++++++++----
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c
index 3acdbbd9aea3..b676d4dbcec1 100644
--- a/drivers/usb/host/xhci-plat.c
+++ b/drivers/usb/host/xhci-plat.c
@@ -450,7 +450,7 @@ void xhci_plat_remove(struct platform_device *dev)
 }
 EXPORT_SYMBOL_GPL(xhci_plat_remove);
 
-static int xhci_plat_suspend(struct device *dev)
+static int xhci_plat_suspend_common(struct device *dev, struct pm_message pmsg)
 {
 	struct usb_hcd	*hcd = dev_get_drvdata(dev);
 	struct xhci_hcd	*xhci = hcd_to_xhci(hcd);
@@ -478,6 +478,21 @@ static int xhci_plat_suspend(struct device *dev)
 	return 0;
 }
 
+static int xhci_plat_suspend(struct device *dev)
+{
+	return xhci_plat_suspend_common(dev, PMSG_SUSPEND);
+}
+
+static int xhci_plat_freeze(struct device *dev)
+{
+	return xhci_plat_suspend_common(dev, PMSG_FREEZE);
+}
+
+static int xhci_plat_poweroff(struct device *dev)
+{
+	return xhci_plat_suspend_common(dev, PMSG_HIBERNATE);
+}
+
 static int xhci_plat_resume_common(struct device *dev, struct pm_message pmsg)
 {
 	struct usb_hcd	*hcd = dev_get_drvdata(dev);
@@ -524,6 +539,11 @@ static int xhci_plat_resume(struct device *dev)
 	return xhci_plat_resume_common(dev, PMSG_RESUME);
 }
 
+static int xhci_plat_thaw(struct device *dev)
+{
+	return xhci_plat_resume_common(dev, PMSG_THAW);
+}
+
 static int xhci_plat_restore(struct device *dev)
 {
 	return xhci_plat_resume_common(dev, PMSG_RESTORE);
@@ -553,9 +573,9 @@ static int __maybe_unused xhci_plat_runtime_resume(struct device *dev)
 const struct dev_pm_ops xhci_plat_pm_ops = {
 	.suspend = pm_sleep_ptr(xhci_plat_suspend),
 	.resume = pm_sleep_ptr(xhci_plat_resume),
-	.freeze = pm_sleep_ptr(xhci_plat_suspend),
-	.thaw = pm_sleep_ptr(xhci_plat_resume),
-	.poweroff = pm_sleep_ptr(xhci_plat_suspend),
+	.freeze = pm_sleep_ptr(xhci_plat_freeze),
+	.thaw = pm_sleep_ptr(xhci_plat_thaw),
+	.poweroff = pm_sleep_ptr(xhci_plat_poweroff),
 	.restore = pm_sleep_ptr(xhci_plat_restore),
 
 	SET_RUNTIME_PM_OPS(xhci_plat_runtime_suspend,
-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v9 3/5] usb: add apis for offload usage tracking

[Guan-Yu Lin]

Introduce offload_usage and corresponding apis to track offload usage
on each USB device. Offload denotes that there is another co-processor
accessing the USB device via the same USB host controller. To optimize
power usage, it's essential to monitor whether the USB device is
actively used by other co-processor. This information is vital when
determining if a USB device can be safely suspended during system power
state transitions.

Signed-off-by: Guan-Yu Lin <guanyulin@google.com>
---
 drivers/usb/core/driver.c | 108 ++++++++++++++++++++++++++++++++++++++
 drivers/usb/core/usb.c    |   4 ++
 include/linux/usb.h       |  19 +++++++
 3 files changed, 131 insertions(+)

diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c
index f203fdbfb6f6..1bbf9592724f 100644
--- a/drivers/usb/core/driver.c
+++ b/drivers/usb/core/driver.c
@@ -2037,6 +2037,114 @@ int usb_disable_usb2_hardware_lpm(struct usb_device *udev)
 
 #endif /* CONFIG_PM */
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+
+/**
+ * usb_offload_get - increment the offload_usage of a USB device
+ * @udev: the USB device to increment its offload_usage
+ *
+ * Incrementing the offload_usage of a usb_device indicates that offload is
+ * enabled on this usb_device; that is, another entity is actively handling USB
+ * transfers. This information allows the USB driver to adjust its power
+ * management policy based on offload activity.
+ *
+ * The caller must hold @udev's device lock.
+ *
+ * Return: 0 on success. A negative error code otherwise.
+ */
+int usb_offload_get(struct usb_device *udev)
+{
+	int ret;
+
+	if (udev->state == USB_STATE_NOTATTACHED ||
+			udev->state == USB_STATE_SUSPENDED)
+		return -EAGAIN;
+
+	/*
+	 * offload_usage could only be modified when the device is active, since
+	 * it will alter the suspend flow of the device.
+	 */
+	ret = pm_runtime_resume_and_get(&udev->dev);
+
+	if (ret < 0)
+		return ret;
+
+	refcount_inc(&udev->offload_usage);
+	pm_runtime_put_sync(&udev->dev);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(usb_offload_get);
+
+/**
+ * usb_offload_put - drop the offload_usage of a USB device
+ * @udev: the USB device to drop its offload_usage
+ *
+ * The inverse operation of usb_offload_get, which drops the offload_usage of
+ * a USB device. This information allows the USB driver to adjust its power
+ * management policy based on offload activity.
+ *
+ * The caller must hold @udev's device lock.
+ *
+ * Return: 0 on success. A negative error code otherwise.
+ */
+int usb_offload_put(struct usb_device *udev)
+{
+	int ret;
+
+	if (udev->state == USB_STATE_NOTATTACHED ||
+			udev->state == USB_STATE_SUSPENDED)
+		return -EAGAIN;
+
+	/*
+	 * offload_usage could only be modified when the device is active, since
+	 * it will alter the suspend flow of the device.
+	 */
+	ret = pm_runtime_resume_and_get(&udev->dev);
+
+	if (ret < 0)
+		return ret;
+
+	/* Drop the count when it wasn't 0, ignore the operation otherwise. */
+	ret = refcount_add_not_zero(-1, &udev->offload_usage);
+	pm_runtime_put_sync(&udev->dev);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(usb_offload_put);
+
+/**
+ * usb_offload_check - check offload activities on a USB device
+ * @udev: the USB device to check its offload activity.
+ *
+ * Check if there are any offload activity on the USB device right now. This
+ * information could be used for power management or other forms of resource
+ * management.
+ *
+ * The caller must hold @udev's device lock.
+ *
+ * Returns true on any offload activity, false otherwise.
+ */
+bool usb_offload_check(struct usb_device *udev)
+{
+	struct usb_device *child;
+	bool active;
+	int port1;
+
+	usb_hub_for_each_child(udev, port1, child) {
+		device_lock(&child->dev);
+		active = usb_offload_check(child);
+		device_unlock(&child->dev);
+		if (active)
+			return true;
+	}
+
+	return !!refcount_read(&udev->offload_usage);
+}
+EXPORT_SYMBOL_GPL(usb_offload_check);
+
+#endif /* CONFIG_USB_XHCI_SIDEBAND */
+
 const struct bus_type usb_bus_type = {
 	.name =		"usb",
 	.match =	usb_device_match,
diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index 0b4685aad2d5..888ab599fd06 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -672,6 +672,10 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
 	dev->lpm_disable_count = 1;
 	atomic_set(&dev->urbnum, 0);
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	refcount_set(&dev->offload_usage, 0);
+#endif
+
 	INIT_LIST_HEAD(&dev->ep0.urb_list);
 	dev->ep0.desc.bLength = USB_DT_ENDPOINT_SIZE;
 	dev->ep0.desc.bDescriptorType = USB_DT_ENDPOINT;
diff --git a/include/linux/usb.h b/include/linux/usb.h
index cfa8005e24f9..9b3f630e763e 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -645,6 +645,7 @@ struct usb3_lpm_parameters {
  *	parent->hub_delay + wHubDelay + tTPTransmissionDelay (40ns)
  *	Will be used as wValue for SetIsochDelay requests.
  * @use_generic_driver: ask driver core to reprobe using the generic driver.
+ * @offload_usage: number of offload activities happening on this usb device.
  *
  * Notes:
  * Usbcore drivers should not set usbdev->state directly.  Instead use
@@ -731,6 +732,10 @@ struct usb_device {
 
 	u16 hub_delay;
 	unsigned use_generic_driver:1;
+
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	refcount_t offload_usage;
+#endif
 };
 
 #define to_usb_device(__dev)	container_of_const(__dev, struct usb_device, dev)
@@ -837,6 +842,20 @@ static inline void usb_mark_last_busy(struct usb_device *udev)
 { }
 #endif
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+extern int usb_offload_get(struct usb_device *udev);
+extern int usb_offload_put(struct usb_device *udev);
+extern bool usb_offload_check(struct usb_device *udev);
+#else
+
+static inline int usb_offload_get(struct usb_device *udev)
+{ return 0; }
+static inline int usb_offload_put(struct usb_device *udev)
+{ return 0; }
+static inline bool usb_offload_check(struct usb_device *udev)
+{ return false; }
+#endif
+
 extern int usb_disable_lpm(struct usb_device *udev);
 extern void usb_enable_lpm(struct usb_device *udev);
 /* Same as above, but these functions lock/unlock the bandwidth_mutex. */
-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v9 4/5] xhci: sideband: add api to trace sideband usage

[Guan-Yu Lin]

The existing sideband driver only registers sidebands without tracking
their active usage. To address this, new apis are introduced to:
- mark sideband usage: record the sideband usage information in the USB
  device driver.
- query sideband status: provide a means for other drivers to fetch
  sideband activity information on a USB host controller.

Signed-off-by: Guan-Yu Lin <guanyulin@google.com>
---
 drivers/usb/host/xhci-sideband.c  | 82 +++++++++++++++++++++++++++++++
 include/linux/usb/xhci-sideband.h |  6 +++
 2 files changed, 88 insertions(+)

diff --git a/drivers/usb/host/xhci-sideband.c b/drivers/usb/host/xhci-sideband.c
index ffa6f1b825ff..d4931315dcf4 100644
--- a/drivers/usb/host/xhci-sideband.c
+++ b/drivers/usb/host/xhci-sideband.c
@@ -358,6 +358,88 @@ xhci_sideband_interrupter_id(struct xhci_sideband *sb)
 }
 EXPORT_SYMBOL_GPL(xhci_sideband_interrupter_id);
 
+/**
+ * xhci_sideband_get - record a new active sideband instance
+ * @sb: sideband instance for this usb device
+ *
+ * An active sideband indicates that another entity is currently using the host
+ * controller. Inform the usb_device associated with this sideband instance via
+ * usb_offload_get(). This allows the corresponding drivers to dynamically
+ * adjust power management actions based on current sideband activities.
+ *
+ * Returns 0 on success, negative error otherwise.
+ */
+int xhci_sideband_get(struct xhci_sideband *sb)
+{
+	struct usb_device *udev;
+	struct usb_hcd *hcd;
+	int ret = 0;
+
+	if (!sb || !sb->xhci)
+		return -ENODEV;
+
+	hcd = xhci_to_hcd(sb->xhci);
+	udev = sb->vdev->udev;
+	device_lock(&udev->dev);
+	ret = usb_offload_get(udev);
+	device_unlock(&udev->dev);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(xhci_sideband_get);
+
+/**
+ * xhci_sideband_put - record a deactivated sideband instance
+ * @sb: sideband instance for this usb device
+ *
+ * The inverse operation of xhci_sideband_get, which informs the associated
+ * usb_device via usb_offload_put(). This allows the corresponding drivers to
+ * dynamically adjust power management actions based on sideband activities.
+ *
+ * Returns 0 on success, negative error otherwise.
+ */
+int xhci_sideband_put(struct xhci_sideband *sb)
+{
+	struct usb_device *udev;
+	struct usb_hcd *hcd;
+	int ret = 0;
+
+	if (!sb || !sb->xhci)
+		return -ENODEV;
+
+	hcd = xhci_to_hcd(sb->xhci);
+	udev = sb->vdev->udev;
+	device_lock(&udev->dev);
+	ret = usb_offload_put(udev);
+	device_unlock(&udev->dev);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(xhci_sideband_put);
+
+/**
+ * xhci_sideband_check - check the existence of active sidebands
+ * @hcd: the host controller driver associated with the target host controller
+ *
+ * Allow other drivers, such as usb controller driver, to check if there are
+ * any sideband activity on the host controller. This information could be used
+ * for power management or other forms of resource management.
+ *
+ * Returns true on any active sideband existence, false otherwise.
+ */
+bool xhci_sideband_check(struct usb_hcd *hcd)
+{
+	struct usb_device *udev = hcd->self.root_hub;
+	bool active;
+
+	device_lock(&udev->dev);
+	active = usb_offload_check(udev);
+	device_unlock(&udev->dev);
+
+	return active;
+}
+EXPORT_SYMBOL_GPL(xhci_sideband_check);
+
 /**
  * xhci_sideband_register - register a sideband for a usb device
  * @intf: usb interface associated with the sideband device
diff --git a/include/linux/usb/xhci-sideband.h b/include/linux/usb/xhci-sideband.h
index 72aa17eb068d..6de1d9c161a1 100644
--- a/include/linux/usb/xhci-sideband.h
+++ b/include/linux/usb/xhci-sideband.h
@@ -11,6 +11,7 @@
 
 #include <linux/scatterlist.h>
 #include <linux/usb.h>
+#include <linux/usb/hcd.h>
 
 #define	EP_CTX_PER_DEV		31	/* FIXME defined twice, from xhci.h */
 
@@ -82,6 +83,11 @@ xhci_sideband_get_endpoint_buffer(struct xhci_sideband *sb,
 				  struct usb_host_endpoint *host_ep);
 struct sg_table *
 xhci_sideband_get_event_buffer(struct xhci_sideband *sb);
+
+int xhci_sideband_get(struct xhci_sideband *sb);
+int xhci_sideband_put(struct xhci_sideband *sb);
+bool xhci_sideband_check(struct usb_hcd *hcd);
+
 int
 xhci_sideband_create_interrupter(struct xhci_sideband *sb, int num_seg,
 				 bool ip_autoclear, u32 imod_interval, int intr_num);
-- 
2.48.0.rc2.279.g1de40edade-goog

[PATCH v9 5/5] usb: host: enable USB offload during system sleep

[Guan-Yu Lin]

Sharing a USB controller with another entity via xhci-sideband driver
creates power management complexities. To prevent the USB controller
from being inadvertently deactivated while in use by the other entity, a
usage-count based mechanism is implemented. This allows the system to
manage power effectively, ensuring the controller remains available
whenever needed.
In order to maintain full functionality of an offloaded USB devices,
several changes are made within the suspend flow of such devices:
- skip usb_suspend_device() so that the port/hub are still active for
  USB transfers via offloaded path.
- not suspending the endpoints which are used by USB interfaces marked
  with needs_remote_wakeup. Namely, skip usb_suspend_interface() and
  usb_hcd_flush_endpoint() on associated USB interfaces. This reserves a
  pending interrupt urb during system suspend for handling the interrupt
  transfer, which is necessary since remote wakeup doesn't apply in the
  offloaded USB devices when controller is still active.
- not flushing the endpoints of actively offloaded USB devices. Given
  that the USB devices is used by another entity, unilaterally flush the
  endpoint might lead to unexpected behavior on another entity.

Signed-off-by: Guan-Yu Lin <guanyulin@google.com>
---
 drivers/usb/core/driver.c         | 23 +++++++++++++++++++----
 drivers/usb/core/endpoint.c       |  8 --------
 drivers/usb/dwc3/core.c           | 28 ++++++++++++++++++++++++++++
 drivers/usb/dwc3/core.h           |  1 +
 drivers/usb/host/xhci-plat.c      | 14 ++++++++++++++
 include/linux/usb.h               |  8 +++++++-
 include/linux/usb/hcd.h           |  7 +++++++
 sound/usb/qcom/qc_audio_offload.c |  3 +++
 8 files changed, 79 insertions(+), 13 deletions(-)

diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c
index 1bbf9592724f..f8d3b50c188a 100644
--- a/drivers/usb/core/driver.c
+++ b/drivers/usb/core/driver.c
@@ -1415,17 +1415,29 @@ static int usb_suspend_both(struct usb_device *udev, pm_message_t msg)
 {
 	int			status = 0;
 	int			i = 0, n = 0;
+	bool			offload = false;
 	struct usb_interface	*intf;
 
 	if (udev->state == USB_STATE_NOTATTACHED ||
 			udev->state == USB_STATE_SUSPENDED)
 		goto done;
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	if (msg.event == PM_EVENT_SUSPEND && usb_offload_check(udev)) {
+		dev_dbg(&udev->dev, "device offload active.\n");
+		offload = true;
+	}
+#endif
+
 	/* Suspend all the interfaces and then udev itself */
 	if (udev->actconfig) {
 		n = udev->actconfig->desc.bNumInterfaces;
 		for (i = n - 1; i >= 0; --i) {
 			intf = udev->actconfig->interface[i];
+			if (offload && intf->needs_remote_wakeup) {
+				dev_dbg(&intf->dev, "interface stays active on an offloaded device\n");
+				continue;
+			}
 			status = usb_suspend_interface(udev, intf, msg);
 
 			/* Ignore errors during system sleep transitions */
@@ -1436,7 +1448,8 @@ static int usb_suspend_both(struct usb_device *udev, pm_message_t msg)
 		}
 	}
 	if (status == 0) {
-		status = usb_suspend_device(udev, msg);
+		if (!offload)
+			status = usb_suspend_device(udev, msg);
 
 		/*
 		 * Ignore errors from non-root-hub devices during
@@ -1481,9 +1494,11 @@ static int usb_suspend_both(struct usb_device *udev, pm_message_t msg)
 	 */
 	} else {
 		udev->can_submit = 0;
-		for (i = 0; i < 16; ++i) {
-			usb_hcd_flush_endpoint(udev, udev->ep_out[i]);
-			usb_hcd_flush_endpoint(udev, udev->ep_in[i]);
+		if (!offload) {
+			for (i = 0; i < 16; ++i) {
+				usb_hcd_flush_endpoint(udev, udev->ep_out[i]);
+				usb_hcd_flush_endpoint(udev, udev->ep_in[i]);
+			}
 		}
 	}
 
diff --git a/drivers/usb/core/endpoint.c b/drivers/usb/core/endpoint.c
index e48399401608..658ef39ebcd1 100644
--- a/drivers/usb/core/endpoint.c
+++ b/drivers/usb/core/endpoint.c
@@ -18,14 +18,6 @@
 #include <linux/usb.h>
 #include "usb.h"
 
-struct ep_device {
-	struct usb_endpoint_descriptor *desc;
-	struct usb_device *udev;
-	struct device dev;
-};
-#define to_ep_device(_dev) \
-	container_of(_dev, struct ep_device, dev)
-
 struct ep_attribute {
 	struct attribute attr;
 	ssize_t (*show)(struct usb_device *,
diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index 0735881d4650..793fbc030fc4 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -2602,8 +2602,22 @@ static int dwc3_runtime_idle(struct device *dev)
 static int dwc3_suspend(struct device *dev)
 {
 	struct dwc3	*dwc = dev_get_drvdata(dev);
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	struct platform_device *xhci = dwc->xhci;
+	struct usb_hcd  *hcd;
+#endif
 	int		ret;
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	if (xhci) {
+		hcd = dev_get_drvdata(&xhci->dev);
+		if (xhci_sideband_check(hcd)) {
+			dev_dbg(dev, "sideband instance active.\n");
+			return 0;
+		}
+	}
+#endif
+
 	ret = dwc3_suspend_common(dwc, PMSG_SUSPEND);
 	if (ret)
 		return ret;
@@ -2616,8 +2630,22 @@ static int dwc3_suspend(struct device *dev)
 static int dwc3_resume(struct device *dev)
 {
 	struct dwc3	*dwc = dev_get_drvdata(dev);
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	struct platform_device *xhci = dwc->xhci;
+	struct usb_hcd  *hcd;
+#endif
 	int		ret = 0;
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	if (xhci) {
+		hcd = dev_get_drvdata(&xhci->dev);
+		if (xhci_sideband_check(hcd)) {
+			dev_dbg(dev, "sideband instance active.\n");
+			return 0;
+		}
+	}
+#endif
+
 	pinctrl_pm_select_default_state(dev);
 
 	pm_runtime_disable(dev);
diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
index 0b6a07202609..57c3e768cdac 100644
--- a/drivers/usb/dwc3/core.h
+++ b/drivers/usb/dwc3/core.h
@@ -26,6 +26,7 @@
 #include <linux/usb/ch9.h>
 #include <linux/usb/gadget.h>
 #include <linux/usb/otg.h>
+#include <linux/usb/hcd.h>
 #include <linux/usb/role.h>
 #include <linux/ulpi/interface.h>
 
diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c
index b676d4dbcec1..9e01450328d7 100644
--- a/drivers/usb/host/xhci-plat.c
+++ b/drivers/usb/host/xhci-plat.c
@@ -456,6 +456,13 @@ static int xhci_plat_suspend_common(struct device *dev, struct pm_message pmsg)
 	struct xhci_hcd	*xhci = hcd_to_xhci(hcd);
 	int ret;
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	if (pmsg.event == PM_EVENT_SUSPEND && xhci_sideband_check(hcd)) {
+		dev_dbg(dev, "sideband instance active.\n");
+		return 0;
+	}
+#endif
+
 	if (pm_runtime_suspended(dev))
 		pm_runtime_resume(dev);
 
@@ -499,6 +506,13 @@ static int xhci_plat_resume_common(struct device *dev, struct pm_message pmsg)
 	struct xhci_hcd	*xhci = hcd_to_xhci(hcd);
 	int ret;
 
+#ifdef CONFIG_USB_XHCI_SIDEBAND
+	if (pmsg.event == PM_EVENT_RESUME && xhci_sideband_check(hcd)) {
+		dev_dbg(dev, "sideband instance active.\n");
+		return 0;
+	}
+#endif
+
 	if (!device_may_wakeup(dev) && (xhci->quirks & XHCI_SUSPEND_RESUME_CLKS)) {
 		ret = clk_prepare_enable(xhci->clk);
 		if (ret)
diff --git a/include/linux/usb.h b/include/linux/usb.h
index 9b3f630e763e..c4ff11ad14d5 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -44,7 +44,13 @@ struct usb_driver;
  * Devices may also have class-specific or vendor-specific descriptors.
  */
 
-struct ep_device;
+struct ep_device {
+	struct usb_endpoint_descriptor *desc;
+	struct usb_device *udev;
+	struct device dev;
+};
+#define to_ep_device(_dev) \
+	container_of(_dev, struct ep_device, dev)
 
 /**
  * struct usb_host_endpoint - host-side endpoint descriptor and queue
diff --git a/include/linux/usb/hcd.h b/include/linux/usb/hcd.h
index ac95e7c89df5..a9577da6ecff 100644
--- a/include/linux/usb/hcd.h
+++ b/include/linux/usb/hcd.h
@@ -766,6 +766,13 @@ extern struct rw_semaphore ehci_cf_port_reset_rwsem;
 #define USB_EHCI_LOADED		2
 extern unsigned long usb_hcds_loaded;
 
+#if IS_ENABLED(CONFIG_USB_XHCI_SIDEBAND)
+extern bool xhci_sideband_check(struct usb_hcd *hcd);
+#else
+static inline bool xhci_sideband_check(struct usb_hcd *hcd)
+{ return false; }
+#endif
+
 #endif /* __KERNEL__ */
 
 #endif /* __USB_CORE_HCD_H */
diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c
index 7dd7e51109df..d201b1ad33b1 100644
--- a/sound/usb/qcom/qc_audio_offload.c
+++ b/sound/usb/qcom/qc_audio_offload.c
@@ -1619,8 +1619,11 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle,
 			mutex_lock(&chip->mutex);
 			subs->opened = 0;
 			mutex_unlock(&chip->mutex);
+		} else {
+			xhci_sideband_get(uadev[pcm_card_num].si);
 		}
 	} else {
+		xhci_sideband_put(uadev[pcm_card_num].si);
 		info = &uadev[pcm_card_num].info[info_idx];
 		if (info->data_ep_pipe) {
 			ep = usb_pipe_endpoint(uadev[pcm_card_num].udev,
-- 
2.48.0.rc2.279.g1de40edade-goog

Re: [PATCH v9 5/5] usb: host: enable USB offload during system sleep

[Michał Pecio]

Hi,

> - not flushing the endpoints of actively offloaded USB devices. Given
>   that the USB devices is used by another entity, unilaterally flush
>   the endpoint might lead to unexpected behavior on another entity.

This doesn't seem right, because flushing applies to URBs managed by
the kernel, so it should have no effect on offloaded endpoints.

As far as I understand from your earlier discussion with Alan Stern,
the real reason is that it disrupted operation of class drivers, in
particular causing kernel-managed interrupt endpoints not to be polled
during suspend and some events were being lost.

Or maybe the real problem was that if the INT IN endpoint isn't being
polled, device events don't trigger xHCI IRQs that wake up the CPU?


And by the way, usb_hcd_flush_endpoint() doc states that no new URBs
may be submitted during this call. I wonder if this can be guaranteed
if the interface has not been suspended first? Perhaps this alone is
good reason not to flush.

Regards,
Michal

Re: [PATCH v9 5/5] usb: host: enable USB offload during system sleep

[Guan-Yu Lin]

On Thu, Feb 6, 2025 at 8:13 AM Michał Pecio <michal.pecio@gmail.com> wrote:
>
> Hi,
>
> > - not flushing the endpoints of actively offloaded USB devices. Given
> >   that the USB devices is used by another entity, unilaterally flush
> >   the endpoint might lead to unexpected behavior on another entity.
>
> This doesn't seem right, because flushing applies to URBs managed by
> the kernel, so it should have no effect on offloaded endpoints.
>
> As far as I understand from your earlier discussion with Alan Stern,
> the real reason is that it disrupted operation of class drivers, in
> particular causing kernel-managed interrupt endpoints not to be polled
> during suspend and some events were being lost.
>
> Or maybe the real problem was that if the INT IN endpoint isn't being
> polled, device events don't trigger xHCI IRQs that wake up the CPU?
>

The main reason is as you described above. Without polling the INT IN
endpoint, some functions (e.g. hid, hub) failed during system suspend.

>
> And by the way, usb_hcd_flush_endpoint() doc states that no new URBs
> may be submitted during this call. I wonder if this can be guaranteed
> if the interface has not been suspended first? Perhaps this alone is
> good reason not to flush.
>
> Regards,
> Michal

To my understanding if the interface creates another URB, then the
interface might be able to submit it. Is there a reason to not
flushing all endpoints on a USB device when we have offloaded transfer
happening during system suspend? If there is, we'll identify what kinds
of endpoints are influenced and not flush them independently.
Otherwise, not flushing all of them might maintain a simpler code
logic.

Regards,
Guan-Yu

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Pierre-Louis Bossart]

On 1/17/25 8:48 AM, Guan-Yu Lin wrote:
> Wesley Cheng and Mathias Nyman's USB offload design enables a co-processor
> to handle some USB transfers, potentially allowing the main system to
> sleep and save power. However, Linux's power management system halts the
> USB host controller when the main system isn't managing any USB transfers.
> To address this, the proposal modifies the system to recognize offloaded
> USB transfers and manage power accordingly.

You probably want to expand on the problem statement and clarify quite a few ambiguous statements.

a) "allowing the main system to sleep and save power". What is the 'main system' and what sort of sleep are you referring to? 
Traditionally when playing audio the audio devices remain in D0. Support for playback during 'S0 idle' is more complicated, I have yet to see a working solution even without USB offload in the picture.

b) when referring to power management, you have to be specific on whether you mean 'runtime_pm' or regular power management. Not the same thing but there are related issues.

c) for audio offload the transactions that go through the DSP or co-processor are only for audio endpoints. Control transactions rely on endpoint0 and are NOT offloaded to the best of my knowledge. Which means that you would need to track control transactions as well, even if there is no audio streaming. Otherwise you would have a risk of the XHCI controller transitioning in and out of sleep states.
 
> This involves two key steps:
> 1. Transfer Status Tracking: Propose xhci_sideband_get and
> xhci_sideband_put to track USB transfers on the co-processor, ensuring the
> system is aware of any ongoing activity.


> 2. Power Management Adjustment:  Modifications to the USB driver stack
> (dwc3 controller driver, xhci host controller driver, and USB device
> drivers) allow the system to sleep without disrupting co-processor managed
> USB transfers. This involves adding conditional checks to bypass some
> power management operations.

This is even more confusing, initially the point was to prevent the controller from sleeping while there are offloaded transactions, but now the goal would be to allow the system to sleep while there are offloaded transactions. This isn't the same problem, is it?
 
> patches depends on series "Introduce QC USB SND audio offloading support" 
> https://lore.kernel.org/lkml/20241213235403.4109199-1-quic_wcheng@quicinc.com/T/
> 
> changelog
> ----------
> Changes in v9:
> - Remove unnecessary white space change.
> 
> Changes in v8:
> - Change the runtime pm api to correct the error handling flow.
> - Not flushing endpoints of actively offloaded USB devices to avoid
>   possible USB transfer conflicts.
> 
> Changes in v7:
> - Remove counting mechanism in struct usb_hcd. The USB device's offload
>   status will be solely recorded in each related struct usb_device.
> - Utilizes `needs_remote_wakeup` attribute in struct usb_interface to
>   control the suspend flow of USB interfaces and associated USB endpoints.
>   This addresses the need to support interrupt transfers generated by
>   offloaded USB devices while the system is suspended.
> - Block any offload_usage change during USB device suspend period.
> 
> Changes in v6:
> - Fix build errors when CONFIG_USB_XHCI_SIDEBAND is disabled.
> - Explicitly specify the data structure of the drvdata refereced in
>   dwc3_suspend(), dwc3_resume().
> - Move the initialization of counters to the patches introducing them.
> 
> Changes in v5:
> - Walk through the USB children in usb_sideband_check() to determine the
>   sideband activity under the specific USB device. 
> - Replace atomic_t by refcount_t.
> - Reduce logs by using dev_dbg & remove __func__.
> 
> Changes in v4:
> - Isolate the feature into USB driver stack.
> - Integrate with series "Introduce QC USB SND audio offloading support"
> 
> Changes in v3:
> - Integrate the feature with the pm core framework.
> 
> Changes in v2:
> - Cosmetics changes on coding style.
> 
> [v3] PM / core: conditionally skip system pm in device/driver model
> [v2] usb: host: enable suspend-to-RAM control in userspace
> [v1] [RFC] usb: host: Allow userspace to control usb suspend flows
> ---
> 
> Guan-Yu Lin (5):
>   usb: dwc3: separate dev_pm_ops for each pm_event
>   usb: xhci-plat: separate dev_pm_ops for each pm_event
>   usb: add apis for offload usage tracking
>   xhci: sideband: add api to trace sideband usage
>   usb: host: enable USB offload during system sleep
> 
>  drivers/usb/core/driver.c         | 131 +++++++++++++++++++++++++++++-
>  drivers/usb/core/endpoint.c       |   8 --
>  drivers/usb/core/usb.c            |   4 +
>  drivers/usb/dwc3/core.c           | 105 +++++++++++++++++++++++-
>  drivers/usb/dwc3/core.h           |   1 +
>  drivers/usb/host/xhci-plat.c      |  42 +++++++++-
>  drivers/usb/host/xhci-sideband.c  |  82 +++++++++++++++++++
>  include/linux/usb.h               |  27 +++++-
>  include/linux/usb/hcd.h           |   7 ++
>  include/linux/usb/xhci-sideband.h |   6 ++
>  sound/usb/qcom/qc_audio_offload.c |   3 +
>  11 files changed, 398 insertions(+), 18 deletions(-)
> 

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Guan-Yu Lin]

On Sat, Jan 18, 2025 at 12:14 AM Pierre-Louis Bossart
<pierre-louis.bossart@linux.dev> wrote:
>
> On 1/17/25 8:48 AM, Guan-Yu Lin wrote:
> > Wesley Cheng and Mathias Nyman's USB offload design enables a co-processor
> > to handle some USB transfers, potentially allowing the main system to
> > sleep and save power. However, Linux's power management system halts the
> > USB host controller when the main system isn't managing any USB transfers.
> > To address this, the proposal modifies the system to recognize offloaded
> > USB transfers and manage power accordingly.
>
> You probably want to expand on the problem statement and clarify quite a few ambiguous statements.
>
> a) "allowing the main system to sleep and save power". What is the 'main system' and what sort of sleep are you referring to?
> Traditionally when playing audio the audio devices remain in D0. Support for playback during 'S0 idle' is more complicated, I have yet to see a working solution even without USB offload in the picture.
>
The main system refers to the device running the linux kernel. The
sleep refers to system suspend in the System Sleep model[1], which
should be S3 state (Suspend to RAM).

> b) when referring to power management, you have to be specific on whether you mean 'runtime_pm' or regular power management. Not the same thing but there are related issues.
>
Thanks for pointing it out. By power management, I mean the System
Sleep model[1].

> c) for audio offload the transactions that go through the DSP or co-processor are only for audio endpoints. Control transactions rely on endpoint0 and are NOT offloaded to the best of my knowledge. Which means that you would need to track control transactions as well, even if there is no audio streaming. Otherwise you would have a risk of the XHCI controller transitioning in and out of sleep states.
>
To my knowledge, the system would not issue control transfer after the
system goes to sleep. If there are any abnormalities in the XHCI
controller that requires the system to handle, the controller would
issue an interrupt to the system. We could allow this interrupt to
wake up the system, and the system could then issue corresponding
control transfers to handle it.

> > This involves two key steps:
> > 1. Transfer Status Tracking: Propose xhci_sideband_get and
> > xhci_sideband_put to track USB transfers on the co-processor, ensuring the
> > system is aware of any ongoing activity.
>
>
> > 2. Power Management Adjustment:  Modifications to the USB driver stack
> > (dwc3 controller driver, xhci host controller driver, and USB device
> > drivers) allow the system to sleep without disrupting co-processor managed
> > USB transfers. This involves adding conditional checks to bypass some
> > power management operations.
>
> This is even more confusing, initially the point was to prevent the controller from sleeping while there are offloaded transactions, but now the goal would be to allow the system to sleep while there are offloaded transactions. This isn't the same problem, is it?
>

The purpose of this series is to allow offloaded usb transfers happen
during system sleep. In order to achieve this, we need to prevent the
controller from sleeping when there's offloaded usb transfer ongoing,
specifically when the system is sleeping.
Without this series, the system could still allow offloaded usb
traffic when the system is active, but the system would put the
controller to sleep when the system is going to sleep, thus we're not
able to suspend the system when we have offloaded usb transfers in the
current system.

[1] https://www.kernel.org/doc/html/v4.12/driver-api/pm/devices.html

Regards,
Guan-Yu

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Pierre-Louis Bossart]

>>> 2. Power Management Adjustment:  Modifications to the USB driver stack
>>> (dwc3 controller driver, xhci host controller driver, and USB device
>>> drivers) allow the system to sleep without disrupting co-processor managed
>>> USB transfers. This involves adding conditional checks to bypass some
>>> power management operations.
>>
>> This is even more confusing, initially the point was to prevent the controller from sleeping while there are offloaded transactions, but now the goal would be to allow the system to sleep while there are offloaded transactions. This isn't the same problem, is it?
>>
> 
> The purpose of this series is to allow offloaded usb transfers happen
> during system sleep. In order to achieve this, we need to prevent the
> controller from sleeping when there's offloaded usb transfer ongoing,
> specifically when the system is sleeping.
> Without this series, the system could still allow offloaded usb
> traffic when the system is active, but the system would put the
> controller to sleep when the system is going to sleep, thus we're not
> able to suspend the system when we have offloaded usb transfers in the
> current system.

I am not following, sorry.

Is the desired outcome to 

a) prevent the system from entering S3 if there is an active USB audio offloaded stream?

or b) allow offloaded transactions even when the system is in S3?


which is it?

a) would be rather interesting, but currently we don't have any such behavior supported. When the system enters S3 all audio stops. The stream will resume when the system goes back to S0. Do we really want the battery to drain in S3?

b) seems rather complicated, once the on-going DMA transfers complete then who's going to refill buffers for the USB offloaded streams? Allowing the lowest level to operate even in S3 is only a small part of the puzzle, someone's got to provide data at some point. Unless the data is generated also by a side DSP having access to mass storage or wireless interfaces?

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Guan-Yu Lin]

On Tue, Jan 28, 2025 at 11:22 PM Pierre-Louis Bossart
<pierre-louis.bossart@linux.dev> wrote:
>
> I am not following, sorry.
>
> Is the desired outcome to
>
> a) prevent the system from entering S3 if there is an active USB audio offloaded stream?
>
> or b) allow offloaded transactions even when the system is in S3?
>
>
> which is it?
>
> a) would be rather interesting, but currently we don't have any such behavior supported. When the system enters S3 all audio stops. The stream will resume when the system goes back to S0. Do we really want the battery to drain in S3?
>
> b) seems rather complicated, once the on-going DMA transfers complete then who's going to refill buffers for the USB offloaded streams? Allowing the lowest level to operate even in S3 is only a small part of the puzzle, someone's got to provide data at some point. Unless the data is generated also by a side DSP having access to mass storage or wireless interfaces?

Thanks for the question, the intent of our proposal should be (b), to
allow offloaded transactions even when the system is in S3.
In our design, the DSP wakes the system before the buffers are fully
drained. This patchset enables the USB controller for offloaded
transfers during system suspend (S3). To be precise, this patchset
focuses solely on enabling the USB controller in S3 and does not
include other necessary components for continuous offloaded USB
transfers. I'll revise the commit message/cover letter to reflect
this.Thanks for highlighting the potential ambiguity.

Regards,
Guan-Yu

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Pierre-Louis Bossart]

On 2/2/25 20:57, Guan-Yu Lin wrote:
> On Tue, Jan 28, 2025 at 11:22 PM Pierre-Louis Bossart
> <pierre-louis.bossart@linux.dev> wrote:
>>
>> I am not following, sorry.
>>
>> Is the desired outcome to
>>
>> a) prevent the system from entering S3 if there is an active USB audio offloaded stream?
>>
>> or b) allow offloaded transactions even when the system is in S3?
>>
>>
>> which is it?
>>
>> a) would be rather interesting, but currently we don't have any such behavior supported. When the system enters S3 all audio stops. The stream will resume when the system goes back to S0. Do we really want the battery to drain in S3?
>>
>> b) seems rather complicated, once the on-going DMA transfers complete then who's going to refill buffers for the USB offloaded streams? Allowing the lowest level to operate even in S3 is only a small part of the puzzle, someone's got to provide data at some point. Unless the data is generated also by a side DSP having access to mass storage or wireless interfaces?
> 
> Thanks for the question, the intent of our proposal should be (b), to
> allow offloaded transactions even when the system is in S3.
> In our design, the DSP wakes the system before the buffers are fully
> drained. This patchset enables the USB controller for offloaded
> transfers during system suspend (S3). To be precise, this patchset
> focuses solely on enabling the USB controller in S3 and does not
> include other necessary components for continuous offloaded USB
> transfers. I'll revise the commit message/cover letter to reflect
> this.Thanks for highlighting the potential ambiguity.

Thanks for the precision.

It was my understanding that anything above S1 could incur a hardware/software latency of two seconds or more to go back to S0. That would imply a buffering scheme that's significantly larger than usual offloaded solutions. In "typical" offload implementations it's rare to go beyond 100s of ms, since at some point you run into user-experience issues when applying volume changes or when changing tracks. It's usually a no-go if the user has to wait for a perceivable amount of time while the buffers drain.

Not to mention that quite a few platforms no longer support S3, since 'Modern Standby' aka "S0 Low Power Idle" or 's2idle' was introduced in the Windows 10 days S3 became largely a legacy feature gradually dropped since no one really uses it.

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Guan-Yu Lin]

On Tue, Feb 4, 2025 at 7:57 AM Pierre-Louis Bossart
<pierre-louis.bossart@linux.dev> wrote:
>
> On 2/2/25 20:57, Guan-Yu Lin wrote:
> > On Tue, Jan 28, 2025 at 11:22 PM Pierre-Louis Bossart
> > <pierre-louis.bossart@linux.dev> wrote:
> >>
> >> I am not following, sorry.
> >>
> >> Is the desired outcome to
> >>
> >> a) prevent the system from entering S3 if there is an active USB audio offloaded stream?
> >>
> >> or b) allow offloaded transactions even when the system is in S3?
> >>
> >>
> >> which is it?
> >>
> >> a) would be rather interesting, but currently we don't have any such behavior supported. When the system enters S3 all audio stops. The stream will resume when the system goes back to S0. Do we really want the battery to drain in S3?
> >>
> >> b) seems rather complicated, once the on-going DMA transfers complete then who's going to refill buffers for the USB offloaded streams? Allowing the lowest level to operate even in S3 is only a small part of the puzzle, someone's got to provide data at some point. Unless the data is generated also by a side DSP having access to mass storage or wireless interfaces?
> >
> > Thanks for the question, the intent of our proposal should be (b), to
> > allow offloaded transactions even when the system is in S3.
> > In our design, the DSP wakes the system before the buffers are fully
> > drained. This patchset enables the USB controller for offloaded
> > transfers during system suspend (S3). To be precise, this patchset
> > focuses solely on enabling the USB controller in S3 and does not
> > include other necessary components for continuous offloaded USB
> > transfers. I'll revise the commit message/cover letter to reflect
> > this.Thanks for highlighting the potential ambiguity.
>
> Thanks for the precision.
>
> It was my understanding that anything above S1 could incur a hardware/software latency of two seconds or more to go back to S0. That would imply a buffering scheme that's significantly larger than usual offloaded solutions. In "typical" offload implementations it's rare to go beyond 100s of ms, since at some point you run into user-experience issues when applying volume changes or when changing tracks. It's usually a no-go if the user has to wait for a perceivable amount of time while the buffers drain.
>
> Not to mention that quite a few platforms no longer support S3, since 'Modern Standby' aka "S0 Low Power Idle" or 's2idle' was introduced in the Windows 10 days S3 became largely a legacy feature gradually dropped since no one really uses it.

I think for mobile devices, the S3 state is still used since the
hardware/software latency wouldn't be as big as you described, so
mobile devices could still use S3 for power saving. How about using a
"knob" to isolate the feature to specific devices? "Knob" could be
dynamically switched by some functions, or we could statically
determine it as a dts attribute or build config. Will a "knob" address
your concern for this feature? Do you have a preference on the "knob"
design?

Regards,
Guan-Yu

Re: [PATCH v9 0/5] Support system sleep with offloaded usb transfers

[Pierre-Louis Bossart]

On 2/7/25 04:54, Guan-Yu Lin wrote:
> On Tue, Feb 4, 2025 at 7:57 AM Pierre-Louis Bossart
> <pierre-louis.bossart@linux.dev> wrote:
>>
>> On 2/2/25 20:57, Guan-Yu Lin wrote:
>>> On Tue, Jan 28, 2025 at 11:22 PM Pierre-Louis Bossart
>>> <pierre-louis.bossart@linux.dev> wrote:
>>>>
>>>> I am not following, sorry.
>>>>
>>>> Is the desired outcome to
>>>>
>>>> a) prevent the system from entering S3 if there is an active USB audio offloaded stream?
>>>>
>>>> or b) allow offloaded transactions even when the system is in S3?
>>>>
>>>>
>>>> which is it?
>>>>
>>>> a) would be rather interesting, but currently we don't have any such behavior supported. When the system enters S3 all audio stops. The stream will resume when the system goes back to S0. Do we really want the battery to drain in S3?
>>>>
>>>> b) seems rather complicated, once the on-going DMA transfers complete then who's going to refill buffers for the USB offloaded streams? Allowing the lowest level to operate even in S3 is only a small part of the puzzle, someone's got to provide data at some point. Unless the data is generated also by a side DSP having access to mass storage or wireless interfaces?
>>>
>>> Thanks for the question, the intent of our proposal should be (b), to
>>> allow offloaded transactions even when the system is in S3.
>>> In our design, the DSP wakes the system before the buffers are fully
>>> drained. This patchset enables the USB controller for offloaded
>>> transfers during system suspend (S3). To be precise, this patchset
>>> focuses solely on enabling the USB controller in S3 and does not
>>> include other necessary components for continuous offloaded USB
>>> transfers. I'll revise the commit message/cover letter to reflect
>>> this.Thanks for highlighting the potential ambiguity.
>>
>> Thanks for the precision.
>>
>> It was my understanding that anything above S1 could incur a hardware/software latency of two seconds or more to go back to S0. That would imply a buffering scheme that's significantly larger than usual offloaded solutions. In "typical" offload implementations it's rare to go beyond 100s of ms, since at some point you run into user-experience issues when applying volume changes or when changing tracks. It's usually a no-go if the user has to wait for a perceivable amount of time while the buffers drain.
>>
>> Not to mention that quite a few platforms no longer support S3, since 'Modern Standby' aka "S0 Low Power Idle" or 's2idle' was introduced in the Windows 10 days S3 became largely a legacy feature gradually dropped since no one really uses it.
> 
> I think for mobile devices, the S3 state is still used since the
> hardware/software latency wouldn't be as big as you described, so
> mobile devices could still use S3 for power saving. How about using a
> "knob" to isolate the feature to specific devices? "Knob" could be
> dynamically switched by some functions, or we could statically
> determine it as a dts attribute or build config. Will a "knob" address
> your concern for this feature? Do you have a preference on the "knob"
> design?

My recollection of mobile devices is that they only relied on S0 platform substates, e.g. S0i1, S0i2, so that the device was always 'active' from a pm_runtime perspective. I have never seen a case where the device was 'active' in S3, I don't think this can work with the current system/pm_runtime power management, can it?
In other words, audio streaming was supported in S0/D0, S0ix/D0ix but S3/D0ix was not a supported configuration.

That said, if you managed to make this work, at the very least this should be a device-specific property, not an unconditional blanket enablement of a capability that raises quite a few questions.