* [PATCH v6 1/6] ath11k: add ath11k_qmi_free_resource() for recovery
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
@ 2022-02-09 6:00 ` Wen Gong
2022-02-21 10:06 ` Kalle Valo
2022-02-09 6:00 ` [PATCH v6 2/6] ath11k: fix invalid m3 buffer address Wen Gong
` (5 subsequent siblings)
6 siblings, 1 reply; 10+ messages in thread
From: Wen Gong @ 2022-02-09 6:00 UTC (permalink / raw)
To: ath11k; +Cc: linux-wireless, quic_wgong
ath11k_qmi_free_target_mem_chunk() and ath11k_qmi_m3_free() is static
in qmi.c, they are needed for recovery, export them in a new function.
Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2
Signed-off-by: Wen Gong <quic_wgong@quicinc.com>
---
drivers/net/wireless/ath/ath11k/qmi.c | 5 +++++
drivers/net/wireless/ath/ath11k/qmi.h | 1 +
2 files changed, 6 insertions(+)
diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c
index d0701e8eca9c..1144e828cf5c 100644
--- a/drivers/net/wireless/ath/ath11k/qmi.c
+++ b/drivers/net/wireless/ath/ath11k/qmi.c
@@ -3025,3 +3025,8 @@ void ath11k_qmi_deinit_service(struct ath11k_base *ab)
}
EXPORT_SYMBOL(ath11k_qmi_deinit_service);
+void ath11k_qmi_free_resource(struct ath11k_base *ab)
+{
+ ath11k_qmi_free_target_mem_chunk(ab);
+ ath11k_qmi_m3_free(ab);
+}
diff --git a/drivers/net/wireless/ath/ath11k/qmi.h b/drivers/net/wireless/ath/ath11k/qmi.h
index ba2eff4d59cb..61678de56ac7 100644
--- a/drivers/net/wireless/ath/ath11k/qmi.h
+++ b/drivers/net/wireless/ath/ath11k/qmi.h
@@ -492,5 +492,6 @@ void ath11k_qmi_event_work(struct work_struct *work);
void ath11k_qmi_msg_recv_work(struct work_struct *work);
void ath11k_qmi_deinit_service(struct ath11k_base *ab);
int ath11k_qmi_init_service(struct ath11k_base *ab);
+void ath11k_qmi_free_resource(struct ath11k_base *ab);
#endif
--
2.31.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v6 2/6] ath11k: fix invalid m3 buffer address
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
2022-02-09 6:00 ` [PATCH v6 1/6] ath11k: add ath11k_qmi_free_resource() for recovery Wen Gong
@ 2022-02-09 6:00 ` Wen Gong
2022-02-09 6:00 ` [PATCH v6 3/6] ath11k: configure RDDM size to mhi for recovery by firmware Wen Gong
` (4 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Wen Gong @ 2022-02-09 6:00 UTC (permalink / raw)
To: ath11k; +Cc: linux-wireless, quic_wgong
From: Carl Huang <quic_cjhuang@quicinc.com>
This is to fix m3 buffer reuse issue as m3_mem->size isn't set to
ZERO in free function, which leads invalid m3 downloading to
firmware and firmware crashed.
Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2
Signed-off-by: Carl Huang <quic_cjhuang@quicinc.com>
Signed-off-by: Wen Gong <quic_wgong@quicinc.com>
---
drivers/net/wireless/ath/ath11k/qmi.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c
index 1144e828cf5c..d4d831566a89 100644
--- a/drivers/net/wireless/ath/ath11k/qmi.c
+++ b/drivers/net/wireless/ath/ath11k/qmi.c
@@ -2342,6 +2342,7 @@ static void ath11k_qmi_m3_free(struct ath11k_base *ab)
dma_free_coherent(ab->dev, m3_mem->size,
m3_mem->vaddr, m3_mem->paddr);
m3_mem->vaddr = NULL;
+ m3_mem->size = 0;
}
static int ath11k_qmi_wlanfw_m3_info_send(struct ath11k_base *ab)
--
2.31.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v6 3/6] ath11k: configure RDDM size to mhi for recovery by firmware
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
2022-02-09 6:00 ` [PATCH v6 1/6] ath11k: add ath11k_qmi_free_resource() for recovery Wen Gong
2022-02-09 6:00 ` [PATCH v6 2/6] ath11k: fix invalid m3 buffer address Wen Gong
@ 2022-02-09 6:00 ` Wen Gong
2022-02-09 6:00 ` [PATCH v6 4/6] ath11k: add support for device recovery for QCA6390/WCN6855 Wen Gong
` (3 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Wen Gong @ 2022-02-09 6:00 UTC (permalink / raw)
To: ath11k; +Cc: linux-wireless, quic_wgong
The rddm_size is needed by firmware while mhi enter RDDM state, add it
to support recovery when ath11k receive MHI_CB_EE_RDDM message.
Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2
Signed-off-by: Wen Gong <quic_wgong@quicinc.com>
---
drivers/net/wireless/ath/ath11k/mhi.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/wireless/ath/ath11k/mhi.c b/drivers/net/wireless/ath/ath11k/mhi.c
index 8b2143802816..fc3524e83e52 100644
--- a/drivers/net/wireless/ath/ath11k/mhi.c
+++ b/drivers/net/wireless/ath/ath11k/mhi.c
@@ -13,6 +13,7 @@
#include "pci.h"
#define MHI_TIMEOUT_DEFAULT_MS 90000
+#define RDDM_DUMP_SIZE 0x420000
static struct mhi_channel_config ath11k_mhi_channels_qca6390[] = {
{
@@ -382,6 +383,7 @@ int ath11k_mhi_register(struct ath11k_pci *ab_pci)
mhi_ctrl->iova_stop = 0xFFFFFFFF;
}
+ mhi_ctrl->rddm_size = RDDM_DUMP_SIZE;
mhi_ctrl->sbl_size = SZ_512K;
mhi_ctrl->seg_len = SZ_512K;
mhi_ctrl->fbc_download = true;
--
2.31.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v6 4/6] ath11k: add support for device recovery for QCA6390/WCN6855
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
` (2 preceding siblings ...)
2022-02-09 6:00 ` [PATCH v6 3/6] ath11k: configure RDDM size to mhi for recovery by firmware Wen Gong
@ 2022-02-09 6:00 ` Wen Gong
2022-02-09 6:00 ` [PATCH v6 5/6] ath11k: add synchronization operation between reconfigure of mac80211 and ath11k_base Wen Gong
` (2 subsequent siblings)
6 siblings, 0 replies; 10+ messages in thread
From: Wen Gong @ 2022-02-09 6:00 UTC (permalink / raw)
To: ath11k; +Cc: linux-wireless, quic_wgong
Currently ath11k has device recovery logic, it is introduced by this
patch "ath11k: Add support for subsystem recovery" which is upstream
by https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=ath11k-bringup&id=3a7b4838b6f6f234239f263ef3dc02e612a083ad.
The patch is for AHB devices such as IPQ8074, it has remote proc module
which is used to download the firmware and boots the processor which
firmware is running on. If firmware crashed, remote proc module will
detect it and download and boot firmware again. Below command will
trigger a firmware crash, and then user can test feature of device
recovery.
Test command:
echo assert > /sys/kernel/debug/ath11k/qca6390\ hw2.0/simulate_fw_crash
echo assert > /sys/kernel/debug/ath11k/wcn6855\ hw2.0/simulate_fw_crash
Unfortunately, QCA6390 is PCIe bus, it does not have the remote proc
module, it use mhi module to communicate between firmware and ath11k.
So ath11k does not support device recovery for QCA6390 currently.
This patch is to add the extra logic which is different for QCA6390.
When firmware crashed, MHI_CB_EE_RDDM event will be indicate by
firmware and then ath11k_mhi_op_status_cb which is the callback of
mhi_controller will receive the MHI_CB_EE_RDDM event, then ath11k
will start to do recovery process, ath11k_core_reset() calls
ath11k_hif_power_down()/ath11k_hif_power_up(), then the mhi/ath11k
will start to download and boot firmware. There are some logic to
avoid deadloop recovery and two simultaneous recovery operations.
And because it has muti-radios for the soc, so it add some logic
in ath11k_mac_op_reconfig_complete() to make sure all radios has
reconfig complete and then complete the device recovery.
Also it add workqueue_aux, because ab->workqueue is used when receive
ATH11K_QMI_EVENT_FW_READY in recovery process(queue_work(ab->workqueue,
&ab->restart_work)), and ath11k_core_reset will wait for max
ATH11K_RESET_TIMEOUT_HZ for the previous restart_work finished, if
ath11k_core_reset also queued in ab->workqueue, then it will delay
restart_work of previous recovery and lead previous recovery fail.
ath11k recovery success for QCA6390/WCN6855 after apply this patch.
Tested-on: QCA6390 hw2.0 PCI WLAN.HST.1.0.1-01740-QCAHSTSWPLZ_V2_TO_X86-1
Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2
Signed-off-by: Wen Gong <quic_wgong@quicinc.com>
---
drivers/net/wireless/ath/ath11k/core.c | 70 ++++++++++++++++++++++++++
drivers/net/wireless/ath/ath11k/core.h | 13 +++++
drivers/net/wireless/ath/ath11k/mac.c | 18 +++++++
drivers/net/wireless/ath/ath11k/mhi.c | 33 ++++++++++++
4 files changed, 134 insertions(+)
diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c
index 7c508e9baa6d..00c83fdb0702 100644
--- a/drivers/net/wireless/ath/ath11k/core.c
+++ b/drivers/net/wireless/ath/ath11k/core.c
@@ -1342,6 +1342,65 @@ static void ath11k_core_restart(struct work_struct *work)
complete(&ab->driver_recovery);
}
+static void ath11k_core_reset(struct work_struct *work)
+{
+ struct ath11k_base *ab = container_of(work, struct ath11k_base, reset_work);
+ int reset_count, fail_cont_count;
+ long time_left;
+
+ if (!(test_bit(ATH11K_FLAG_REGISTERED, &ab->dev_flags))) {
+ ath11k_warn(ab, "ignore reset dev flags 0x%lx\n", ab->dev_flags);
+ return;
+ }
+
+ /* Sometimes the recovery will fail and then the next all recovery fail,
+ * this is to avoid infinite recovery since it can not recovery success.
+ */
+ fail_cont_count = atomic_read(&ab->fail_cont_count);
+
+ if (fail_cont_count >= ATH11K_RESET_MAX_FAIL_COUNT_FINAL)
+ return;
+
+ if (fail_cont_count >= ATH11K_RESET_MAX_FAIL_COUNT_FIRST &&
+ time_before(jiffies, ab->reset_fail_timeout))
+ return;
+
+ reset_count = atomic_inc_return(&ab->reset_count);
+
+ if (reset_count > 1) {
+ /* Sometimes it happened another reset worker before the previous one
+ * completed, then the second reset worker will destroy the previous one,
+ * thus below is to avoid that.
+ */
+ ath11k_warn(ab, "already reseting count %d\n", reset_count);
+
+ reinit_completion(&ab->reset_complete);
+ time_left = wait_for_completion_timeout(&ab->reset_complete,
+ ATH11K_RESET_TIMEOUT_HZ);
+
+ if (time_left) {
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "to skip reset\n");
+ atomic_dec(&ab->reset_count);
+ return;
+ }
+
+ ab->reset_fail_timeout = jiffies + ATH11K_RESET_FAIL_TIMEOUT_HZ;
+ /* Record the continuous recovery fail count when recovery failed*/
+ fail_cont_count = atomic_inc_return(&ab->fail_cont_count);
+ }
+
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "reset starting\n");
+
+ ab->is_reset = true;
+ atomic_set(&ab->recovery_count, 0);
+
+ ath11k_hif_power_down(ab);
+ ath11k_qmi_free_resource(ab);
+ ath11k_hif_power_up(ab);
+
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "reset started\n");
+}
+
static int ath11k_init_hw_params(struct ath11k_base *ab)
{
const struct ath11k_hw_params *hw_params = NULL;
@@ -1411,6 +1470,9 @@ EXPORT_SYMBOL(ath11k_core_deinit);
void ath11k_core_free(struct ath11k_base *ab)
{
+ flush_workqueue(ab->workqueue_aux);
+ destroy_workqueue(ab->workqueue_aux);
+
flush_workqueue(ab->workqueue);
destroy_workqueue(ab->workqueue);
@@ -1434,9 +1496,14 @@ struct ath11k_base *ath11k_core_alloc(struct device *dev, size_t priv_size,
if (!ab->workqueue)
goto err_sc_free;
+ ab->workqueue_aux = create_singlethread_workqueue("ath11k_aux_wq");
+ if (!ab->workqueue_aux)
+ goto err_free_wq;
+
mutex_init(&ab->core_lock);
spin_lock_init(&ab->base_lock);
mutex_init(&ab->vdev_id_11d_lock);
+ init_completion(&ab->reset_complete);
INIT_LIST_HEAD(&ab->peers);
init_waitqueue_head(&ab->peer_mapping_wq);
@@ -1445,6 +1512,7 @@ struct ath11k_base *ath11k_core_alloc(struct device *dev, size_t priv_size,
INIT_WORK(&ab->restart_work, ath11k_core_restart);
INIT_WORK(&ab->update_11d_work, ath11k_update_11d);
INIT_WORK(&ab->rfkill_work, ath11k_rfkill_work);
+ INIT_WORK(&ab->reset_work, ath11k_core_reset);
timer_setup(&ab->rx_replenish_retry, ath11k_ce_rx_replenish_retry, 0);
init_completion(&ab->htc_suspend);
init_completion(&ab->wow.wakeup_completed);
@@ -1455,6 +1523,8 @@ struct ath11k_base *ath11k_core_alloc(struct device *dev, size_t priv_size,
return ab;
+err_free_wq:
+ destroy_workqueue(ab->workqueue);
err_sc_free:
kfree(ab);
return NULL;
diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h
index 10846e9e871a..3fc49e633c29 100644
--- a/drivers/net/wireless/ath/ath11k/core.h
+++ b/drivers/net/wireless/ath/ath11k/core.h
@@ -39,6 +39,10 @@
extern unsigned int ath11k_frame_mode;
#define ATH11K_MON_TIMER_INTERVAL 10
+#define ATH11K_RESET_TIMEOUT_HZ (20 * HZ)
+#define ATH11K_RESET_MAX_FAIL_COUNT_FIRST 3
+#define ATH11K_RESET_MAX_FAIL_COUNT_FINAL 5
+#define ATH11K_RESET_FAIL_TIMEOUT_HZ (20 * HZ)
enum ath11k_supported_bw {
ATH11K_BW_20 = 0,
@@ -787,6 +791,15 @@ struct ath11k_base {
struct work_struct restart_work;
struct work_struct update_11d_work;
u8 new_alpha2[3];
+ struct workqueue_struct *workqueue_aux;
+ struct work_struct reset_work;
+ atomic_t reset_count;
+ atomic_t recovery_count;
+ bool is_reset;
+ struct completion reset_complete;
+ /* continuous recovery fail count */
+ atomic_t fail_cont_count;
+ unsigned long reset_fail_timeout;
struct {
/* protected by data_lock */
u32 fw_crash_counter;
diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c
index ed899055944e..c57a8f2c7820 100644
--- a/drivers/net/wireless/ath/ath11k/mac.c
+++ b/drivers/net/wireless/ath/ath11k/mac.c
@@ -7860,6 +7860,8 @@ ath11k_mac_op_reconfig_complete(struct ieee80211_hw *hw,
enum ieee80211_reconfig_type reconfig_type)
{
struct ath11k *ar = hw->priv;
+ struct ath11k_base *ab = ar->ab;
+ int recovery_count;
if (reconfig_type != IEEE80211_RECONFIG_TYPE_RESTART)
return;
@@ -7871,6 +7873,22 @@ ath11k_mac_op_reconfig_complete(struct ieee80211_hw *hw,
ar->pdev->pdev_id);
ar->state = ATH11K_STATE_ON;
ieee80211_wake_queues(ar->hw);
+
+ if (ab->is_reset) {
+ recovery_count = atomic_inc_return(&ab->recovery_count);
+ ath11k_dbg(ab, ATH11K_DBG_BOOT,
+ "recovery count %d\n", recovery_count);
+ /* When there are multiple radios in an SOC,
+ * the recovery has to be done for each radio
+ */
+ if (recovery_count == ab->num_radios) {
+ atomic_dec(&ab->reset_count);
+ complete(&ab->reset_complete);
+ ab->is_reset = false;
+ atomic_set(&ab->fail_cont_count, 0);
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "reset success\n");
+ }
+ }
}
mutex_unlock(&ar->conf_mutex);
diff --git a/drivers/net/wireless/ath/ath11k/mhi.c b/drivers/net/wireless/ath/ath11k/mhi.c
index fc3524e83e52..61d83be4841f 100644
--- a/drivers/net/wireless/ath/ath11k/mhi.c
+++ b/drivers/net/wireless/ath/ath11k/mhi.c
@@ -292,15 +292,48 @@ static void ath11k_mhi_op_runtime_put(struct mhi_controller *mhi_cntrl)
{
}
+static char *ath11k_mhi_op_callback_to_str(enum mhi_callback reason)
+{
+ switch (reason) {
+ case MHI_CB_IDLE:
+ return "MHI_CB_IDLE";
+ case MHI_CB_PENDING_DATA:
+ return "MHI_CB_PENDING_DATA";
+ case MHI_CB_LPM_ENTER:
+ return "MHI_CB_LPM_ENTER";
+ case MHI_CB_LPM_EXIT:
+ return "MHI_CB_LPM_EXIT";
+ case MHI_CB_EE_RDDM:
+ return "MHI_CB_EE_RDDM";
+ case MHI_CB_EE_MISSION_MODE:
+ return "MHI_CB_EE_MISSION_MODE";
+ case MHI_CB_SYS_ERROR:
+ return "MHI_CB_SYS_ERROR";
+ case MHI_CB_FATAL_ERROR:
+ return "MHI_CB_FATAL_ERROR";
+ case MHI_CB_BW_REQ:
+ return "MHI_CB_BW_REQ";
+ default:
+ return "UNKNOWN";
+ }
+};
+
static void ath11k_mhi_op_status_cb(struct mhi_controller *mhi_cntrl,
enum mhi_callback cb)
{
struct ath11k_base *ab = dev_get_drvdata(mhi_cntrl->cntrl_dev);
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "mhi notify status reason %s\n",
+ ath11k_mhi_op_callback_to_str(cb));
+
switch (cb) {
case MHI_CB_SYS_ERROR:
ath11k_warn(ab, "firmware crashed: MHI_CB_SYS_ERROR\n");
break;
+ case MHI_CB_EE_RDDM:
+ if (!(test_bit(ATH11K_FLAG_UNREGISTERING, &ab->dev_flags)))
+ queue_work(ab->workqueue_aux, &ab->reset_work);
+ break;
default:
break;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v6 5/6] ath11k: add synchronization operation between reconfigure of mac80211 and ath11k_base
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
` (3 preceding siblings ...)
2022-02-09 6:00 ` [PATCH v6 4/6] ath11k: add support for device recovery for QCA6390/WCN6855 Wen Gong
@ 2022-02-09 6:00 ` Wen Gong
2022-02-09 6:00 ` [PATCH v6 6/6] ath11k: Add hw-restart option to simulate_fw_crash Wen Gong
2022-02-21 10:20 ` [PATCH v6 0/6] ath11k: add feature for device recovery Kalle Valo
6 siblings, 0 replies; 10+ messages in thread
From: Wen Gong @ 2022-02-09 6:00 UTC (permalink / raw)
To: ath11k; +Cc: linux-wireless, quic_wgong
ieee80211_reconfig() of mac80211 is the main function for recovery of
each ieee80211_hw and ath11k, and ath11k_core_reconfigure_on_crash()
is the main function for recovery of ath11k_base, it has more than
one ieee80211_hw and ath11k for each ath11k_base, so it need to add
synchronization between them, otherwise it has many issue.
For example, when ath11k_core_reconfigure_on_crash() is not complete,
mac80211 send a hw scan request to ath11k, it leads firmware crash,
because firmware has not been initialized at that moment, firmware
is only finished downloaded and loaded, it can not receive scan
command.
Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2
Signed-off-by: Wen Gong <quic_wgong@quicinc.com>
---
drivers/net/wireless/ath/ath11k/core.c | 51 ++++++++++++++++++++++----
drivers/net/wireless/ath/ath11k/core.h | 5 +++
drivers/net/wireless/ath/ath11k/mac.c | 22 +++++++++++
3 files changed, 70 insertions(+), 8 deletions(-)
diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c
index 00c83fdb0702..8ce93fd9e1e6 100644
--- a/drivers/net/wireless/ath/ath11k/core.c
+++ b/drivers/net/wireless/ath/ath11k/core.c
@@ -1263,12 +1263,11 @@ static void ath11k_update_11d(struct work_struct *work)
}
}
-static void ath11k_core_restart(struct work_struct *work)
+static void ath11k_core_pre_reconfigure_recovery(struct ath11k_base *ab)
{
- struct ath11k_base *ab = container_of(work, struct ath11k_base, restart_work);
struct ath11k *ar;
struct ath11k_pdev *pdev;
- int i, ret = 0;
+ int i;
spin_lock_bh(&ab->base_lock);
ab->stats.fw_crash_counter++;
@@ -1301,12 +1300,13 @@ static void ath11k_core_restart(struct work_struct *work)
wake_up(&ab->wmi_ab.tx_credits_wq);
wake_up(&ab->peer_mapping_wq);
+}
- ret = ath11k_core_reconfigure_on_crash(ab);
- if (ret) {
- ath11k_err(ab, "failed to reconfigure driver on crash recovery\n");
- return;
- }
+static void ath11k_core_post_reconfigure_recovery(struct ath11k_base *ab)
+{
+ struct ath11k *ar;
+ struct ath11k_pdev *pdev;
+ int i;
for (i = 0; i < ab->num_radios; i++) {
pdev = &ab->pdevs[i];
@@ -1342,6 +1342,27 @@ static void ath11k_core_restart(struct work_struct *work)
complete(&ab->driver_recovery);
}
+static void ath11k_core_restart(struct work_struct *work)
+{
+ struct ath11k_base *ab = container_of(work, struct ath11k_base, restart_work);
+ int ret;
+
+ if (!ab->is_reset)
+ ath11k_core_pre_reconfigure_recovery(ab);
+
+ ret = ath11k_core_reconfigure_on_crash(ab);
+ if (ret) {
+ ath11k_err(ab, "failed to reconfigure driver on crash recovery\n");
+ return;
+ }
+
+ if (ab->is_reset)
+ complete_all(&ab->reconfigure_complete);
+
+ if (!ab->is_reset)
+ ath11k_core_post_reconfigure_recovery(ab);
+}
+
static void ath11k_core_reset(struct work_struct *work)
{
struct ath11k_base *ab = container_of(work, struct ath11k_base, reset_work);
@@ -1393,6 +1414,18 @@ static void ath11k_core_reset(struct work_struct *work)
ab->is_reset = true;
atomic_set(&ab->recovery_count, 0);
+ reinit_completion(&ab->recovery_start);
+ atomic_set(&ab->recovery_start_count, 0);
+
+ ath11k_core_pre_reconfigure_recovery(ab);
+
+ reinit_completion(&ab->reconfigure_complete);
+ ath11k_core_post_reconfigure_recovery(ab);
+
+ ath11k_dbg(ab, ATH11K_DBG_BOOT, "waiting recovery start...\n");
+
+ time_left = wait_for_completion_timeout(&ab->recovery_start,
+ ATH11K_RECOVER_START_TIMEOUT_HZ);
ath11k_hif_power_down(ab);
ath11k_qmi_free_resource(ab);
@@ -1504,6 +1537,8 @@ struct ath11k_base *ath11k_core_alloc(struct device *dev, size_t priv_size,
spin_lock_init(&ab->base_lock);
mutex_init(&ab->vdev_id_11d_lock);
init_completion(&ab->reset_complete);
+ init_completion(&ab->reconfigure_complete);
+ init_completion(&ab->recovery_start);
INIT_LIST_HEAD(&ab->peers);
init_waitqueue_head(&ab->peer_mapping_wq);
diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h
index 3fc49e633c29..0fd5a7ee6304 100644
--- a/drivers/net/wireless/ath/ath11k/core.h
+++ b/drivers/net/wireless/ath/ath11k/core.h
@@ -43,6 +43,8 @@ extern unsigned int ath11k_frame_mode;
#define ATH11K_RESET_MAX_FAIL_COUNT_FIRST 3
#define ATH11K_RESET_MAX_FAIL_COUNT_FINAL 5
#define ATH11K_RESET_FAIL_TIMEOUT_HZ (20 * HZ)
+#define ATH11K_RECONFIGURE_TIMEOUT_HZ (10 * HZ)
+#define ATH11K_RECOVER_START_TIMEOUT_HZ (20 * HZ)
enum ath11k_supported_bw {
ATH11K_BW_20 = 0,
@@ -795,8 +797,11 @@ struct ath11k_base {
struct work_struct reset_work;
atomic_t reset_count;
atomic_t recovery_count;
+ atomic_t recovery_start_count;
bool is_reset;
struct completion reset_complete;
+ struct completion reconfigure_complete;
+ struct completion recovery_start;
/* continuous recovery fail count */
atomic_t fail_cont_count;
unsigned long reset_fail_timeout;
diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c
index c57a8f2c7820..1a735bb804e1 100644
--- a/drivers/net/wireless/ath/ath11k/mac.c
+++ b/drivers/net/wireless/ath/ath11k/mac.c
@@ -5726,6 +5726,27 @@ static int ath11k_mac_config_mon_status_default(struct ath11k *ar, bool enable)
return ret;
}
+static void ath11k_mac_wait_reconfigure(struct ath11k_base *ab)
+{
+ int recovery_start_count;
+
+ if (!ab->is_reset)
+ return;
+
+ recovery_start_count = atomic_inc_return(&ab->recovery_start_count);
+ ath11k_dbg(ab, ATH11K_DBG_MAC, "recovery start count %d\n", recovery_start_count);
+
+ if (recovery_start_count == ab->num_radios) {
+ complete(&ab->recovery_start);
+ ath11k_dbg(ab, ATH11K_DBG_MAC, "recovery started success\n");
+ }
+
+ ath11k_dbg(ab, ATH11K_DBG_MAC, "waiting reconfigure...\n");
+
+ wait_for_completion_timeout(&ab->reconfigure_complete,
+ ATH11K_RECONFIGURE_TIMEOUT_HZ);
+}
+
static int ath11k_mac_op_start(struct ieee80211_hw *hw)
{
struct ath11k *ar = hw->priv;
@@ -5742,6 +5763,7 @@ static int ath11k_mac_op_start(struct ieee80211_hw *hw)
break;
case ATH11K_STATE_RESTARTING:
ar->state = ATH11K_STATE_RESTARTED;
+ ath11k_mac_wait_reconfigure(ab);
break;
case ATH11K_STATE_RESTARTED:
case ATH11K_STATE_WEDGED:
--
2.31.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v6 6/6] ath11k: Add hw-restart option to simulate_fw_crash
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
` (4 preceding siblings ...)
2022-02-09 6:00 ` [PATCH v6 5/6] ath11k: add synchronization operation between reconfigure of mac80211 and ath11k_base Wen Gong
@ 2022-02-09 6:00 ` Wen Gong
2022-02-21 10:20 ` [PATCH v6 0/6] ath11k: add feature for device recovery Kalle Valo
6 siblings, 0 replies; 10+ messages in thread
From: Wen Gong @ 2022-02-09 6:00 UTC (permalink / raw)
To: ath11k; +Cc: linux-wireless, quic_wgong
Add hw-restart to directly restart wlan. Like UTF mode start it will
restart hardware and download firmware again.
Usage:
1. Run command:
echo hw-restart > /sys/kernel/debug/ath11k/qca6390\ hw2.0/simulate_fw_crash
echo hw-restart > /sys/kernel/debug/ath11k/wcn6855\ hw2.0/simulate_fw_crash
2. wlan will be restart and do recovery process and success.
Tested-on: WCN6855 hw2.0 PCI WLAN.HSP.1.1-03003-QCAHSPSWPL_V1_V2_SILICONZ_LITE-2
Signed-off-by: Wen Gong <quic_wgong@quicinc.com>
---
drivers/net/wireless/ath/ath11k/debugfs.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/net/wireless/ath/ath11k/debugfs.c b/drivers/net/wireless/ath/ath11k/debugfs.c
index c0ebdb1262f0..7375defb0f23 100644
--- a/drivers/net/wireless/ath/ath11k/debugfs.c
+++ b/drivers/net/wireless/ath/ath11k/debugfs.c
@@ -557,6 +557,10 @@ static ssize_t ath11k_write_simulate_fw_crash(struct file *file,
ret = ath11k_wmi_force_fw_hang_cmd(ar,
ATH11K_WMI_FW_HANG_ASSERT_TYPE,
ATH11K_WMI_FW_HANG_DELAY);
+ } else if (!strcmp(buf, "hw-restart")) {
+ ath11k_info(ab, "user requested hw restart\n");
+ queue_work(ab->workqueue_aux, &ab->reset_work);
+ ret = 0;
} else {
ret = -EINVAL;
goto exit;
--
2.31.1
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v6 0/6] ath11k: add feature for device recovery
2022-02-09 6:00 [PATCH v6 0/6] ath11k: add feature for device recovery Wen Gong
` (5 preceding siblings ...)
2022-02-09 6:00 ` [PATCH v6 6/6] ath11k: Add hw-restart option to simulate_fw_crash Wen Gong
@ 2022-02-21 10:20 ` Kalle Valo
2022-02-25 5:33 ` Wen Gong
6 siblings, 1 reply; 10+ messages in thread
From: Kalle Valo @ 2022-02-21 10:20 UTC (permalink / raw)
To: Wen Gong; +Cc: ath11k, linux-wireless
Wen Gong <quic_wgong@quicinc.com> writes:
> Currently recovery is work success for QCA6390/WCN6855 without the RDDM dump feature,
> because patch "ath11k: configure RDDM size to mhi for recovery by firmware"
> added in v5.
The firmware restart works now but I get warnings, see log below. I
applied patches 1-3, but patches 4-6 need more work. I'm using:
ath11k_pci 0000:06:00.0: MSI vectors: 32
ath11k_pci 0000:06:00.0: qca6390 hw2.0
ath11k_pci 0000:06:00.0: chip_id 0x0 chip_family 0xb board_id 0xff soc_id 0xffffffff
ath11k_pci 0000:06:00.0: fw_version 0x10121492 fw_build_timestamp 2021-11-04 11:23 fw_build_id
Here's the log:
[ 1965.642121] ath11k_pci 0000:06:00.0: simulating firmware assert crash
[ 1968.471364] ieee80211 phy0: Hardware restart was requested
[ 1968.511305] ------------[ cut here ]------------
[ 1968.511368] WARNING: CPU: 3 PID: 1546 at drivers/bus/mhi/core/pm.c:505 mhi_pm_disable_transition+0xb37/0xda0 [mhi]
[ 1968.511443] Modules linked in: ath11k_pci ath11k mac80211 libarc4 cfg80211 qmi_helpers qrtr_mhi mhi qrtr nvme nvme_core
[ 1968.511563] CPU: 3 PID: 1546 Comm: kworker/u17:0 Kdump: loaded Tainted: G W 5.17.0-rc3-wt-ath+ #579
[ 1968.511629] Hardware name: Intel(R) Client Systems NUC8i7HVK/NUC8i7HVB, BIOS HNKBLi70.86A.0067.2021.0528.1339 05/28/2021
[ 1968.511704] Workqueue: mhi_hiprio_wq mhi_pm_st_worker [mhi]
[ 1968.511787] RIP: 0010:mhi_pm_disable_transition+0xb37/0xda0 [mhi]
[ 1968.511870] Code: a9 fe ff ff 4c 89 ff 44 89 04 24 e8 03 46 f6 e5 44 8b 04 24 41 83 f8 01 0f 84 21 fe ff ff e9 4c fd ff ff 0f 0b e9 af f8 ff ff <0f> 0b e9 5c f8 ff ff 48 89 df e8 da 9e ee e3 e9 12 fd ff ff 4c 89
[ 1968.511923] RSP: 0018:ffffc900024efbf0 EFLAGS: 00010286
[ 1968.511969] RAX: 00000000ffffffff RBX: ffff88811d241250 RCX: ffffffffc0176922
[ 1968.512014] RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff888118a90a24
[ 1968.512059] RBP: ffff888118a90800 R08: 0000000000000000 R09: ffff888118a90a27
[ 1968.512102] R10: ffffed1023152144 R11: 0000000000000001 R12: ffff888118a908ac
[ 1968.512229] R13: ffff888118a90928 R14: dffffc0000000000 R15: ffff888118a90a24
[ 1968.512310] FS: 0000000000000000(0000) GS:ffff888234200000(0000) knlGS:0000000000000000
[ 1968.512405] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1968.512493] CR2: 00007f5538f443a8 CR3: 000000016dc28001 CR4: 00000000003706e0
[ 1968.512587] Call Trace:
[ 1968.512672] <TASK>
[ 1968.512751] ? _raw_spin_unlock_irq+0x1f/0x40
[ 1968.512859] mhi_pm_st_worker+0x3ac/0x790 [mhi]
[ 1968.512959] ? mhi_pm_mission_mode_transition.isra.0+0x7d0/0x7d0 [mhi]
[ 1968.513063] process_one_work+0x86a/0x1400
[ 1968.513184] ? pwq_dec_nr_in_flight+0x230/0x230
[ 1968.513312] ? move_linked_works+0x125/0x290
[ 1968.513416] worker_thread+0x6db/0xf60
[ 1968.513536] ? process_one_work+0x1400/0x1400
[ 1968.513627] kthread+0x241/0x2d0
[ 1968.513733] ? kthread_complete_and_exit+0x20/0x20
[ 1968.513821] ret_from_fork+0x22/0x30
[ 1968.513924] </TASK>
[ 1968.514000] irq event stamp: 38683
[ 1968.514079] hardirqs last enabled at (38691): [<ffffffffa3c6eb52>] __up_console_sem+0x62/0x70
[ 1968.514185] hardirqs last disabled at (38700): [<ffffffffa3c6eb37>] __up_console_sem+0x47/0x70
[ 1968.514266] softirqs last enabled at (37928): [<ffffffffc01740ac>] mhi_reset_chan+0x76c/0xaa0 [mhi]
[ 1968.514356] softirqs last disabled at (37924): [<ffffffffc0173a71>] mhi_reset_chan+0x131/0xaa0 [mhi]
[ 1968.514448] ---[ end trace 0000000000000000 ]---
[ 1968.844604] mhi mhi0: Requested to power ON
[ 1968.845578] mhi mhi0: Power on setup success
[ 1969.152617] mhi mhi0: Wait for device to enter SBL or Mission mode
[ 1969.268367] ath11k_pci 0000:06:00.0: chip_id 0x0 chip_family 0xb board_id 0xff soc_id 0xffffffff
[ 1969.268441] ath11k_pci 0000:06:00.0: fw_version 0x10121492 fw_build_timestamp 2021-11-04 11:23 fw_build_id
[ 1969.293648] ath11k_pci 0000:06:00.0: Last interrupt received for each CE:
[ 1969.293712] ath11k_pci 0000:06:00.0: CE_id 0 pipe_num 0 126895ms before
[ 1969.293803] ath11k_pci 0000:06:00.0: CE_id 1 pipe_num 1 19337ms before
[ 1969.293851] ath11k_pci 0000:06:00.0: CE_id 2 pipe_num 2 3648ms before
[ 1969.293913] ath11k_pci 0000:06:00.0: CE_id 3 pipe_num 3 3649ms before
[ 1969.293974] ath11k_pci 0000:06:00.0: CE_id 5 pipe_num 5 126895ms before
[ 1969.294034] ath11k_pci 0000:06:00.0: CE_id 7 pipe_num 7 1668238ms before
[ 1969.294095] ath11k_pci 0000:06:00.0: CE_id 8 pipe_num 8 1668238ms before
[ 1969.294279] ath11k_pci 0000:06:00.0: #012[ 1969.294279] Last interrupt received for each group:
[ 1969.294343] ath11k_pci 0000:06:00.0: group_id 0 6782ms before
[ 1969.294408] ath11k_pci 0000:06:00.0: group_id 1 1668239ms before
[ 1969.294473] ath11k_pci 0000:06:00.0: group_id 2 1668239ms before
[ 1969.294535] ath11k_pci 0000:06:00.0: group_id 3 1668239ms before
[ 1969.295035] ath11k_pci 0000:06:00.0: group_id 4 1668239ms before
[ 1969.295103] ath11k_pci 0000:06:00.0: group_id 5 1668239ms before
[ 1969.295390] ath11k_pci 0000:06:00.0: group_id 6 1668240ms before
[ 1969.295442] ath11k_pci 0000:06:00.0: group_id 7 1668240ms before
[ 1969.295518] ath11k_pci 0000:06:00.0: group_id 8 6783ms before
[ 1969.295579] ath11k_pci 0000:06:00.0: group_id 9 1668240ms before
[ 1969.295640] ath11k_pci 0000:06:00.0: group_id 10 1668240ms before
[ 1969.295715] ath11k_pci 0000:06:00.0: dst srng id 0 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668240ms
[ 1969.295783] ath11k_pci 0000:06:00.0: dst srng id 1 tp 208, cur hp 208, cached hp 208 last hp 208 napi processed before 6783ms
[ 1969.295845] ath11k_pci 0000:06:00.0: dst srng id 2 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668240ms
[ 1969.295908] ath11k_pci 0000:06:00.0: dst srng id 3 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668240ms
[ 1969.295971] ath11k_pci 0000:06:00.0: dst srng id 4 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 6782ms
[ 1969.296038] ath11k_pci 0000:06:00.0: src srng id 5 hp 0, reap_hp 248, cur tp 0, cached tp 0 last tp 0 napi processed before 1668240ms
[ 1969.296103] ath11k_pci 0000:06:00.0: src srng id 8 hp 350, reap_hp 350, cur tp 180, cached tp 180 last tp 180 napi processed before 814ms
[ 1969.296239] ath11k_pci 0000:06:00.0: dst srng id 9 tp 468, cur hp 468, cached hp 468 last hp 468 napi processed before 6783ms
[ 1969.296290] ath11k_pci 0000:06:00.0: src srng id 16 hp 120, reap_hp 120, cur tp 120, cached tp 112 last tp 112 napi processed before 6801ms
[ 1969.296356] ath11k_pci 0000:06:00.0: src srng id 24 hp 0, reap_hp 248, cur tp 0, cached tp 0 last tp 0 napi processed before 1668241ms
[ 1969.296419] ath11k_pci 0000:06:00.0: dst srng id 25 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668241ms
[ 1969.296483] ath11k_pci 0000:06:00.0: src srng id 32 hp 12, reap_hp 8, cur tp 12, cached tp 12 last tp 8 napi processed before 126898ms
[ 1969.296544] ath11k_pci 0000:06:00.0: src srng id 35 hp 28, reap_hp 24, cur tp 28, cached tp 28 last tp 24 napi processed before 3654ms
[ 1969.296605] ath11k_pci 0000:06:00.0: src srng id 36 hp 48, reap_hp 44, cur tp 48, cached tp 48 last tp 44 napi processed before 126484ms
[ 1969.296667] ath11k_pci 0000:06:00.0: src srng id 39 hp 0, reap_hp 124, cur tp 0, cached tp 0 last tp 0 napi processed before 1668241ms
[ 1969.296728] ath11k_pci 0000:06:00.0: src srng id 57 hp 0, reap_hp 0, cur tp 4, cached tp 4 last tp 4 napi processed before 19339ms
[ 1969.296791] ath11k_pci 0000:06:00.0: src srng id 58 hp 792, reap_hp 792, cur tp 796, cached tp 796 last tp 796 napi processed before 3650ms
[ 1969.296853] ath11k_pci 0000:06:00.0: src srng id 61 hp 1020, reap_hp 1020, cur tp 0, cached tp 0 last tp 0 napi processed before 126898ms
[ 1969.296916] ath11k_pci 0000:06:00.0: dst srng id 81 tp 8, cur hp 8, cached hp 8 last hp 8 napi processed before 19340ms
[ 1969.296977] ath11k_pci 0000:06:00.0: dst srng id 82 tp 1592, cur hp 1592, cached hp 1592 last hp 1592 napi processed before 3650ms
[ 1969.297058] ath11k_pci 0000:06:00.0: dst srng id 85 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 126898ms
[ 1969.297125] ath11k_pci 0000:06:00.0: src srng id 104 hp 65532, reap_hp 65532, cur tp 0, cached tp 0 last tp 0 napi processed before 126999ms
[ 1969.297256] ath11k_pci 0000:06:00.0: src srng id 105 hp 0, reap_hp 504, cur tp 0, cached tp 0 last tp 0 napi processed before 1668242ms
[ 1969.297321] ath11k_pci 0000:06:00.0: dst srng id 106 tp 176, cur hp 176, cached hp 176 last hp 176 napi processed before 6784ms
[ 1969.297388] ath11k_pci 0000:06:00.0: dst srng id 109 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 6784ms
[ 1969.297824] ath11k_pci 0000:06:00.0: src srng id 128 hp 24, reap_hp 24, cur tp 4182, cached tp 4182 last tp 4182 napi processed before 6784ms
[ 1969.297888] ath11k_pci 0000:06:00.0: src srng id 129 hp 0, reap_hp 2046, cur tp 0, cached tp 0 last tp 0 napi processed before 1668242ms
[ 1969.297951] ath11k_pci 0000:06:00.0: src srng id 132 hp 20, reap_hp 20, cur tp 22, cached tp 22 last tp 22 napi processed before 4ms
[ 1969.298014] ath11k_pci 0000:06:00.0: dst srng id 133 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 6784ms
[ 1969.298079] ath11k_pci 0000:06:00.0: src srng id 144 hp 0, reap_hp 2046, cur tp 0, cached tp 0 last tp 0 napi processed before 1668242ms
[ 1969.298191] ath11k_pci 0000:06:00.0: src srng id 147 hp 110, reap_hp 110, cur tp 112, cached tp 112 last tp 112 napi processed before 5ms
[ 1969.298255] ath11k_pci 0000:06:00.0: dst srng id 148 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668243ms
[ 1970.945274] ath11k_pci 0000:06:00.0: Already processed, so ignoring dma ring caps
[ 1973.850557] ath11k_pci 0000:06:00.0: pdev 0 successfully recovered
--
https://patchwork.kernel.org/project/linux-wireless/list/
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v6 0/6] ath11k: add feature for device recovery
2022-02-21 10:20 ` [PATCH v6 0/6] ath11k: add feature for device recovery Kalle Valo
@ 2022-02-25 5:33 ` Wen Gong
0 siblings, 0 replies; 10+ messages in thread
From: Wen Gong @ 2022-02-25 5:33 UTC (permalink / raw)
To: Kalle Valo; +Cc: ath11k, linux-wireless
Thanks Kalle.
I have found the reason of the warning.
I will send a new path together with the patches 4-6.
On 2/21/2022 6:20 PM, Kalle Valo wrote:
> Wen Gong <quic_wgong@quicinc.com> writes:
>
>> Currently recovery is work success for QCA6390/WCN6855 without the RDDM dump feature,
>> because patch "ath11k: configure RDDM size to mhi for recovery by firmware"
>> added in v5.
> The firmware restart works now but I get warnings, see log below. I
> applied patches 1-3, but patches 4-6 need more work. I'm using:
>
> ath11k_pci 0000:06:00.0: MSI vectors: 32
> ath11k_pci 0000:06:00.0: qca6390 hw2.0
> ath11k_pci 0000:06:00.0: chip_id 0x0 chip_family 0xb board_id 0xff soc_id 0xffffffff
> ath11k_pci 0000:06:00.0: fw_version 0x10121492 fw_build_timestamp 2021-11-04 11:23 fw_build_id
>
> Here's the log:
>
> [ 1965.642121] ath11k_pci 0000:06:00.0: simulating firmware assert crash
> [ 1968.471364] ieee80211 phy0: Hardware restart was requested
> [ 1968.511305] ------------[ cut here ]------------
> [ 1968.511368] WARNING: CPU: 3 PID: 1546 at drivers/bus/mhi/core/pm.c:505 mhi_pm_disable_transition+0xb37/0xda0 [mhi]
> [ 1968.511443] Modules linked in: ath11k_pci ath11k mac80211 libarc4 cfg80211 qmi_helpers qrtr_mhi mhi qrtr nvme nvme_core
> [ 1968.511563] CPU: 3 PID: 1546 Comm: kworker/u17:0 Kdump: loaded Tainted: G W 5.17.0-rc3-wt-ath+ #579
> [ 1968.511629] Hardware name: Intel(R) Client Systems NUC8i7HVK/NUC8i7HVB, BIOS HNKBLi70.86A.0067.2021.0528.1339 05/28/2021
> [ 1968.511704] Workqueue: mhi_hiprio_wq mhi_pm_st_worker [mhi]
> [ 1968.511787] RIP: 0010:mhi_pm_disable_transition+0xb37/0xda0 [mhi]
> [ 1968.511870] Code: a9 fe ff ff 4c 89 ff 44 89 04 24 e8 03 46 f6 e5 44 8b 04 24 41 83 f8 01 0f 84 21 fe ff ff e9 4c fd ff ff 0f 0b e9 af f8 ff ff <0f> 0b e9 5c f8 ff ff 48 89 df e8 da 9e ee e3 e9 12 fd ff ff 4c 89
> [ 1968.511923] RSP: 0018:ffffc900024efbf0 EFLAGS: 00010286
> [ 1968.511969] RAX: 00000000ffffffff RBX: ffff88811d241250 RCX: ffffffffc0176922
> [ 1968.512014] RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff888118a90a24
> [ 1968.512059] RBP: ffff888118a90800 R08: 0000000000000000 R09: ffff888118a90a27
> [ 1968.512102] R10: ffffed1023152144 R11: 0000000000000001 R12: ffff888118a908ac
> [ 1968.512229] R13: ffff888118a90928 R14: dffffc0000000000 R15: ffff888118a90a24
> [ 1968.512310] FS: 0000000000000000(0000) GS:ffff888234200000(0000) knlGS:0000000000000000
> [ 1968.512405] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1968.512493] CR2: 00007f5538f443a8 CR3: 000000016dc28001 CR4: 00000000003706e0
> [ 1968.512587] Call Trace:
> [ 1968.512672] <TASK>
> [ 1968.512751] ? _raw_spin_unlock_irq+0x1f/0x40
> [ 1968.512859] mhi_pm_st_worker+0x3ac/0x790 [mhi]
> [ 1968.512959] ? mhi_pm_mission_mode_transition.isra.0+0x7d0/0x7d0 [mhi]
> [ 1968.513063] process_one_work+0x86a/0x1400
> [ 1968.513184] ? pwq_dec_nr_in_flight+0x230/0x230
> [ 1968.513312] ? move_linked_works+0x125/0x290
> [ 1968.513416] worker_thread+0x6db/0xf60
> [ 1968.513536] ? process_one_work+0x1400/0x1400
> [ 1968.513627] kthread+0x241/0x2d0
> [ 1968.513733] ? kthread_complete_and_exit+0x20/0x20
> [ 1968.513821] ret_from_fork+0x22/0x30
> [ 1968.513924] </TASK>
> [ 1968.514000] irq event stamp: 38683
> [ 1968.514079] hardirqs last enabled at (38691): [<ffffffffa3c6eb52>] __up_console_sem+0x62/0x70
> [ 1968.514185] hardirqs last disabled at (38700): [<ffffffffa3c6eb37>] __up_console_sem+0x47/0x70
> [ 1968.514266] softirqs last enabled at (37928): [<ffffffffc01740ac>] mhi_reset_chan+0x76c/0xaa0 [mhi]
> [ 1968.514356] softirqs last disabled at (37924): [<ffffffffc0173a71>] mhi_reset_chan+0x131/0xaa0 [mhi]
> [ 1968.514448] ---[ end trace 0000000000000000 ]---
> [ 1968.844604] mhi mhi0: Requested to power ON
> [ 1968.845578] mhi mhi0: Power on setup success
> [ 1969.152617] mhi mhi0: Wait for device to enter SBL or Mission mode
> [ 1969.268367] ath11k_pci 0000:06:00.0: chip_id 0x0 chip_family 0xb board_id 0xff soc_id 0xffffffff
> [ 1969.268441] ath11k_pci 0000:06:00.0: fw_version 0x10121492 fw_build_timestamp 2021-11-04 11:23 fw_build_id
> [ 1969.293648] ath11k_pci 0000:06:00.0: Last interrupt received for each CE:
> [ 1969.293712] ath11k_pci 0000:06:00.0: CE_id 0 pipe_num 0 126895ms before
> [ 1969.293803] ath11k_pci 0000:06:00.0: CE_id 1 pipe_num 1 19337ms before
> [ 1969.293851] ath11k_pci 0000:06:00.0: CE_id 2 pipe_num 2 3648ms before
> [ 1969.293913] ath11k_pci 0000:06:00.0: CE_id 3 pipe_num 3 3649ms before
> [ 1969.293974] ath11k_pci 0000:06:00.0: CE_id 5 pipe_num 5 126895ms before
> [ 1969.294034] ath11k_pci 0000:06:00.0: CE_id 7 pipe_num 7 1668238ms before
> [ 1969.294095] ath11k_pci 0000:06:00.0: CE_id 8 pipe_num 8 1668238ms before
> [ 1969.294279] ath11k_pci 0000:06:00.0: #012[ 1969.294279] Last interrupt received for each group:
> [ 1969.294343] ath11k_pci 0000:06:00.0: group_id 0 6782ms before
> [ 1969.294408] ath11k_pci 0000:06:00.0: group_id 1 1668239ms before
> [ 1969.294473] ath11k_pci 0000:06:00.0: group_id 2 1668239ms before
> [ 1969.294535] ath11k_pci 0000:06:00.0: group_id 3 1668239ms before
> [ 1969.295035] ath11k_pci 0000:06:00.0: group_id 4 1668239ms before
> [ 1969.295103] ath11k_pci 0000:06:00.0: group_id 5 1668239ms before
> [ 1969.295390] ath11k_pci 0000:06:00.0: group_id 6 1668240ms before
> [ 1969.295442] ath11k_pci 0000:06:00.0: group_id 7 1668240ms before
> [ 1969.295518] ath11k_pci 0000:06:00.0: group_id 8 6783ms before
> [ 1969.295579] ath11k_pci 0000:06:00.0: group_id 9 1668240ms before
> [ 1969.295640] ath11k_pci 0000:06:00.0: group_id 10 1668240ms before
> [ 1969.295715] ath11k_pci 0000:06:00.0: dst srng id 0 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668240ms
> [ 1969.295783] ath11k_pci 0000:06:00.0: dst srng id 1 tp 208, cur hp 208, cached hp 208 last hp 208 napi processed before 6783ms
> [ 1969.295845] ath11k_pci 0000:06:00.0: dst srng id 2 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668240ms
> [ 1969.295908] ath11k_pci 0000:06:00.0: dst srng id 3 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668240ms
> [ 1969.295971] ath11k_pci 0000:06:00.0: dst srng id 4 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 6782ms
> [ 1969.296038] ath11k_pci 0000:06:00.0: src srng id 5 hp 0, reap_hp 248, cur tp 0, cached tp 0 last tp 0 napi processed before 1668240ms
> [ 1969.296103] ath11k_pci 0000:06:00.0: src srng id 8 hp 350, reap_hp 350, cur tp 180, cached tp 180 last tp 180 napi processed before 814ms
> [ 1969.296239] ath11k_pci 0000:06:00.0: dst srng id 9 tp 468, cur hp 468, cached hp 468 last hp 468 napi processed before 6783ms
> [ 1969.296290] ath11k_pci 0000:06:00.0: src srng id 16 hp 120, reap_hp 120, cur tp 120, cached tp 112 last tp 112 napi processed before 6801ms
> [ 1969.296356] ath11k_pci 0000:06:00.0: src srng id 24 hp 0, reap_hp 248, cur tp 0, cached tp 0 last tp 0 napi processed before 1668241ms
> [ 1969.296419] ath11k_pci 0000:06:00.0: dst srng id 25 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668241ms
> [ 1969.296483] ath11k_pci 0000:06:00.0: src srng id 32 hp 12, reap_hp 8, cur tp 12, cached tp 12 last tp 8 napi processed before 126898ms
> [ 1969.296544] ath11k_pci 0000:06:00.0: src srng id 35 hp 28, reap_hp 24, cur tp 28, cached tp 28 last tp 24 napi processed before 3654ms
> [ 1969.296605] ath11k_pci 0000:06:00.0: src srng id 36 hp 48, reap_hp 44, cur tp 48, cached tp 48 last tp 44 napi processed before 126484ms
> [ 1969.296667] ath11k_pci 0000:06:00.0: src srng id 39 hp 0, reap_hp 124, cur tp 0, cached tp 0 last tp 0 napi processed before 1668241ms
> [ 1969.296728] ath11k_pci 0000:06:00.0: src srng id 57 hp 0, reap_hp 0, cur tp 4, cached tp 4 last tp 4 napi processed before 19339ms
> [ 1969.296791] ath11k_pci 0000:06:00.0: src srng id 58 hp 792, reap_hp 792, cur tp 796, cached tp 796 last tp 796 napi processed before 3650ms
> [ 1969.296853] ath11k_pci 0000:06:00.0: src srng id 61 hp 1020, reap_hp 1020, cur tp 0, cached tp 0 last tp 0 napi processed before 126898ms
> [ 1969.296916] ath11k_pci 0000:06:00.0: dst srng id 81 tp 8, cur hp 8, cached hp 8 last hp 8 napi processed before 19340ms
> [ 1969.296977] ath11k_pci 0000:06:00.0: dst srng id 82 tp 1592, cur hp 1592, cached hp 1592 last hp 1592 napi processed before 3650ms
> [ 1969.297058] ath11k_pci 0000:06:00.0: dst srng id 85 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 126898ms
> [ 1969.297125] ath11k_pci 0000:06:00.0: src srng id 104 hp 65532, reap_hp 65532, cur tp 0, cached tp 0 last tp 0 napi processed before 126999ms
> [ 1969.297256] ath11k_pci 0000:06:00.0: src srng id 105 hp 0, reap_hp 504, cur tp 0, cached tp 0 last tp 0 napi processed before 1668242ms
> [ 1969.297321] ath11k_pci 0000:06:00.0: dst srng id 106 tp 176, cur hp 176, cached hp 176 last hp 176 napi processed before 6784ms
> [ 1969.297388] ath11k_pci 0000:06:00.0: dst srng id 109 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 6784ms
> [ 1969.297824] ath11k_pci 0000:06:00.0: src srng id 128 hp 24, reap_hp 24, cur tp 4182, cached tp 4182 last tp 4182 napi processed before 6784ms
> [ 1969.297888] ath11k_pci 0000:06:00.0: src srng id 129 hp 0, reap_hp 2046, cur tp 0, cached tp 0 last tp 0 napi processed before 1668242ms
> [ 1969.297951] ath11k_pci 0000:06:00.0: src srng id 132 hp 20, reap_hp 20, cur tp 22, cached tp 22 last tp 22 napi processed before 4ms
> [ 1969.298014] ath11k_pci 0000:06:00.0: dst srng id 133 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 6784ms
> [ 1969.298079] ath11k_pci 0000:06:00.0: src srng id 144 hp 0, reap_hp 2046, cur tp 0, cached tp 0 last tp 0 napi processed before 1668242ms
> [ 1969.298191] ath11k_pci 0000:06:00.0: src srng id 147 hp 110, reap_hp 110, cur tp 112, cached tp 112 last tp 112 napi processed before 5ms
> [ 1969.298255] ath11k_pci 0000:06:00.0: dst srng id 148 tp 0, cur hp 0, cached hp 0 last hp 0 napi processed before 1668243ms
> [ 1970.945274] ath11k_pci 0000:06:00.0: Already processed, so ignoring dma ring caps
> [ 1973.850557] ath11k_pci 0000:06:00.0: pdev 0 successfully recovered
>
^ permalink raw reply [flat|nested] 10+ messages in thread