[RESEND PATCH v3 0/2] change UIC command handling

[RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

v2 -> v3: rule out the change of polling w/ pmc from this thread.
(I'll post the change later)
v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd

Kiwoong Kim (2):
  ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
  ufs: poll HCS.UCRDY before issuing a UIC command

 drivers/ufs/core/ufshcd.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

-- 
2.7.4

[RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

v2 -> v3: rule out the change of polling w/ pmc from this thread.
(I'll post the change later)
v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd

Kiwoong Kim (2):
  ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
  ufs: poll HCS.UCRDY before issuing a UIC command

 drivers/ufs/core/ufshcd.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

-- 
2.7.4

[RESEND PATCH v3 1/2] ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock

[Kiwoong Kim]

__ufshcd_send_uic_cmd is wrapped uic_cmd_mutex and
its related contexts are accessed within the period wrappted
by uic_cmd_mutex. Thus, wrapping with host_lock is
redundant.

Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
---
 drivers/ufs/core/ufshcd.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 27e1a49..6300ed6 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -2414,7 +2414,6 @@ __ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd,
 		      bool completion)
 {
 	lockdep_assert_held(&hba->uic_cmd_mutex);
-	lockdep_assert_held(hba->host->host_lock);
 
 	if (!ufshcd_ready_for_uic_cmd(hba)) {
 		dev_err(hba->dev,
@@ -2441,7 +2440,6 @@ __ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd,
 int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd)
 {
 	int ret;
-	unsigned long flags;
 
 	if (hba->quirks & UFSHCD_QUIRK_BROKEN_UIC_CMD)
 		return 0;
@@ -2450,9 +2448,7 @@ int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd)
 	mutex_lock(&hba->uic_cmd_mutex);
 	ufshcd_add_delay_before_dme_cmd(hba);
 
-	spin_lock_irqsave(hba->host->host_lock, flags);
 	ret = __ufshcd_send_uic_cmd(hba, uic_cmd, true);
-	spin_unlock_irqrestore(hba->host->host_lock, flags);
 	if (!ret)
 		ret = ufshcd_wait_for_uic_cmd(hba, uic_cmd);
 
@@ -4161,8 +4157,8 @@ static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd)
 		wmb();
 		reenable_intr = true;
 	}
-	ret = __ufshcd_send_uic_cmd(hba, cmd, false);
 	spin_unlock_irqrestore(hba->host->host_lock, flags);
+	ret = __ufshcd_send_uic_cmd(hba, cmd, false);
 	if (ret) {
 		dev_err(hba->dev,
 			"pwr ctrl cmd 0x%x with mode 0x%x uic error %d\n",
-- 
2.7.4

[RESEND PATCH v3 2/2] ufs: poll HCS.UCRDY before issuing a UIC command

[Kiwoong Kim]

With auto hibern8 enabled, UIC could be working
for a while to process a hibern8 operation and HCI
reports UIC not ready for a short term through HCS.UCRDY.
And UFS driver can't recognize the operation.
UFSHCI spec specifies UCRDY like this:
whether the host controller is ready to process UIC COMMAND

The 'ready' could be seen as many different meanings. If the meaning
includes not processing any request from HCI, processing a hibern8
operation can be 'not ready'. In this situation, the driver needs to
wait until the operations is completed.

Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Reviewed-by: Adrian Hunter <adrian.hunter@intel.com>
---
 drivers/ufs/core/ufshcd.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 6300ed6..7bc3fc4 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/regulator/consumer.h>
 #include <linux/sched/clock.h>
+#include <linux/iopoll.h>
 #include <scsi/scsi_cmnd.h>
 #include <scsi/scsi_dbg.h>
 #include <scsi/scsi_driver.h>
@@ -2322,7 +2323,11 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba)
  */
 static inline bool ufshcd_ready_for_uic_cmd(struct ufs_hba *hba)
 {
-	return ufshcd_readl(hba, REG_CONTROLLER_STATUS) & UIC_COMMAND_READY;
+	u32 val;
+	int ret = read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
+				    500, UIC_CMD_TIMEOUT * 1000, false, hba,
+				    REG_CONTROLLER_STATUS);
+	return ret == 0 ? true : false;
 }
 
 /**
-- 
2.7.4

Re: [RESEND PATCH v3 1/2] ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock

[Chanwoo Lee]

> __ufshcd_send_uic_cmd is wrapped uic_cmd_mutex and
> its related contexts are accessed within the period wrappted
> by uic_cmd_mutex. Thus, wrapping with host_lock is
> redundant.
> 
> Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
> Reviewed-by: Bart Van Assche <bvanassche@acm.org>
> ---

Reviewed-by: Junwoo Lee <junwoo80.lee@samsung.com>
Reviewed-by: Chanwoo Lee <cw9316.lee@samsung.com>

Re: [RESEND,v3,2/2] ufs: poll HCS.UCRDY before issuing a UIC command

[Chanwoo Lee]

On 9/04/23 5:31 PM, cw9316.lee@samsung.com wrote:
> With auto hibern8 enabled, UIC could be working
> for a while to process a hibern8 operation and HCI
> reports UIC not ready for a short term through HCS.UCRDY.
> And UFS driver can't recognize the operation.
> UFSHCI spec specifies UCRDY like this:
> whether the host controller is ready to process UIC COMMAND
> 
> The 'ready' could be seen as many different meanings. If the meaning
> includes not processing any request from HCI, processing a hibern8
> operation can be 'not ready'. In this situation, the driver needs to
> wait until the operations is completed.
> 
> Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
> Reviewed-by: Adrian Hunter <adrian.hunter@intel.com>
> ---

Reviewed-by: Junwoo Lee <junwoo80.lee@samsung.com>
Reviewed-by: Chanwoo Lee <cw9316.lee@samsung.com>

Re: [RESEND PATCH v3 0/2] change UIC command handling

[Martin K. Petersen]

Kiwoong,

> v2 -> v3: rule out the change of polling w/ pmc from this thread.
> (I'll post the change later)
> v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd
>
> Kiwoong Kim (2):
>   ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
>   ufs: poll HCS.UCRDY before issuing a UIC command

Applied to 6.6/scsi-staging, thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

RE: [RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

> > v2 -> v3: rule out the change of polling w/ pmc from this thread.
> > (I'll post the change later)
> > v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd
> >
> > Kiwoong Kim (2):
> >   ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
> >   ufs: poll HCS.UCRDY before issuing a UIC command
> 
> Applied to 6.6/scsi-staging, thanks!
> 
> --
> Martin K. Petersen	Oracle Linux Engineering

Hi, Martin

The following patch seems to make trouble because of using
read_poll_timeout.
Its initial version used udelay and after discussion it's been changed.
Could you revert this patch set?

> ufs: poll HCS.UCRDY before issuing a UIC command

[ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
kworker/u20:29/17140/0x00000002
..
[ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388
[ 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
[ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94
[ 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64
[ 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8
[ 4671.228824] [3: kworker/u20:29:17140]
schedule_hrtimeout_range_clock+0x98/0x114
[ 4671.228841] [3: kworker/u20:29:17140]  schedule_hrtimeout_range+0x14/0x24
[ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
[ 4671.228871] [3: kworker/u20:29:17140]  __ufshcd_send_uic_cmd+0xa0/0x1c4
[ 4671.228893] [3: kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390
[ 4671.228908] [3: kworker/u20:29:17140]
ufshcd_uic_hibern8_enter+0x9c/0x25c
[ 4671.228922] [3: kworker/u20:29:17140]
ufshcd_link_state_transition+0x34/0xb0
[ 4671.228939] [3: kworker/u20:29:17140]  __ufshcd_wl_suspend+0x3f0/0x4b4

Thanks you.

Re: [RESEND PATCH v3 0/2] change UIC command handling

[Adrian Hunter]

On 11/09/23 04:35, Kiwoong Kim wrote:
>>> v2 -> v3: rule out the change of polling w/ pmc from this thread.
>>> (I'll post the change later)
>>> v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd
>>>
>>> Kiwoong Kim (2):
>>>   ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
>>>   ufs: poll HCS.UCRDY before issuing a UIC command
>>
>> Applied to 6.6/scsi-staging, thanks!
>>
>> --
>> Martin K. Petersen	Oracle Linux Engineering
> 
> Hi, Martin
> 
> The following patch seems to make trouble because of using
> read_poll_timeout.
> Its initial version used udelay and after discussion it's been changed.
> Could you revert this patch set?
> 
>> ufs: poll HCS.UCRDY before issuing a UIC command
> 
> [ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
> kworker/u20:29/17140/0x00000002
> ..
> [ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388
> [ 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
> [ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94
> [ 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64
> [ 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8
> [ 4671.228824] [3: kworker/u20:29:17140]
> schedule_hrtimeout_range_clock+0x98/0x114
> [ 4671.228841] [3: kworker/u20:29:17140]  schedule_hrtimeout_range+0x14/0x24
> [ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
> [ 4671.228871] [3: kworker/u20:29:17140]  __ufshcd_send_uic_cmd+0xa0/0x1c4
> [ 4671.228893] [3: kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390
> [ 4671.228908] [3: kworker/u20:29:17140]
> ufshcd_uic_hibern8_enter+0x9c/0x25c
> [ 4671.228922] [3: kworker/u20:29:17140]
> ufshcd_link_state_transition+0x34/0xb0
> [ 4671.228939] [3: kworker/u20:29:17140]  __ufshcd_wl_suspend+0x3f0/0x4b4

Do you know what is in that path that makes it an atomic context?

RE: [RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

> >> ufs: poll HCS.UCRDY before issuing a UIC command
> >
> > [ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
> > kworker/u20:29/17140/0x00000002
> > ..
> > [ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388 [
> > 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
> > [ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94 [
> > 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64 [
> > 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8 [
> > 4671.228824] [3: kworker/u20:29:17140]
> > schedule_hrtimeout_range_clock+0x98/0x114
> > [ 4671.228841] [3: kworker/u20:29:17140]
> > schedule_hrtimeout_range+0x14/0x24
> > [ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
> > [ 4671.228871] [3: kworker/u20:29:17140]
> > __ufshcd_send_uic_cmd+0xa0/0x1c4 [ 4671.228893] [3:
> > kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390 [ 4671.228908]
> > [3: kworker/u20:29:17140] ufshcd_uic_hibern8_enter+0x9c/0x25c
> > [ 4671.228922] [3: kworker/u20:29:17140]
> > ufshcd_link_state_transition+0x34/0xb0
> > [ 4671.228939] [3: kworker/u20:29:17140]
> > __ufshcd_wl_suspend+0x3f0/0x4b4
> 
> Do you know what is in that path that makes it an atomic context?

Hi,
This made that.

static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd)
..
        bool reenable_intr = false;

        mutex_lock(&hba->uic_cmd_mutex); <<<<


At first, I was willing to post together w/ the following patch but I've got a suggestion to split the patch set because of different topic and I split the patch set.
- This patch removes the mutex, so it can fix the issue.
https://lore.kernel.org/linux-scsi/1694051306-172962-1-git-send-email-kwmad.kim@samsung.com/


But now I'm thinking again that simply removing the mutex could hurt atomicity of UIC command process
that the original code intended for the first time.
So I think this polling UCRDY should be modified rather than applying removal of the mutex.

Re: [RESEND PATCH v3 0/2] change UIC command handling

[Adrian Hunter]

On 11/09/23 09:32, Kiwoong Kim wrote:
>>>> ufs: poll HCS.UCRDY before issuing a UIC command
>>>
>>> [ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
>>> kworker/u20:29/17140/0x00000002
>>> ..
>>> [ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388 [
>>> 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
>>> [ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94 [
>>> 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64 [
>>> 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8 [
>>> 4671.228824] [3: kworker/u20:29:17140]
>>> schedule_hrtimeout_range_clock+0x98/0x114
>>> [ 4671.228841] [3: kworker/u20:29:17140]
>>> schedule_hrtimeout_range+0x14/0x24
>>> [ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
>>> [ 4671.228871] [3: kworker/u20:29:17140]
>>> __ufshcd_send_uic_cmd+0xa0/0x1c4 [ 4671.228893] [3:
>>> kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390 [ 4671.228908]
>>> [3: kworker/u20:29:17140] ufshcd_uic_hibern8_enter+0x9c/0x25c
>>> [ 4671.228922] [3: kworker/u20:29:17140]
>>> ufshcd_link_state_transition+0x34/0xb0
>>> [ 4671.228939] [3: kworker/u20:29:17140]
>>> __ufshcd_wl_suspend+0x3f0/0x4b4
>>
>> Do you know what is in that path that makes it an atomic context?
> 
> Hi,
> This made that.
> 
> static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd)
> ..
>         bool reenable_intr = false;
> 
>         mutex_lock(&hba->uic_cmd_mutex); <<<<

It is OK to schedule while holding a mutex.  Are you sure
this is the problem?

> 
> 
> At first, I was willing to post together w/ the following patch but I've got a suggestion to split the patch set because of different topic and I split the patch set.
> - This patch removes the mutex, so it can fix the issue.
> https://lore.kernel.org/linux-scsi/1694051306-172962-1-git-send-email-kwmad.kim@samsung.com/
> 
> 
> But now I'm thinking again that simply removing the mutex could hurt atomicity of UIC command process
> that the original code intended for the first time.
> So I think this polling UCRDY should be modified rather than applying removal of the mutex.
> 
> 
> 

RE: [RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

> > static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command
> > *cmd) ..
> >         bool reenable_intr = false;
> >
> >         mutex_lock(&hba->uic_cmd_mutex); <<<<
> 
> It is OK to schedule while holding a mutex.  Are you sure this is the
> problem?

Ah, I mis-understood it. It was for not applying this.
https://lore.kernel.org/linux-scsi/782ba5f26f0a96e58d85dff50751787d2d2a6b2b.1693790060.git.kwmad.kim@samsung.com/

So this patch set has no problem. Sorry for bothering all of you.

Thanks.