[RESEND PATCH v3 0/2] change UIC command handling

[RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

v2 -> v3: rule out the change of polling w/ pmc from this thread.
(I'll post the change later)
v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd

Kiwoong Kim (2):
  ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
  ufs: poll HCS.UCRDY before issuing a UIC command

 drivers/ufs/core/ufshcd.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

-- 
2.7.4

[RESEND PATCH v3 1/2] ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock

[Kiwoong Kim]

__ufshcd_send_uic_cmd is wrapped uic_cmd_mutex and
its related contexts are accessed within the period wrappted
by uic_cmd_mutex. Thus, wrapping with host_lock is
redundant.

Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
---
 drivers/ufs/core/ufshcd.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 9434328..a89d39a 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -2457,7 +2457,6 @@ __ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd,
 		      bool completion)
 {
 	lockdep_assert_held(&hba->uic_cmd_mutex);
-	lockdep_assert_held(hba->host->host_lock);
 
 	if (!ufshcd_ready_for_uic_cmd(hba)) {
 		dev_err(hba->dev,
@@ -2484,7 +2483,6 @@ __ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd,
 int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd)
 {
 	int ret;
-	unsigned long flags;
 
 	if (hba->quirks & UFSHCD_QUIRK_BROKEN_UIC_CMD)
 		return 0;
@@ -2493,9 +2491,7 @@ int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd)
 	mutex_lock(&hba->uic_cmd_mutex);
 	ufshcd_add_delay_before_dme_cmd(hba);
 
-	spin_lock_irqsave(hba->host->host_lock, flags);
 	ret = __ufshcd_send_uic_cmd(hba, uic_cmd, true);
-	spin_unlock_irqrestore(hba->host->host_lock, flags);
 	if (!ret)
 		ret = ufshcd_wait_for_uic_cmd(hba, uic_cmd);
 
@@ -4180,8 +4176,8 @@ static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd)
 		wmb();
 		reenable_intr = true;
 	}
-	ret = __ufshcd_send_uic_cmd(hba, cmd, false);
 	spin_unlock_irqrestore(hba->host->host_lock, flags);
+	ret = __ufshcd_send_uic_cmd(hba, cmd, false);
 	if (ret) {
 		dev_err(hba->dev,
 			"pwr ctrl cmd 0x%x with mode 0x%x uic error %d\n",
-- 
2.7.4

[RESEND PATCH v3 2/2] ufs: poll HCS.UCRDY before issuing a UIC command

[Kiwoong Kim]

With auto hibern8 enabled, UIC could be working
for a while to process a hibern8 operation and HCI
reports UIC not ready for a short term through HCS.UCRDY.
And UFS driver can't recognize the operation.
UFSHCI spec specifies UCRDY like this:
whether the host controller is ready to process UIC COMMAND

The 'ready' could be seen as many different meanings. If the meaning
includes not processing any request from HCI, processing a hibern8
operation can be 'not ready'. In this situation, the driver needs to
wait until the operations is completed.

Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
---
 drivers/ufs/core/ufshcd.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index a89d39a..10ccc85 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/regulator/consumer.h>
 #include <linux/sched/clock.h>
+#include <linux/iopoll.h>
 #include <scsi/scsi_cmnd.h>
 #include <scsi/scsi_dbg.h>
 #include <scsi/scsi_driver.h>
@@ -2365,7 +2366,11 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba)
  */
 static inline bool ufshcd_ready_for_uic_cmd(struct ufs_hba *hba)
 {
-	return ufshcd_readl(hba, REG_CONTROLLER_STATUS) & UIC_COMMAND_READY;
+	u32 val;
+	int ret = read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
+				    500, UIC_CMD_TIMEOUT * 1000, false, hba,
+				    REG_CONTROLLER_STATUS);
+	return ret == 0 ? true : false;
 }
 
 /**
-- 
2.7.4

Re: [RESEND PATCH v3 2/2] ufs: poll HCS.UCRDY before issuing a UIC command

[Adrian Hunter]

On 2/08/23 04:28, Kiwoong Kim wrote:
> With auto hibern8 enabled, UIC could be working
> for a while to process a hibern8 operation and HCI
> reports UIC not ready for a short term through HCS.UCRDY.
> And UFS driver can't recognize the operation.
> UFSHCI spec specifies UCRDY like this:
> whether the host controller is ready to process UIC COMMAND
> 
> The 'ready' could be seen as many different meanings. If the meaning
> includes not processing any request from HCI, processing a hibern8
> operation can be 'not ready'. In this situation, the driver needs to
> wait until the operations is completed.
> 
> Signed-off-by: Kiwoong Kim <kwmad.kim@samsung.com>
> ---
>  drivers/ufs/core/ufshcd.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
> index a89d39a..10ccc85 100644
> --- a/drivers/ufs/core/ufshcd.c
> +++ b/drivers/ufs/core/ufshcd.c
> @@ -22,6 +22,7 @@
>  #include <linux/module.h>
>  #include <linux/regulator/consumer.h>
>  #include <linux/sched/clock.h>
> +#include <linux/iopoll.h>
>  #include <scsi/scsi_cmnd.h>
>  #include <scsi/scsi_dbg.h>
>  #include <scsi/scsi_driver.h>
> @@ -2365,7 +2366,11 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba)
>   */
>  static inline bool ufshcd_ready_for_uic_cmd(struct ufs_hba *hba)
>  {
> -	return ufshcd_readl(hba, REG_CONTROLLER_STATUS) & UIC_COMMAND_READY;
> +	u32 val;
> +	int ret = read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
> +				    500, UIC_CMD_TIMEOUT * 1000, false, hba,
> +				    REG_CONTROLLER_STATUS);
> +	return ret == 0 ? true : false;

Could use a comment in the code.

And perhaps the following is neater:

	u32 val;

	return !read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
				  500, UIC_CMD_TIMEOUT * 1000, false, hba,
				  REG_CONTROLLER_STATUS);

>  }
>  
>  /**

Re: [RESEND PATCH v3 2/2] ufs: poll HCS.UCRDY before issuing a UIC command

[Bart Van Assche]

On 8/14/23 04:26, Adrian Hunter wrote:
> And perhaps the following is neater:
> 
> 	u32 val;
> 
> 	return !read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
> 				  500, UIC_CMD_TIMEOUT * 1000, false, hba,
> 				  REG_CONTROLLER_STATUS);

Would the above make readers of that code wonder whether read_poll_timeout()
perhaps returns a boolean? Wouldn't it be better to test the
read_poll_timeout() return value as follows?

  	return read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
  				  500, UIC_CMD_TIMEOUT * 1000, false, hba,
  				  REG_CONTROLLER_STATUS) == 0;

Thanks,

Bart.

Re: [RESEND PATCH v3 2/2] ufs: poll HCS.UCRDY before issuing a UIC command

[Adrian Hunter]

On 17/08/23 18:02, Bart Van Assche wrote:
> On 8/14/23 04:26, Adrian Hunter wrote:
>> And perhaps the following is neater:
>>
>>     u32 val;
>>
>>     return !read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
>>                   500, UIC_CMD_TIMEOUT * 1000, false, hba,
>>                   REG_CONTROLLER_STATUS);
> 
> Would the above make readers of that code wonder whether read_poll_timeout()
> perhaps returns a boolean? Wouldn't it be better to test the
> read_poll_timeout() return value as follows?
> 
>      return read_poll_timeout(ufshcd_readl, val, val & UIC_COMMAND_READY,
>                    500, UIC_CMD_TIMEOUT * 1000, false, hba,
>                    REG_CONTROLLER_STATUS) == 0;
> 

Either is fine, otherwise:

Reviewed-by: Adrian Hunter <adrian.hunter@intel.com>

[RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

v2 -> v3: rule out the change of polling w/ pmc from this thread.
(I'll post the change later)
v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd

Kiwoong Kim (2):
  ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
  ufs: poll HCS.UCRDY before issuing a UIC command

 drivers/ufs/core/ufshcd.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

-- 
2.7.4

Re: [RESEND PATCH v3 0/2] change UIC command handling

[Martin K. Petersen]

Kiwoong,

> v2 -> v3: rule out the change of polling w/ pmc from this thread.
> (I'll post the change later)
> v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd
>
> Kiwoong Kim (2):
>   ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
>   ufs: poll HCS.UCRDY before issuing a UIC command

Applied to 6.6/scsi-staging, thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

RE: [RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

> > v2 -> v3: rule out the change of polling w/ pmc from this thread.
> > (I'll post the change later)
> > v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd
> >
> > Kiwoong Kim (2):
> >   ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
> >   ufs: poll HCS.UCRDY before issuing a UIC command
> 
> Applied to 6.6/scsi-staging, thanks!
> 
> --
> Martin K. Petersen	Oracle Linux Engineering

Hi, Martin

The following patch seems to make trouble because of using
read_poll_timeout.
Its initial version used udelay and after discussion it's been changed.
Could you revert this patch set?

> ufs: poll HCS.UCRDY before issuing a UIC command

[ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
kworker/u20:29/17140/0x00000002
..
[ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388
[ 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
[ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94
[ 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64
[ 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8
[ 4671.228824] [3: kworker/u20:29:17140]
schedule_hrtimeout_range_clock+0x98/0x114
[ 4671.228841] [3: kworker/u20:29:17140]  schedule_hrtimeout_range+0x14/0x24
[ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
[ 4671.228871] [3: kworker/u20:29:17140]  __ufshcd_send_uic_cmd+0xa0/0x1c4
[ 4671.228893] [3: kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390
[ 4671.228908] [3: kworker/u20:29:17140]
ufshcd_uic_hibern8_enter+0x9c/0x25c
[ 4671.228922] [3: kworker/u20:29:17140]
ufshcd_link_state_transition+0x34/0xb0
[ 4671.228939] [3: kworker/u20:29:17140]  __ufshcd_wl_suspend+0x3f0/0x4b4

Thanks you.

Re: [RESEND PATCH v3 0/2] change UIC command handling

[Adrian Hunter]

On 11/09/23 04:35, Kiwoong Kim wrote:
>>> v2 -> v3: rule out the change of polling w/ pmc from this thread.
>>> (I'll post the change later)
>>> v1 -> v2: remove an unused variable in __ufshcd_send_uic_cmd
>>>
>>> Kiwoong Kim (2):
>>>   ufs: make __ufshcd_send_uic_cmd not wrapped by host_lock
>>>   ufs: poll HCS.UCRDY before issuing a UIC command
>>
>> Applied to 6.6/scsi-staging, thanks!
>>
>> --
>> Martin K. Petersen	Oracle Linux Engineering
> 
> Hi, Martin
> 
> The following patch seems to make trouble because of using
> read_poll_timeout.
> Its initial version used udelay and after discussion it's been changed.
> Could you revert this patch set?
> 
>> ufs: poll HCS.UCRDY before issuing a UIC command
> 
> [ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
> kworker/u20:29/17140/0x00000002
> ..
> [ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388
> [ 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
> [ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94
> [ 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64
> [ 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8
> [ 4671.228824] [3: kworker/u20:29:17140]
> schedule_hrtimeout_range_clock+0x98/0x114
> [ 4671.228841] [3: kworker/u20:29:17140]  schedule_hrtimeout_range+0x14/0x24
> [ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
> [ 4671.228871] [3: kworker/u20:29:17140]  __ufshcd_send_uic_cmd+0xa0/0x1c4
> [ 4671.228893] [3: kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390
> [ 4671.228908] [3: kworker/u20:29:17140]
> ufshcd_uic_hibern8_enter+0x9c/0x25c
> [ 4671.228922] [3: kworker/u20:29:17140]
> ufshcd_link_state_transition+0x34/0xb0
> [ 4671.228939] [3: kworker/u20:29:17140]  __ufshcd_wl_suspend+0x3f0/0x4b4

Do you know what is in that path that makes it an atomic context?

RE: [RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

> >> ufs: poll HCS.UCRDY before issuing a UIC command
> >
> > [ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
> > kworker/u20:29/17140/0x00000002
> > ..
> > [ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388 [
> > 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
> > [ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94 [
> > 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64 [
> > 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8 [
> > 4671.228824] [3: kworker/u20:29:17140]
> > schedule_hrtimeout_range_clock+0x98/0x114
> > [ 4671.228841] [3: kworker/u20:29:17140]
> > schedule_hrtimeout_range+0x14/0x24
> > [ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
> > [ 4671.228871] [3: kworker/u20:29:17140]
> > __ufshcd_send_uic_cmd+0xa0/0x1c4 [ 4671.228893] [3:
> > kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390 [ 4671.228908]
> > [3: kworker/u20:29:17140] ufshcd_uic_hibern8_enter+0x9c/0x25c
> > [ 4671.228922] [3: kworker/u20:29:17140]
> > ufshcd_link_state_transition+0x34/0xb0
> > [ 4671.228939] [3: kworker/u20:29:17140]
> > __ufshcd_wl_suspend+0x3f0/0x4b4
> 
> Do you know what is in that path that makes it an atomic context?

Hi,
This made that.

static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd)
..
        bool reenable_intr = false;

        mutex_lock(&hba->uic_cmd_mutex); <<<<


At first, I was willing to post together w/ the following patch but I've got a suggestion to split the patch set because of different topic and I split the patch set.
- This patch removes the mutex, so it can fix the issue.
https://lore.kernel.org/linux-scsi/1694051306-172962-1-git-send-email-kwmad.kim@samsung.com/


But now I'm thinking again that simply removing the mutex could hurt atomicity of UIC command process
that the original code intended for the first time.
So I think this polling UCRDY should be modified rather than applying removal of the mutex.

Re: [RESEND PATCH v3 0/2] change UIC command handling

[Adrian Hunter]

On 11/09/23 09:32, Kiwoong Kim wrote:
>>>> ufs: poll HCS.UCRDY before issuing a UIC command
>>>
>>> [ 4671.226480] [3: kworker/u20:29:17140] BUG: scheduling while atomic:
>>> kworker/u20:29/17140/0x00000002
>>> ..
>>> [ 4671.228723] [3: kworker/u20:29:17140]  panic+0x16c/0x388 [
>>> 4671.228745] [3: kworker/u20:29:17140]  check_panic_on_warn+0x60/0x94
>>> [ 4671.228764] [3: kworker/u20:29:17140]  __schedule_bug+0x6c/0x94 [
>>> 4671.228786] [3: kworker/u20:29:17140]  __schedule+0x6f4/0xa64 [
>>> 4671.228806] [3: kworker/u20:29:17140]  schedule+0x7c/0xe8 [
>>> 4671.228824] [3: kworker/u20:29:17140]
>>> schedule_hrtimeout_range_clock+0x98/0x114
>>> [ 4671.228841] [3: kworker/u20:29:17140]
>>> schedule_hrtimeout_range+0x14/0x24
>>> [ 4671.228856] [3: kworker/u20:29:17140]  usleep_range_state+0x60/0x94
>>> [ 4671.228871] [3: kworker/u20:29:17140]
>>> __ufshcd_send_uic_cmd+0xa0/0x1c4 [ 4671.228893] [3:
>>> kworker/u20:29:17140]  ufshcd_uic_pwr_ctrl+0x15c/0x390 [ 4671.228908]
>>> [3: kworker/u20:29:17140] ufshcd_uic_hibern8_enter+0x9c/0x25c
>>> [ 4671.228922] [3: kworker/u20:29:17140]
>>> ufshcd_link_state_transition+0x34/0xb0
>>> [ 4671.228939] [3: kworker/u20:29:17140]
>>> __ufshcd_wl_suspend+0x3f0/0x4b4
>>
>> Do you know what is in that path that makes it an atomic context?
> 
> Hi,
> This made that.
> 
> static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command *cmd)
> ..
>         bool reenable_intr = false;
> 
>         mutex_lock(&hba->uic_cmd_mutex); <<<<

It is OK to schedule while holding a mutex.  Are you sure
this is the problem?

> 
> 
> At first, I was willing to post together w/ the following patch but I've got a suggestion to split the patch set because of different topic and I split the patch set.
> - This patch removes the mutex, so it can fix the issue.
> https://lore.kernel.org/linux-scsi/1694051306-172962-1-git-send-email-kwmad.kim@samsung.com/
> 
> 
> But now I'm thinking again that simply removing the mutex could hurt atomicity of UIC command process
> that the original code intended for the first time.
> So I think this polling UCRDY should be modified rather than applying removal of the mutex.
> 
> 
> 

RE: [RESEND PATCH v3 0/2] change UIC command handling

[Kiwoong Kim]

> > static int ufshcd_uic_pwr_ctrl(struct ufs_hba *hba, struct uic_command
> > *cmd) ..
> >         bool reenable_intr = false;
> >
> >         mutex_lock(&hba->uic_cmd_mutex); <<<<
> 
> It is OK to schedule while holding a mutex.  Are you sure this is the
> problem?

Ah, I mis-understood it. It was for not applying this.
https://lore.kernel.org/linux-scsi/782ba5f26f0a96e58d85dff50751787d2d2a6b2b.1693790060.git.kwmad.kim@samsung.com/

So this patch set has no problem. Sorry for bothering all of you.

Thanks.