[PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Guennadi Liakhovetski]

Performing MMC block IO with simultaneous STR can lead to a deadlock: the
mmc_pm_notify() function claims the host and then calls bus .remove()
method, which lands in mmc_blk_remove(), which calls mmc_blk_remove_req()
then it goes to -> mmc_cleanup_queue() -> kthread_stop(), which waits for
the mmc-block thread to stop. If the mmc-block thread at that time is
processing block requests, it will also try to claim the host in
mmc_blk_issue_rq() and block there. This patch fixes the problem by
calling .remove() before claiming the host.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/mmc/core/core.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index a2aa860..a68f085 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -2476,11 +2476,11 @@ int mmc_pm_notify(struct notifier_block *notify_block,
 		if (!host->bus_ops || host->bus_ops->suspend)
 			break;
 
-		mmc_claim_host(host);
-
+		/* On 2 occasions above bus_ops->remove() is called unlocked */
 		if (host->bus_ops->remove)
 			host->bus_ops->remove(host);
 
+		mmc_claim_host(host);
 		mmc_detach_bus(host);
 		mmc_power_off(host);
 		mmc_release_host(host);
-- 
1.7.2.5

Re: [PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Chris Ball]

Hi,

On Wed, Jan 04 2012, Guennadi Liakhovetski wrote:
> Performing MMC block IO with simultaneous STR can lead to a deadlock: the
> mmc_pm_notify() function claims the host and then calls bus .remove()
> method, which lands in mmc_blk_remove(), which calls mmc_blk_remove_req()
> then it goes to -> mmc_cleanup_queue() -> kthread_stop(), which waits for
> the mmc-block thread to stop. If the mmc-block thread at that time is
> processing block requests, it will also try to claim the host in
> mmc_blk_issue_rq() and block there. This patch fixes the problem by
> calling .remove() before claiming the host.
>
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> ---
>  drivers/mmc/core/core.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
> index a2aa860..a68f085 100644
> --- a/drivers/mmc/core/core.c
> +++ b/drivers/mmc/core/core.c
> @@ -2476,11 +2476,11 @@ int mmc_pm_notify(struct notifier_block *notify_block,
>  		if (!host->bus_ops || host->bus_ops->suspend)
>  			break;
>  
> -		mmc_claim_host(host);
> -
> +		/* On 2 occasions above bus_ops->remove() is called unlocked */
>  		if (host->bus_ops->remove)
>  			host->bus_ops->remove(host);
>  
> +		mmc_claim_host(host);
>  		mmc_detach_bus(host);
>  		mmc_power_off(host);
>  		mmc_release_host(host);

Thanks.  The commit message explanation is very good, but the comment is
a bit cryptic.  Shall we make it longer?  I think even just:

/* Calling bus_ops->remove() with a claimed host can deadlock */

would be better.

- Chris.
-- 
Chris Ball   <cjb@laptop.org>   <http://printf.net/>
One Laptop Per Child

RE: [PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Nath, Arindam]

Hi Chris,

> -----Original Message-----
> From: linux-mmc-owner@vger.kernel.org [mailto:linux-mmc-
> owner@vger.kernel.org] On Behalf Of Chris Ball
> Sent: Wednesday, January 04, 2012 8:07 PM
> To: Guennadi Liakhovetski
> Cc: linux-mmc@vger.kernel.org
> Subject: Re: [PATCH] mmc: fix a deadlock between system suspend and MMC
> block IO
> 
> Hi,
> 
> On Wed, Jan 04 2012, Guennadi Liakhovetski wrote:
> > Performing MMC block IO with simultaneous STR can lead to a deadlock:
> the
> > mmc_pm_notify() function claims the host and then calls bus .remove()
> > method, which lands in mmc_blk_remove(), which calls
> mmc_blk_remove_req()
> > then it goes to -> mmc_cleanup_queue() -> kthread_stop(), which waits
> for
> > the mmc-block thread to stop. If the mmc-block thread at that time is
> > processing block requests, it will also try to claim the host in
> > mmc_blk_issue_rq() and block there. This patch fixes the problem by
> > calling .remove() before claiming the host.
> >
> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > ---
> >  drivers/mmc/core/core.c |    4 ++--
> >  1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
> > index a2aa860..a68f085 100644
> > --- a/drivers/mmc/core/core.c
> > +++ b/drivers/mmc/core/core.c
> > @@ -2476,11 +2476,11 @@ int mmc_pm_notify(struct notifier_block
> *notify_block,
> >  		if (!host->bus_ops || host->bus_ops->suspend)
> >  			break;
> >
> > -		mmc_claim_host(host);
> > -
> > +		/* On 2 occasions above bus_ops->remove() is called
> unlocked */
> >  		if (host->bus_ops->remove)
> >  			host->bus_ops->remove(host);
> >
> > +		mmc_claim_host(host);
> >  		mmc_detach_bus(host);
> >  		mmc_power_off(host);
> >  		mmc_release_host(host);
> 
> Thanks.  The commit message explanation is very good, but the comment
> is
> a bit cryptic.  Shall we make it longer?  I think even just:
> 
> /* Calling bus_ops->remove() with a claimed host can deadlock */
> 
> would be better.

This patch will actually fix a long standing issue for most of us. But if I remember correctly, Andrei had some comments on the same patch some time back. Would it be good to include him for comments?

Thanks,
Arindam

> 
> - Chris.
> --
> Chris Ball   <cjb@laptop.org>   <http://printf.net/>
> One Laptop Per Child
> --
> To unsubscribe from this list: send the line "unsubscribe linux-mmc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Guennadi Liakhovetski]

Hi Chris

On Wed, 4 Jan 2012, Chris Ball wrote:

> Hi,
> 
> On Wed, Jan 04 2012, Guennadi Liakhovetski wrote:
> > Performing MMC block IO with simultaneous STR can lead to a deadlock: the
> > mmc_pm_notify() function claims the host and then calls bus .remove()
> > method, which lands in mmc_blk_remove(), which calls mmc_blk_remove_req()
> > then it goes to -> mmc_cleanup_queue() -> kthread_stop(), which waits for
> > the mmc-block thread to stop. If the mmc-block thread at that time is
> > processing block requests, it will also try to claim the host in
> > mmc_blk_issue_rq() and block there. This patch fixes the problem by
> > calling .remove() before claiming the host.
> >
> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > ---
> >  drivers/mmc/core/core.c |    4 ++--
> >  1 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
> > index a2aa860..a68f085 100644
> > --- a/drivers/mmc/core/core.c
> > +++ b/drivers/mmc/core/core.c
> > @@ -2476,11 +2476,11 @@ int mmc_pm_notify(struct notifier_block *notify_block,
> >  		if (!host->bus_ops || host->bus_ops->suspend)
> >  			break;
> >  
> > -		mmc_claim_host(host);
> > -
> > +		/* On 2 occasions above bus_ops->remove() is called unlocked */
> >  		if (host->bus_ops->remove)
> >  			host->bus_ops->remove(host);
> >  
> > +		mmc_claim_host(host);
> >  		mmc_detach_bus(host);
> >  		mmc_power_off(host);
> >  		mmc_release_host(host);
> 
> Thanks.  The commit message explanation is very good, but the comment is
> a bit cryptic.  Shall we make it longer?  I think even just:
> 
> /* Calling bus_ops->remove() with a claimed host can deadlock */
> 
> would be better.

Right, agree. Would you replace it yourself or should I send a v2?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Chris Ball]

Hi,

Adding Andrei to CC.  Thanks,

- Chris.

On Wed, Jan 04 2012, Nath, Arindam wrote:
> Hi Chris,
>
>> -----Original Message-----
>> From: linux-mmc-owner@vger.kernel.org [mailto:linux-mmc-
>> owner@vger.kernel.org] On Behalf Of Chris Ball
>> Sent: Wednesday, January 04, 2012 8:07 PM
>> To: Guennadi Liakhovetski
>> Cc: linux-mmc@vger.kernel.org
>> Subject: Re: [PATCH] mmc: fix a deadlock between system suspend and MMC
>> block IO
>> 
>> Hi,
>> 
>> On Wed, Jan 04 2012, Guennadi Liakhovetski wrote:
>> > Performing MMC block IO with simultaneous STR can lead to a deadlock:
>> the
>> > mmc_pm_notify() function claims the host and then calls bus .remove()
>> > method, which lands in mmc_blk_remove(), which calls
>> mmc_blk_remove_req()
>> > then it goes to -> mmc_cleanup_queue() -> kthread_stop(), which waits
>> for
>> > the mmc-block thread to stop. If the mmc-block thread at that time is
>> > processing block requests, it will also try to claim the host in
>> > mmc_blk_issue_rq() and block there. This patch fixes the problem by
>> > calling .remove() before claiming the host.
>> >
>> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
>> > ---
>> >  drivers/mmc/core/core.c |    4 ++--
>> >  1 files changed, 2 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
>> > index a2aa860..a68f085 100644
>> > --- a/drivers/mmc/core/core.c
>> > +++ b/drivers/mmc/core/core.c
>> > @@ -2476,11 +2476,11 @@ int mmc_pm_notify(struct notifier_block
>> *notify_block,
>> >  		if (!host->bus_ops || host->bus_ops->suspend)
>> >  			break;
>> >
>> > -		mmc_claim_host(host);
>> > -
>> > +		/* On 2 occasions above bus_ops->remove() is called
>> unlocked */
>> >  		if (host->bus_ops->remove)
>> >  			host->bus_ops->remove(host);
>> >
>> > +		mmc_claim_host(host);
>> >  		mmc_detach_bus(host);
>> >  		mmc_power_off(host);
>> >  		mmc_release_host(host);
>> 
>> Thanks.  The commit message explanation is very good, but the comment
>> is
>> a bit cryptic.  Shall we make it longer?  I think even just:
>> 
>> /* Calling bus_ops->remove() with a claimed host can deadlock */
>> 
>> would be better.
>
> This patch will actually fix a long standing issue for most of us. But
> if I remember correctly, Andrei had some comments on the same patch
> some time back. Would it be good to include him for comments?

-- 
Chris Ball   <cjb@laptop.org>   <http://printf.net/>
One Laptop Per Child

RE: [PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Nath, Arindam]

Hi Chris,

> -----Original Message-----
> From: Chris Ball [mailto:cjb@laptop.org]
> Sent: Wednesday, January 04, 2012 8:34 PM
> To: Nath, Arindam
> Cc: Guennadi Liakhovetski; linux-mmc@vger.kernel.org; Andrei Warkentin
> Subject: Re: [PATCH] mmc: fix a deadlock between system suspend and MMC
> block IO
> 
> Hi,
> 
> Adding Andrei to CC.  Thanks,
> 
> - Chris.
> 
> On Wed, Jan 04 2012, Nath, Arindam wrote:
> > Hi Chris,
> >
> >> -----Original Message-----
> >> From: linux-mmc-owner@vger.kernel.org [mailto:linux-mmc-
> >> owner@vger.kernel.org] On Behalf Of Chris Ball
> >> Sent: Wednesday, January 04, 2012 8:07 PM
> >> To: Guennadi Liakhovetski
> >> Cc: linux-mmc@vger.kernel.org
> >> Subject: Re: [PATCH] mmc: fix a deadlock between system suspend and
> MMC
> >> block IO
> >>
> >> Hi,
> >>
> >> On Wed, Jan 04 2012, Guennadi Liakhovetski wrote:
> >> > Performing MMC block IO with simultaneous STR can lead to a
> deadlock:
> >> the
> >> > mmc_pm_notify() function claims the host and then calls bus
> .remove()
> >> > method, which lands in mmc_blk_remove(), which calls
> >> mmc_blk_remove_req()
> >> > then it goes to -> mmc_cleanup_queue() -> kthread_stop(), which
> waits
> >> for
> >> > the mmc-block thread to stop. If the mmc-block thread at that time
> is
> >> > processing block requests, it will also try to claim the host in
> >> > mmc_blk_issue_rq() and block there. This patch fixes the problem
> by
> >> > calling .remove() before claiming the host.
> >> >
> >> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> >> > ---
> >> >  drivers/mmc/core/core.c |    4 ++--
> >> >  1 files changed, 2 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
> >> > index a2aa860..a68f085 100644
> >> > --- a/drivers/mmc/core/core.c
> >> > +++ b/drivers/mmc/core/core.c
> >> > @@ -2476,11 +2476,11 @@ int mmc_pm_notify(struct notifier_block
> >> *notify_block,
> >> >  		if (!host->bus_ops || host->bus_ops->suspend)
> >> >  			break;
> >> >
> >> > -		mmc_claim_host(host);
> >> > -
> >> > +		/* On 2 occasions above bus_ops->remove() is called
> >> unlocked */
> >> >  		if (host->bus_ops->remove)
> >> >  			host->bus_ops->remove(host);
> >> >
> >> > +		mmc_claim_host(host);
> >> >  		mmc_detach_bus(host);
> >> >  		mmc_power_off(host);
> >> >  		mmc_release_host(host);
> >>
> >> Thanks.  The commit message explanation is very good, but the
> comment
> >> is
> >> a bit cryptic.  Shall we make it longer?  I think even just:
> >>
> >> /* Calling bus_ops->remove() with a claimed host can deadlock */
> >>
> >> would be better.
> >
> > This patch will actually fix a long standing issue for most of us.
> But
> > if I remember correctly, Andrei had some comments on the same patch
> > some time back. Would it be good to include him for comments?

I think we can push this patch now. If someone has any objections, we can discuss later.

Acked-by: Arindam Nath <arindam.nath@amd.com>

Thanks,
Arindam

> 
> --
> Chris Ball   <cjb@laptop.org>   <http://printf.net/>
> One Laptop Per Child

Re: [PATCH] mmc: fix a deadlock between system suspend and MMC block IO

[Chris Ball]

Hi,

On Wed, Jan 11 2012, Nath, Arindam wrote:
> I think we can push this patch now. If someone has any objections, we
> can discuss later.
>
> Acked-by: Arindam Nath <arindam.nath@amd.com>

Thanks, I agree.  Guennadi, I've merged it for 3.3 with the following change:

diff --git a/drivers/mmc/core/core.c b/drivers/mmc/core/core.c
index f0b464f..bec0bf2 100644
--- a/drivers/mmc/core/core.c
+++ b/drivers/mmc/core/core.c
@@ -2175,6 +2175,7 @@ void mmc_stop_host(struct mmc_host *host)
 
 	mmc_bus_get(host);
 	if (host->bus_ops && !host->bus_dead) {
+		/* Calling bus_ops->remove() with a claimed host can deadlock */
 		if (host->bus_ops->remove)
 			host->bus_ops->remove(host);
 
@@ -2398,7 +2399,9 @@ int mmc_suspend_host(struct mmc_host *host)
 			if (err == -ENOSYS || !host->bus_ops->resume) {
 				/*
 				 * We simply "remove" the card in this case.
-				 * It will be redetected on resume.
+				 * It will be redetected on resume.  (Calling
+				 * bus_ops->remove() with a claimed host can
+				 * deadlock.)
 				 */
 				if (host->bus_ops->remove)
 					host->bus_ops->remove(host);
@@ -2491,7 +2494,7 @@ int mmc_pm_notify(struct notifier_block *notify_block,
 		if (!host->bus_ops || host->bus_ops->suspend)
 			break;
 
-		/* On 2 occasions above bus_ops->remove() is called unlocked */
+		/* Calling bus_ops->remove() with a claimed host can deadlock */
 		if (host->bus_ops->remove)
 			host->bus_ops->remove(host);
 
-- 
Chris Ball   <cjb@laptop.org>   <http://printf.net/>
One Laptop Per Child