Re: SDHC card affected by preemption model in 2.6.35

Re: SDHC card affected by preemption model in 2.6.35

[Johannes Reif]

Hello Venkat, Hello Mathieu,

>  No. I was working on some other things lately. What card are you using and
>  how often are you seeing this ?
>
>  Regards,
>  Venkat.

We get the timeout Problem every time:
[    1.888732] Waiting for root device /dev/mmcblk0p2...
[    4.193603] mmc0: error -110 whilst initialising SD card

Right now the issue occurs with following cards CONFIG_PREEMPT=y enabled:

- SanDisk 4GB micro SDHC Class 2
- SanDisk 2GB micro SD


These cards worked fine:

- Transcend 2GB micro SD
- Transcend 4GB micro SDHC Class 6
- SanDisk 4GB micro SDHC Class 6 (Mobile Ultra)


Thanks,
Johannes

Re: SDHC card affected by preemption model in 2.6.35

[Dave Hylands]

Hi guys,

On Mon, Feb 21, 2011 at 4:53 AM, Johannes Reif <reif@corscience.de> wrote:
> Hello Venkat, Hello Mathieu,
>
>>  No. I was working on some other things lately. What card are you using
>> and
>>  how often are you seeing this ?
>>
>>  Regards,
>>  Venkat.
>
> We get the timeout Problem every time:
> [    1.888732] Waiting for root device /dev/mmcblk0p2...
> [    4.193603] mmc0: error -110 whilst initialising SD card
>
> Right now the issue occurs with following cards CONFIG_PREEMPT=y enabled:

There is a bug in the kernel workqueues. I observed it in 2.6.36.3 (it
seems to have been introduced in 2.6.36 and is still in 2.6.37.1), One
of my colleagues was investigating and contacted the author of the
workqueue code (Tejun Heo). Tejun sent us the following patch:

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 11869fa..90a17ca 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -2047,6 +2047,15 @@ repeat:
 				move_linked_works(work, scheduled, &n);

 		process_scheduled_works(rescuer);
+
+		/*
+		 * Leave this gcwq.  If keep_working() is %true, notify a
+		 * regular worker; otherwise, we end up with 0 concurrency
+		 * and stalling the execution.
+		 */
+		if (keep_working(gcwq))
+			wake_up_worker(gcwq);
+
 		spin_unlock_irq(&gcwq->lock);
 	}


For us, this was causing card insertion events to not be processed.
For us it was a race condition and depended on when the timer tick
occured in relation to other processing, and since you're talking
about PREEMPTION making a difference, I thought I would throw this out
as maybe being relevant.

Dave Hylands
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SDHC card affected by preemption model in 2.6.35

[Johannes Reif]

Hi all,

On 22.02.2011 02:43, Dave Hylands wrote:

> Hi guys,
>
> There is a bug in the kernel workqueues. I observed it in 2.6.36.3 (it
> seems to have been introduced in 2.6.36 and is still in 2.6.37.1), One
> of my colleagues was investigating and contacted the author of the
> workqueue code (Tejun Heo). Tejun sent us the following patch:
>
> diff --git a/kernel/workqueue.c b/kernel/workqueue.c
> index 11869fa..90a17ca 100644
> --- a/kernel/workqueue.c
> +++ b/kernel/workqueue.c
> @@ -2047,6 +2047,15 @@ repeat:
>   				move_linked_works(work, scheduled,&n);
>
>   		process_scheduled_works(rescuer);
> +
> +		/*
> +		 * Leave this gcwq.  If keep_working() is %true, notify a
> +		 * regular worker; otherwise, we end up with 0 concurrency
> +		 * and stalling the execution.
> +		 */
> +		if (keep_working(gcwq))
> +			wake_up_worker(gcwq);
> +
>   		spin_unlock_irq(&gcwq->lock);
>   	}
>
>
> For us, this was causing card insertion events to not be processed.
> For us it was a race condition and depended on when the timer tick
> occured in relation to other processing, and since you're talking
> about PREEMPTION making a difference, I thought I would throw this out
> as maybe being relevant.
>
> Dave Hylands

unfortunately the cards didn't work with the patch.

We tried updating from 2.6.37-rc8 to 2.6.38-rc5 and the timeout error still occurs. However the cards get detected after the error and seem to work fine:

[    2.456512] Waiting for root device /dev/mmcblk0p2...
[    4.803375] mmc0: error -110 whilst initialising SD card
[    5.201019] mmc0: new SD card at address aaaa
[    5.206665] mmcblk0: mmc0:aaaa SU02G 1.84 GiB
[    5.214477]  mmcblk0: p1 p2

Regards,
Johannes

SDHC card affected by preemption model in 2.6.35

[Mathieu Poirier]

HW: Beagleboard rev. C2 and C4
Processor: OMAP3
Kernel: 2.6.35-rc2
Driver: mmci-omap-hs

I am faced with an SDHC card problem on a beagleboard.  Some cards
cannot be initialized on startup while others work perfectly.  I tracked
the issue down to one single kernel config: PREEMPT_VOLUNTARY.  

When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.

When booting, a failing card (PREEMPT_VOLUNTARY) will output the
following: 
[ 2.283355] mmc0: error -110 whilst initialising SD card

I have also seen transfer errors such as this one: 
[ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
26, card status 0xc00 

When working properly (PREEMPT_NONE), you get: 
[   27.026519] mmc0: new high speed SDHC card at address 0007
[   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB 

We seem to have a little timing problem - has anyone seen the same
issue ?  Can driver "mmci-omap-hs" actually work under
PREEMPT_VOLUNTARY ? 

Thanks, Mathieu.

Re: SDHC card affected by preemption model in 2.6.35

[Venkatraman S]

 Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
> HW: Beagleboard rev. C2 and C4
> Processor: OMAP3
> Kernel: 2.6.35-rc2
> Driver: mmci-omap-hs
>
> I am faced with an SDHC card problem on a beagleboard.  Some cards
> cannot be initialized on startup while others work perfectly.  I tracked
> the issue down to one single kernel config: PREEMPT_VOLUNTARY.
>
> When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.
>
> When booting, a failing card (PREEMPT_VOLUNTARY) will output the
> following:
> [ 2.283355] mmc0: error -110 whilst initialising SD card
>
> I have also seen transfer errors such as this one:
> [ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
> 26, card status 0xc00
>
> When working properly (PREEMPT_NONE), you get:
> [   27.026519] mmc0: new high speed SDHC card at address 0007
> [   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB
>
> We seem to have a little timing problem - has anyone seen the same
> issue ?  Can driver "mmci-omap-hs" actually work under
> PREEMPT_VOLUNTARY ?
>
> Thanks, Mathieu.
>

I will check this out - not tested with PREEMPT_VOLUNTARY so far.
If it's not too much trouble, can you provide a log with CONFIG_MMC_DEBUG ?
Also, some details about the failing card would be helpful.

Thanks,
Venkat.
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SDHC card affected by preemption model in 2.6.35

[Mathieu Poirier]

On Tue, 2010-06-15 at 20:58 +0530, Venkatraman S wrote:
> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
> > HW: Beagleboard rev. C2 and C4
> > Processor: OMAP3
> > Kernel: 2.6.35-rc2
> > Driver: mmci-omap-hs
> >
> > I am faced with an SDHC card problem on a beagleboard.  Some cards
> > cannot be initialized on startup while others work perfectly.  I tracked
> > the issue down to one single kernel config: PREEMPT_VOLUNTARY.
> >
> > When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.
> >
> > When booting, a failing card (PREEMPT_VOLUNTARY) will output the
> > following:
> > [ 2.283355] mmc0: error -110 whilst initialising SD card
> >
> > I have also seen transfer errors such as this one:
> > [ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
> > 26, card status 0xc00
> >
> > When working properly (PREEMPT_NONE), you get:
> > [   27.026519] mmc0: new high speed SDHC card at address 0007
> > [   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB
> >
> > We seem to have a little timing problem - has anyone seen the same
> > issue ?  Can driver "mmci-omap-hs" actually work under
> > PREEMPT_VOLUNTARY ?
> >
> > Thanks, Mathieu.
> >
> 
> I will check this out - not tested with PREEMPT_VOLUNTARY so far.
> If it's not too much trouble, can you provide a log with CONFIG_MMC_DEBUG ?
> Also, some details about the failing card would be helpful.
> 
> Thanks,
> Venkat.

Venkat,

Unfortunately enabling CONFIG_MMC_DEBUG doesn't yield more information -
the error message is the same and no additional output shows on the
console.

As for the cards, had failures with 3 different manufacturer:
- Patriot Memory, MicroSD card, 8GB, class 4, SDHC.  
- Kinston, 4GB, class 6, SDHC.
- Sandisk, 4GB, Class 2, SDHC.

I am more than willing to test kernels for you if need be.

Thanks, Mathieu.

Re: SDHC card affected by preemption model in 2.6.35

[David Brownell]

> > > When going from PREEMPT_VOLUNTARY to PREEMPT_NONE
> the problem goes away.

Have you found whether this is a problem
with the OMAP HSMMC driver
versus the MMC/SD framework code

Re: SDHC card affected by preemption model in 2.6.35

[Mathieu Poirier]

On Tue, 2010-06-15 at 14:55 -0700, David Brownell wrote:
> > > > When going from PREEMPT_VOLUNTARY to PREEMPT_NONE
> > the problem goes away.
> 
> Have you found whether this is a problem
> with the OMAP HSMMC driver
> versus the MMC/SD framework code
> 
Not yet - I'm in the code, investigating.

Re: SDHC card affected by preemption model in 2.6.35

[Venkatraman S]

 Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
> On Tue, 2010-06-15 at 20:58 +0530, Venkatraman S wrote:
>> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
>> > HW: Beagleboard rev. C2 and C4
>> > Processor: OMAP3
>> > Kernel: 2.6.35-rc2
>> > Driver: mmci-omap-hs
>> >
>> > I am faced with an SDHC card problem on a beagleboard.  Some cards
>> > cannot be initialized on startup while others work perfectly.  I tracked
>> > the issue down to one single kernel config: PREEMPT_VOLUNTARY.
>> >
>> > When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.
>> >
>> > When booting, a failing card (PREEMPT_VOLUNTARY) will output the
>> > following:
>> > [ 2.283355] mmc0: error -110 whilst initialising SD card
>> >
>> > I have also seen transfer errors such as this one:
>> > [ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
>> > 26, card status 0xc00
>> >
>> > When working properly (PREEMPT_NONE), you get:
>> > [   27.026519] mmc0: new high speed SDHC card at address 0007
>> > [   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB
>> >
>> > We seem to have a little timing problem - has anyone seen the same
>> > issue ?  Can driver "mmci-omap-hs" actually work under
>> > PREEMPT_VOLUNTARY ?
>> >
>> > Thanks, Mathieu.
>> >
>>
>> I will check this out - not tested with PREEMPT_VOLUNTARY so far.
>> If it's not too much trouble, can you provide a log with CONFIG_MMC_DEBUG ?
>> Also, some details about the failing card would be helpful.
>>
>> Thanks,
>> Venkat.
>
> Venkat,
>
> Unfortunately enabling CONFIG_MMC_DEBUG doesn't yield more information -
> the error message is the same and no additional output shows on the
> console.
>
> As for the cards, had failures with 3 different manufacturer:
> - Patriot Memory, MicroSD card, 8GB, class 4, SDHC.
> - Kinston, 4GB, class 6, SDHC.
> - Sandisk, 4GB, Class 2, SDHC.
>
> I am more than willing to test kernels for you if need be.
>
> Thanks, Mathieu.
>

For MMC/SD logs to be sent to the console, you need to
a) "echo 8 > /proc/sys/kernel/printk" in the shell and
b) insert the card

If you are booting from the card itself, then this won't work and
DEBUG_LL has to be enabled (in addition to CONFIG_MMC_DEBUG)

Apologies - I should have explained this initially.

Regards,
Venkat.
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SDHC card affected by preemption model in 2.6.35

[Mathieu Poirier]

On Wed, 2010-06-16 at 14:13 +0530, Venkatraman S wrote:
> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
> > On Tue, 2010-06-15 at 20:58 +0530, Venkatraman S wrote:
> >> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
> >> > HW: Beagleboard rev. C2 and C4
> >> > Processor: OMAP3
> >> > Kernel: 2.6.35-rc2
> >> > Driver: mmci-omap-hs
> >> >
> >> > I am faced with an SDHC card problem on a beagleboard.  Some cards
> >> > cannot be initialized on startup while others work perfectly.  I tracked
> >> > the issue down to one single kernel config: PREEMPT_VOLUNTARY.
> >> >
> >> > When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.
> >> >
> >> > When booting, a failing card (PREEMPT_VOLUNTARY) will output the
> >> > following:
> >> > [ 2.283355] mmc0: error -110 whilst initialising SD card
> >> >
> >> > I have also seen transfer errors such as this one:
> >> > [ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
> >> > 26, card status 0xc00
> >> >
> >> > When working properly (PREEMPT_NONE), you get:
> >> > [   27.026519] mmc0: new high speed SDHC card at address 0007
> >> > [   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB
> >> >
> >> > We seem to have a little timing problem - has anyone seen the same
> >> > issue ?  Can driver "mmci-omap-hs" actually work under
> >> > PREEMPT_VOLUNTARY ?
> >> >
> >> > Thanks, Mathieu.
> >> >
> >>
> >> I will check this out - not tested with PREEMPT_VOLUNTARY so far.
> >> If it's not too much trouble, can you provide a log with CONFIG_MMC_DEBUG ?
> >> Also, some details about the failing card would be helpful.
> >>
> >> Thanks,
> >> Venkat.
> >
> > Venkat,
> >
> > Unfortunately enabling CONFIG_MMC_DEBUG doesn't yield more information -
> > the error message is the same and no additional output shows on the
> > console.
> >
> > As for the cards, had failures with 3 different manufacturer:
> > - Patriot Memory, MicroSD card, 8GB, class 4, SDHC.
> > - Kinston, 4GB, class 6, SDHC.
> > - Sandisk, 4GB, Class 2, SDHC.
> >
> > I am more than willing to test kernels for you if need be.
> >
> > Thanks, Mathieu.
> >
> 
> For MMC/SD logs to be sent to the console, you need to
> a) "echo 8 > /proc/sys/kernel/printk" in the shell and
> b) insert the card
> 
> If you are booting from the card itself, then this won't work and
> DEBUG_LL has to be enabled (in addition to CONFIG_MMC_DEBUG)
> 
> Apologies - I should have explained this initially.
> 
> Regards,
> Venkat.

Venkat,

I am indeed booting from the card itself, making things more difficult.
DEBUG_LL has been configured since the very beginning and still not much
to look at on the console.  To see something I had to pass loglevel=8 on
the kernel command line.  At that point there is tones of stuff coming
out and the card is initialized properly, which points to a timing
issue. 

Since I couldn't reproduce the failure when debug messages are enabled,
I turned them back off and started to instrument the code on the hunt
for the failure. 

I have cornered the source of the problem in
"drivers/mmc/core/sd_ops.c", function "mmc_sd_switch".  When the kernel
is configured with PREEMPT_NONE, the value of "data.error" is set to "0"
after "mmc_wait_for_req" returns.  When PREEMPT_VOLUNTARY is configured,
"data.error" gets set to "-110" upon "mmc_wait_for_req" returning, which
prevent the remaining of the configuration to take place.

I am out of time for today but tomorrow I'll dive in "mmc_wait_for_req".
Send me a line if the above rings a bell or you find something.

Ghorai, please find an answer to your question in the above trail.

Thanks, Mathieu. 

Re: SDHC card affected by preemption model in 2.6.35

[Venkatraman S]

 Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
> On Wed, 2010-06-16 at 14:13 +0530, Venkatraman S wrote:
>> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
>> > On Tue, 2010-06-15 at 20:58 +0530, Venkatraman S wrote:
>> >> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
>> >> > HW: Beagleboard rev. C2 and C4
>> >> > Processor: OMAP3
>> >> > Kernel: 2.6.35-rc2
>> >> > Driver: mmci-omap-hs
>> >> >
>> >> > I am faced with an SDHC card problem on a beagleboard.  Some cards
>> >> > cannot be initialized on startup while others work perfectly.  I tracked
>> >> > the issue down to one single kernel config: PREEMPT_VOLUNTARY.
>> >> >
>> >> > When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.
>> >> >
>> >> > When booting, a failing card (PREEMPT_VOLUNTARY) will output the
>> >> > following:
>> >> > [ 2.283355] mmc0: error -110 whilst initialising SD card
>> >> >
>> >> > I have also seen transfer errors such as this one:
>> >> > [ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
>> >> > 26, card status 0xc00
>> >> >
>> >> > When working properly (PREEMPT_NONE), you get:
>> >> > [   27.026519] mmc0: new high speed SDHC card at address 0007
>> >> > [   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB
>> >> >
>> >> > We seem to have a little timing problem - has anyone seen the same
>> >> > issue ?  Can driver "mmci-omap-hs" actually work under
>> >> > PREEMPT_VOLUNTARY ?
>> >> >
>> >> > Thanks, Mathieu.
>> >> >
>> >>
>> >> I will check this out - not tested with PREEMPT_VOLUNTARY so far.
>> >> If it's not too much trouble, can you provide a log with CONFIG_MMC_DEBUG ?
>> >> Also, some details about the failing card would be helpful.
>> >>
>> >> Thanks,
>> >> Venkat.
>> >
>> > Venkat,
>> >
>> > Unfortunately enabling CONFIG_MMC_DEBUG doesn't yield more information -
>> > the error message is the same and no additional output shows on the
>> > console.
>> >
>> > As for the cards, had failures with 3 different manufacturer:
>> > - Patriot Memory, MicroSD card, 8GB, class 4, SDHC.
>> > - Kinston, 4GB, class 6, SDHC.
>> > - Sandisk, 4GB, Class 2, SDHC.
>> >
>> > I am more than willing to test kernels for you if need be.
>> >
>> > Thanks, Mathieu.
>> >
>>
>> For MMC/SD logs to be sent to the console, you need to
>> a) "echo 8 > /proc/sys/kernel/printk" in the shell and
>> b) insert the card
>>
>> If you are booting from the card itself, then this won't work and
>> DEBUG_LL has to be enabled (in addition to CONFIG_MMC_DEBUG)
>>
>> Apologies - I should have explained this initially.
>>
>> Regards,
>> Venkat.
>
> Venkat,
>
> I am indeed booting from the card itself, making things more difficult.
> DEBUG_LL has been configured since the very beginning and still not much
> to look at on the console.  To see something I had to pass loglevel=8 on
> the kernel command line.  At that point there is tones of stuff coming
> out and the card is initialized properly, which points to a timing
> issue.
>
> Since I couldn't reproduce the failure when debug messages are enabled,
> I turned them back off and started to instrument the code on the hunt
> for the failure.
>
> I have cornered the source of the problem in
> "drivers/mmc/core/sd_ops.c", function "mmc_sd_switch".  When the kernel
> is configured with PREEMPT_NONE, the value of "data.error" is set to "0"
> after "mmc_wait_for_req" returns.  When PREEMPT_VOLUNTARY is configured,
> "data.error" gets set to "-110" upon "mmc_wait_for_req" returning, which
> prevent the remaining of the configuration to take place.
>

-110 = -ETIMEDOUT.
The error is set from omap_hsmmc.c line 1029, when the controller
returns a DATA_TIMEOUT error.
This can happen for any mmc_wait_for_req() call, not just in mmc_sd_switch().

A simple workaround is to increase the timeout value, as below. Could
you please try with this patch ?
Yes, it's a debugging hack and not the right solution.
[I have so far not reproduced the problem with my setup. Still trying..]

----
diff --git a/drivers/mmc/host/omap_hsmmc.c b/drivers/mmc/host/omap_hsmmc.c
index b032828..9ca399e 100644
--- a/drivers/mmc/host/omap_hsmmc.c
+++ b/drivers/mmc/host/omap_hsmmc.c
@@ -1413,7 +1413,7 @@ omap_hsmmc_prepare_data(struct omap_hsmmc_host
*host, struct mmc_request *req)

        OMAP_HSMMC_WRITE(host->base, BLK, (req->data->blksz)
                                        | (req->data->blocks << 16));
-       set_data_timeout(host, req->data->timeout_ns, req->data->timeout_clks);
+       set_data_timeout(host, 100000000U, 0);

        if (host->use_dma) {
                ret = omap_hsmmc_start_dma_transfer(host, req);
----

  This provides a good 100ms window for delays due to scheduling variations.
I have to still find out which section is sensitive to it.
Regards,
Venkat.
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SDHC card affected by preemption model in 2.6.35

[Thomas Weber]

Hello Mathieu, hello Venkat,

I hope it is not too old and you remember this:

Am 17.06.2010 16:33, schrieb Venkatraman S:
>  Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
>> On Wed, 2010-06-16 at 14:13 +0530, Venkatraman S wrote:
>>> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
>>>> On Tue, 2010-06-15 at 20:58 +0530, Venkatraman S wrote:
>>>>> Mathieu Poirier <mathieu.poirier@canonical.com> wrote:
>>>>>> HW: Beagleboard rev. C2 and C4
>>>>>> Processor: OMAP3
>>>>>> Kernel: 2.6.35-rc2
>>>>>> Driver: mmci-omap-hs
>>>>>>
>>>>>> I am faced with an SDHC card problem on a beagleboard.  Some cards
>>>>>> cannot be initialized on startup while others work perfectly.  I tracked
>>>>>> the issue down to one single kernel config: PREEMPT_VOLUNTARY.
>>>>>>
>>>>>> When going from PREEMPT_VOLUNTARY to PREEMPT_NONE the problem goes away.
>>>>>>
>>>>>> When booting, a failing card (PREEMPT_VOLUNTARY) will output the
>>>>>> following:
>>>>>> [ 2.283355] mmc0: error -110 whilst initialising SD card
>>>>>>
>>>>>> I have also seen transfer errors such as this one:
>>>>>> [ 105.343780] mmcblk0: error -110 transferring data, sector 798431, nr
>>>>>> 26, card status 0xc00
>>>>>>
>>>>>> When working properly (PREEMPT_NONE), you get:
>>>>>> [   27.026519] mmc0: new high speed SDHC card at address 0007
>>>>>> [   27.075775] mmcblk0: mmc0:0007 SD08G 7.49 GiB
>>>>>>
>>>>>> We seem to have a little timing problem - has anyone seen the same
>>>>>> issue ?  Can driver "mmci-omap-hs" actually work under
>>>>>> PREEMPT_VOLUNTARY ?
>>>>>>
>>>>>> Thanks, Mathieu.
>>>>>>
>>>>>
>>>>> I will check this out - not tested with PREEMPT_VOLUNTARY so far.
>>>>> If it's not too much trouble, can you provide a log with CONFIG_MMC_DEBUG ?
>>>>> Also, some details about the failing card would be helpful.
>>>>>
>>>>> Thanks,
>>>>> Venkat.
>>>>
>>>> Venkat,
>>>>
>>>> Unfortunately enabling CONFIG_MMC_DEBUG doesn't yield more information -
>>>> the error message is the same and no additional output shows on the
>>>> console.
>>>>
>>>> As for the cards, had failures with 3 different manufacturer:
>>>> - Patriot Memory, MicroSD card, 8GB, class 4, SDHC.
>>>> - Kinston, 4GB, class 6, SDHC.
>>>> - Sandisk, 4GB, Class 2, SDHC.
>>>>
>>>> I am more than willing to test kernels for you if need be.
>>>>
>>>> Thanks, Mathieu.
>>>>
>>>
>>> For MMC/SD logs to be sent to the console, you need to
>>> a) "echo 8 > /proc/sys/kernel/printk" in the shell and
>>> b) insert the card
>>>
>>> If you are booting from the card itself, then this won't work and
>>> DEBUG_LL has to be enabled (in addition to CONFIG_MMC_DEBUG)
>>>
>>> Apologies - I should have explained this initially.
>>>
>>> Regards,
>>> Venkat.
>>
>> Venkat,
>>
>> I am indeed booting from the card itself, making things more difficult.
>> DEBUG_LL has been configured since the very beginning and still not much
>> to look at on the console.  To see something I had to pass loglevel=8 on
>> the kernel command line.  At that point there is tones of stuff coming
>> out and the card is initialized properly, which points to a timing
>> issue.
>>
>> Since I couldn't reproduce the failure when debug messages are enabled,
>> I turned them back off and started to instrument the code on the hunt
>> for the failure.
>>
>> I have cornered the source of the problem in
>> "drivers/mmc/core/sd_ops.c", function "mmc_sd_switch".  When the kernel
>> is configured with PREEMPT_NONE, the value of "data.error" is set to "0"
>> after "mmc_wait_for_req" returns.  When PREEMPT_VOLUNTARY is configured,
>> "data.error" gets set to "-110" upon "mmc_wait_for_req" returning, which
>> prevent the remaining of the configuration to take place.
>>
> 
> -110 = -ETIMEDOUT.
> The error is set from omap_hsmmc.c line 1029, when the controller
> returns a DATA_TIMEOUT error.
> This can happen for any mmc_wait_for_req() call, not just in mmc_sd_switch().
> 
> A simple workaround is to increase the timeout value, as below. Could
> you please try with this patch ?
> Yes, it's a debugging hack and not the right solution.
> [I have so far not reproduced the problem with my setup. Still trying..]
> 
> ----
> diff --git a/drivers/mmc/host/omap_hsmmc.c b/drivers/mmc/host/omap_hsmmc.c
> index b032828..9ca399e 100644
> --- a/drivers/mmc/host/omap_hsmmc.c
> +++ b/drivers/mmc/host/omap_hsmmc.c
> @@ -1413,7 +1413,7 @@ omap_hsmmc_prepare_data(struct omap_hsmmc_host
> *host, struct mmc_request *req)
> 
>         OMAP_HSMMC_WRITE(host->base, BLK, (req->data->blksz)
>                                         | (req->data->blocks << 16));
> -       set_data_timeout(host, req->data->timeout_ns, req->data->timeout_clks);
> +       set_data_timeout(host, 100000000U, 0);
> 
>         if (host->use_dma) {
>                 ret = omap_hsmmc_start_dma_transfer(host, req);
> ----
> 
>   This provides a good 100ms window for delays due to scheduling variations.
> I have to still find out which section is sensitive to it.
> Regards,
> Venkat.
> --

We have a custom omap3 board and use kernel 2.6.37-rc8.  We have the
problem that we sometimes cannot mount our rootfs from sd-card. This
happens only when CONFIG_PREEMPT=y. When using CONFIG_PREEMPT_NONE or
enabling CONFIG_MMC_DEBUG everything works fine.

The "100 ms patch" doesn't work for us. Do you found other solution(s)?

Regards,
Thomas

Re: SDHC card affected by preemption model in 2.6.35

[S, Venkatraman]

On Sun, Feb 20, 2011 at 10:42 PM, S, Venkatraman <svenkatr@ti.com> wrote:
>
> On Fri, Feb 18, 2011 at 6:27 PM, Thomas Weber <weber@corscience.de> wrote:
>>
>> Hello Mathieu, hello Venkat,
>>
>> I hope it is not too old and you remember this:
>> >
>> >   This provides a good 100ms window for delays due to scheduling variations.
>> > I have to still find out which section is sensitive to it.
>> > Regards,
>> > Venkat.
>> > --
>>
>> We have a custom omap3 board and use kernel 2.6.37-rc8.  We have the
>> problem that we sometimes cannot mount our rootfs from sd-card. This
>> happens only when CONFIG_PREEMPT=y. When using CONFIG_PREEMPT_NONE or
>> enabling CONFIG_MMC_DEBUG everything works fine.
>>
>> The "100 ms patch" doesn't work for us. Do you found other solution(s)?
>>

No - I was working on some other things lately - I'll check this out this week.
What type of card are you using ?
Regards,
Venkat.
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html