[PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

Function mxcmci_request() throw an exception on a imx27 cpu.
This patch fix the problem by test the pointer before access it.

[  516.783407] Unable to handle kernel NULL pointer dereference at virtual address 00000004
[  516.791639] pgd = c0004000
[  516.794445] [00000004] *pgd=00000000
[  516.798088] Internal error: Oops: 17 [#1] ARM
[  516.802472] Modules linked in:
[  516.805593] CPU: 0 PID: 569 Comm: mmcqd/0 Not tainted 3.13.0-rc1-next-20131125-00006-g5f6bb77-dirty #66
[  516.815027] task: cfb8a2a0 ti: cfb88000 task.ti: cfb88000
[  516.820484] PC is at mxcmci_request+0xd0/0x2f8
[  516.824974] LR is at mxcmci_request+0xf8/0x2f8
[  516.829466] pc : [<c02b6048>]    lr : [<c02b6070>]    psr: 00000013
[  516.829466] sp : cfb89dd0  ip : cfb89dd0  fp : cfb89e1c
[  516.840988] r10: 000001ff  r9 : 00000002  r8 : cfa10620
[  516.846250] r7 : 00000200  r6 : cfa0f02c  r5 : cfa0f0ec  r4 : cfa10400
[  516.852812] r3 : a0000013  r2 : cfa0f410  r1 : 00040000  r0 : 00000000
[  516.859376] Flags: nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
[  516.866723] Control: 0005317f  Table: afa74000  DAC: 00000017
[  516.872503] Process mmcqd/0 (pid: 569, stack limit = 0xcfb881c0)
[  516.878543] Stack: (0xcfb89dd0 to 0xcfb8a000)

(gdb) list *(mxcmci_request+0xd0)
0xcac is at drivers/mmc/host/mxcmmc.c:350.
345
346        if (!mxcmci_use_dma(host))
347            return 0;
348
349        for_each_sg(data->sg, sg, data->sg_len, i) {
350            if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
351                host->do_dma = 0;
352                return 0;
353            }
354        }

Signed-off-by: Chris Ruehl <chris.ruehl@gtsys.com.hk>
---
 drivers/mmc/host/mxcmmc.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
index f7199c8..8645d6a 100644
--- a/drivers/mmc/host/mxcmmc.c
+++ b/drivers/mmc/host/mxcmmc.c
@@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
 		return 0;
 
 	for_each_sg(data->sg, sg, data->sg_len, i) {
-		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
+		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
 			host->do_dma = 0;
 			return 0;
 		}
-- 
1.7.10.4

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Sascha Hauer]

On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
> Function mxcmci_request() throw an exception on a imx27 cpu.
> This patch fix the problem by test the pointer before access it.
> 
> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
> index f7199c8..8645d6a 100644
> --- a/drivers/mmc/host/mxcmmc.c
> +++ b/drivers/mmc/host/mxcmmc.c
> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>  		return 0;
>  
>  	for_each_sg(data->sg, sg, data->sg_len, i) {
> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {

The question is why sg is NULL. I think this shouldn't happen.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

On Wednesday, January 22, 2014 02:42 PM, Sascha Hauer wrote:
> On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
>> Function mxcmci_request() throw an exception on a imx27 cpu.
>> This patch fix the problem by test the pointer before access it.
>>
>> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
>> index f7199c8..8645d6a 100644
>> --- a/drivers/mmc/host/mxcmmc.c
>> +++ b/drivers/mmc/host/mxcmmc.c
>> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>>   		return 0;
>>   
>>   	for_each_sg(data->sg, sg, data->sg_len, i) {
>> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
>> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
> The question is why sg is NULL. I think this shouldn't happen.
>
> Sascha
>
Thats was my idea too. But had no time to debug it out.
As for_each_sg() iterate over the sg-list may its not set properly 
before - In an earlier post
I ask for help but no-one returned so I quick fixed it.  Maybe now the 
bell is loud enough ;-)

Chris

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Sascha Hauer]

On Wed, Jan 22, 2014 at 03:19:48PM +0800, Chris Ruehl wrote:
> On Wednesday, January 22, 2014 02:42 PM, Sascha Hauer wrote:
> >On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
> >>Function mxcmci_request() throw an exception on a imx27 cpu.
> >>This patch fix the problem by test the pointer before access it.
> >>
> >>diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
> >>index f7199c8..8645d6a 100644
> >>--- a/drivers/mmc/host/mxcmmc.c
> >>+++ b/drivers/mmc/host/mxcmmc.c
> >>@@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
> >>  		return 0;
> >>  	for_each_sg(data->sg, sg, data->sg_len, i) {
> >>-		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
> >>+		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
> >The question is why sg is NULL. I think this shouldn't happen.
> >
> >Sascha
> >
> Thats was my idea too. But had no time to debug it out.
> As for_each_sg() iterate over the sg-list may its not set properly
> before - In an earlier post
> I ask for help but no-one returned so I quick fixed it.  Maybe now
> the bell is loud enough ;-)

Please try to track it further down. Does this happen every time you are
in this function or does it happen only every once in a while? Does it
happen in the first iteration of the loop? is it an SDIO card or regular
SD card?

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

On Wednesday, January 22, 2014 03:23 PM, Sascha Hauer wrote:
> On Wed, Jan 22, 2014 at 03:19:48PM +0800, Chris Ruehl wrote:
>> On Wednesday, January 22, 2014 02:42 PM, Sascha Hauer wrote:
>>> On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
>>>> Function mxcmci_request() throw an exception on a imx27 cpu.
>>>> This patch fix the problem by test the pointer before access it.
>>>>
>>>> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
>>>> index f7199c8..8645d6a 100644
>>>> --- a/drivers/mmc/host/mxcmmc.c
>>>> +++ b/drivers/mmc/host/mxcmmc.c
>>>> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>>>>   		return 0;
>>>>   	for_each_sg(data->sg, sg, data->sg_len, i) {
>>>> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
>>>> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
>>> The question is why sg is NULL. I think this shouldn't happen.
>>>
>>> Sascha
>>>
>> Thats was my idea too. But had no time to debug it out.
>> As for_each_sg() iterate over the sg-list may its not set properly
>> before - In an earlier post
>> I ask for help but no-one returned so I quick fixed it.  Maybe now
>> the bell is loud enough ;-)
> Please try to track it further down. Does this happen every time you are
> in this function or does it happen only every once in a while? Does it
> happen in the first iteration of the loop? is it an SDIO card or regular
> SD card?
>
> Sascha
>
Its happened once in a while unexpected. I used a Kingston MicroSD HC 
Class 4
4/8GB and Samsung HC Class 10 16GB.
I was not able to debug it online, my boards are mounted in RFID reader. 
All I had was
the OOPS in the Serial output. After it happen I cannot access the board 
any more - doomed

Chris

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

On Wednesday, January 22, 2014 03:23 PM, Sascha Hauer wrote:
> On Wed, Jan 22, 2014 at 03:19:48PM +0800, Chris Ruehl wrote:
>> On Wednesday, January 22, 2014 02:42 PM, Sascha Hauer wrote:
>>> On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
>>>> Function mxcmci_request() throw an exception on a imx27 cpu.
>>>> This patch fix the problem by test the pointer before access it.
>>>>
>>>> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
>>>> index f7199c8..8645d6a 100644
>>>> --- a/drivers/mmc/host/mxcmmc.c
>>>> +++ b/drivers/mmc/host/mxcmmc.c
>>>> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>>>>   		return 0;
>>>>   	for_each_sg(data->sg, sg, data->sg_len, i) {
>>>> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
>>>> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
>>> The question is why sg is NULL. I think this shouldn't happen.
>>>
>>> Sascha
>>>
>> Thats was my idea too. But had no time to debug it out.
>> As for_each_sg() iterate over the sg-list may its not set properly
>> before - In an earlier post
>> I ask for help but no-one returned so I quick fixed it.  Maybe now
>> the bell is loud enough ;-)
>
> Please try to track it further down. Does this happen every time you are
> in this function or does it happen only every once in a while? Does it
> happen in the first iteration of the loop? is it an SDIO card or regular
> SD card?
>
> Sascha
>

I add this temporarily to see where the loop counter is
   else {
       if (!sg) {
             dev_err(mmc_dev(host->mmc),
                  "%s: bad sg, loop:%d len:%d \n",
                   i, data->sg_len
                   __func__,
               );
       }
   }

we will see.

Chris

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Russell King - ARM Linux]

On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
> index f7199c8..8645d6a 100644
> --- a/drivers/mmc/host/mxcmmc.c
> +++ b/drivers/mmc/host/mxcmmc.c
> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>  		return 0;
>  
>  	for_each_sg(data->sg, sg, data->sg_len, i) {
> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {

sg should never be NULL here - so this is probably papering over a bug.

-- 
FTTC broadband for 0.8mile line: 5.8Mbps down 500kbps up.  Estimation
in database were 13.1 to 19Mbit for a good line, about 7.5+ for a bad.
Estimate before purchase was "up to 13.2Mbit".

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

On Wednesday, January 22, 2014 06:11 PM, Russell King - ARM Linux wrote:
> On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
>> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
>> index f7199c8..8645d6a 100644
>> --- a/drivers/mmc/host/mxcmmc.c
>> +++ b/drivers/mmc/host/mxcmmc.c
>> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>>   		return 0;
>>   
>>   	for_each_sg(data->sg, sg, data->sg_len, i) {
>> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
>> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
> sg should never be NULL here - so this is probably papering over a bug.
>
I agree, and this patch should not apply to the kernel,, . Its my fix 
for my production mainboard
only.  I report the Oops but not get any response back. I'm sorry to 
wast your time.

Chris

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

On Wednesday, January 22, 2014 03:23 PM, Sascha Hauer wrote:
> On Wed, Jan 22, 2014 at 03:19:48PM +0800, Chris Ruehl wrote:
>> On Wednesday, January 22, 2014 02:42 PM, Sascha Hauer wrote:
>>> On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
>>>> Function mxcmci_request() throw an exception on a imx27 cpu.
>>>> This patch fix the problem by test the pointer before access it.
>>>>
>>>> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
>>>> index f7199c8..8645d6a 100644
>>>> --- a/drivers/mmc/host/mxcmmc.c
>>>> +++ b/drivers/mmc/host/mxcmmc.c
>>>> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>>>>   		return 0;
>>>>   	for_each_sg(data->sg, sg, data->sg_len, i) {
>>>> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
>>>> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
>>> The question is why sg is NULL. I think this shouldn't happen.
>>>
>>> Sascha
>>>
>> Thats was my idea too. But had no time to debug it out.
>> As for_each_sg() iterate over the sg-list may its not set properly
>> before - In an earlier post
>> I ask for help but no-one returned so I quick fixed it.  Maybe now
>> the bell is loud enough ;-)
> Please try to track it further down. Does this happen every time you are
> in this function or does it happen only every once in a while? Does it
> happen in the first iteration of the loop? is it an SDIO card or regular
> SD card?
>
> Sascha
>
Sascha,

there was a fix for a race condition:

commit 70aa6109597ea6955a93f16430b588b5ee5ba547
mmc: mxcmmc: fix race conditions for host->req and host->data access

do you think its possible that the current oops is kind of race?


Regards
Chris

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Sascha Hauer]

On Thu, Jan 23, 2014 at 05:01:49PM +0800, Chris Ruehl wrote:
> On Wednesday, January 22, 2014 03:23 PM, Sascha Hauer wrote:
> >On Wed, Jan 22, 2014 at 03:19:48PM +0800, Chris Ruehl wrote:
> >>On Wednesday, January 22, 2014 02:42 PM, Sascha Hauer wrote:
> >>>On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
> >>>>Function mxcmci_request() throw an exception on a imx27 cpu.
> >>>>This patch fix the problem by test the pointer before access it.
> >>>>
> >>>>diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
> >>>>index f7199c8..8645d6a 100644
> >>>>--- a/drivers/mmc/host/mxcmmc.c
> >>>>+++ b/drivers/mmc/host/mxcmmc.c
> >>>>@@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
> >>>>  		return 0;
> >>>>  	for_each_sg(data->sg, sg, data->sg_len, i) {
> >>>>-		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
> >>>>+		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
> >>>The question is why sg is NULL. I think this shouldn't happen.
> >>>
> >>>Sascha
> >>>
> >>Thats was my idea too. But had no time to debug it out.
> >>As for_each_sg() iterate over the sg-list may its not set properly
> >>before - In an earlier post
> >>I ask for help but no-one returned so I quick fixed it.  Maybe now
> >>the bell is loud enough ;-)
> >Please try to track it further down. Does this happen every time you are
> >in this function or does it happen only every once in a while? Does it
> >happen in the first iteration of the loop? is it an SDIO card or regular
> >SD card?
> >
> >Sascha
> >
> Sascha,
> 
> there was a fix for a race condition:
> 
> commit 70aa6109597ea6955a93f16430b588b5ee5ba547
> mmc: mxcmmc: fix race conditions for host->req and host->data access
> 
> do you think its possible that the current oops is kind of race?

Could be, yes. I remember problems that the dma completion interrupt and
the data transfer interrupt came in the wrong (or better: unexpected)
order. Maybe this is not completely solved.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH] mxcmmc: Internal error: Oops: 17 [#1] ARM from sg->offset

[Chris Ruehl]

On Wednesday, January 22, 2014 06:11 PM, Russell King - ARM Linux wrote:
> On Wed, Jan 22, 2014 at 12:32:39PM +0800, Chris Ruehl wrote:
>> diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
>> index f7199c8..8645d6a 100644
>> --- a/drivers/mmc/host/mxcmmc.c
>> +++ b/drivers/mmc/host/mxcmmc.c
>> @@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host *host, struct mmc_data *data)
>>   		return 0;
>>   
>>   	for_each_sg(data->sg, sg, data->sg_len, i) {
>> -		if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
>> +		if (sg && (sg->offset & 3 || sg->length & 3 || sg->length < 512)) {
> sg should never be NULL here - so this is probably papering over a bug.
>

I'd had some time and look into the meaning of the sg->xx & 0x3 and 
understand this
check validate the alignment of the data. If failed the dma handling is 
canceled and a fall-back to pio is done.

In a earlier patch for the unexpected dma & interrupts are synchronized 
using the spinlock. I pickup this
idea and protect the setup-data using a look. Until now the oops are gone.

PLEASE comment!

diff --git a/drivers/mmc/host/mxcmmc.c b/drivers/mmc/host/mxcmmc.c
index 8645d6a..b11d3c4 100644
--- a/drivers/mmc/host/mxcmmc.c
+++ b/drivers/mmc/host/mxcmmc.c
@@ -347,7 +347,7 @@ static int mxcmci_setup_data(struct mxcmci_host 
*host, struct mmc_data *data)
                 return 0;

         for_each_sg(data->sg, sg, data->sg_len, i) {
-               if (sg && (sg->offset & 3 || sg->length & 3 || 
sg->length < 512)) {
+               if (sg->offset & 3 || sg->length & 3 || sg->length < 512) {
                         host->do_dma = 0;
                         return 0;
                 }
@@ -800,9 +800,12 @@ static void mxcmci_request(struct mmc_host *mmc, 
struct mmc_request *req)
         struct mxcmci_host *host = mmc_priv(mmc);
         unsigned int cmdat = host->cmdat;
         int error;
+       unsigned long flags;

         WARN_ON(host->req != NULL);

+       spin_lock_irqsave(&host->lock, flags);
+
         host->req = req;
         host->cmdat &= ~CMD_DAT_CONT_INIT;

@@ -813,6 +816,7 @@ static void mxcmci_request(struct mmc_host *mmc, 
struct mmc_request *req)
                 error = mxcmci_setup_data(host, req->data);
                 if (error) {
                         req->cmd->error = error;
+                       spin_unlock_irqrestore(&host->lock, flags);
                         goto out;
                 }

@@ -823,6 +827,8 @@ static void mxcmci_request(struct mmc_host *mmc, 
struct mmc_request *req)
                         cmdat |= CMD_DAT_CONT_WRITE;
         }

+       spin_unlock_irqrestore(&host->lock, flags);



With kind regards
Chris