Re: imprecise external abort using the flexcan driver on i.MX6Q

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 8767 bytes --]

On 09/26/2013 04:01 PM, =?utf-8?Q?Lothar_Wa=C3=9Fmann?= wrote:
> Hi,
> 
> when enabling the can interface with 'ifconfig can0 up' (after
> configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
> getting the following kernel dump:
> 
> |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
> |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
> |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc

Looks like a NULL pointer deref to me. But it doesn't make any sense,
because the offset is way beyond the length of the struct flexcan_regs.

> |Internal error: : 1c06 [#1] SMP ARM
> |Modules linked in: flexcan can_dev
> |CPU: 2 PID: 1215 Comm: ifconfig Not tainted 3.12.0-rc1-next-20130919-karo+ #91
> |task: beac3000 ti: be1fc000 task.ti: be1fc000
> |PC is at flexcan_chip_start+0x200/0x344 [flexcan]
> |LR is at flexcan_chip_start+0x1d4/0x344 [flexcan]
> |pc : [<7f007560>]    lr : [<7f007534>]    psr: 80000013
> |sp : be1fde40  ip : c0a1808c  fp : 00000001
> |r10: 00000000  r9 : 00008914  r8 : 7e894c38
> |r7 : 04000000  r6 : c0a18088  r5 : be168000  r4 : c0a18000
> |r3 : 00000001  r2 : c0a18090  r1 : 00000000  r0 : 00000000
> |Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
> |Control: 10c5387d  Table: 4e28804a  DAC: 00000015
> |Process ifconfig (pid: 1215, stack limit = 0xbe1fc240)
> |Stack: (0xbe1fde40 to 0xbe1fe000)
> |de40: 0a21ac53 0a212003 be8f0d00 be168000 00000000 be821580 be16802c 7f008170
> |de60: be168000 be168000 be168000 000000c1 7f008678 80357868 803577ec be168000
> |de80: 000000c1 00000001 00040080 80357aa0 beac3000 be168000 00040080 be9fd800
> |dea0: be168000 80357ba0 00000000 be9fd80c be9fd800 803a1b68 00000000 01894c38
> |dec0: 306e6163 00000000 00000000 00000000 000000c1 76f4f4d0 7e894f01 000566ce
> |dee0: 00000080 00008914 be1a8e40 7e894c38 00008914 00000003 be1fc000 7e894c38
> |df00: be500020 80343350 7e894c38 be1a8e40 00000003 800bb464 7e894c38 800bbfa8
> |df20: bea8df80 00000003 8041e680 800add4c 00000020 00000003 805ccff8 be1fdf60
> |df40: 00000003 800ade08 805c6bf8 805ccff8 be500000 00000000 00000002 80342b0c
> |df60: be880e50 7e894c38 be1a8e40 00000000 00008914 00000003 be1fc000 00000000
> |df80: 00000000 800bc030 00000003 00000000 0005868f 00000004 00054f14 00000036
> |dfa0: 8000e7e4 8000e660 0005868f 00000004 00000003 00008914 7e894c38 0005868f
> |dfc0: 0005868f 00000004 00054f14 00000036 00000000 00000000 7e894f0f 00000000
> |dfe0: 00000000 7e894c20 0000cad4 76e1a87c 20000010 00000003 4eff1811 4eff1c11
> |[<7f007560>] (flexcan_chip_start+0x200/0x344 [flexcan]) from [<7f008170>] (flexcan_open+0x74/0x118 [flexcan])
> |[<7f008170>] (flexcan_open+0x74/0x118 [flexcan]) from [<80357868>] (__dev_open+0x7c/0xfc)
> |[<80357868>] (__dev_open+0x7c/0xfc) from [<80357aa0>] (__dev_change_flags+0x8c/0x118)
> |[<80357aa0>] (__dev_change_flags+0x8c/0x118) from [<80357ba0>] (dev_change_flags+0x10/0x44)
> |[<80357ba0>] (dev_change_flags+0x10/0x44) from [<803a1b68>] (devinet_ioctl+0x2a4/0x62c)
> |[<803a1b68>] (devinet_ioctl+0x2a4/0x62c) from [<80343350>] (sock_ioctl+0x220/0x274)
> |[<80343350>] (sock_ioctl+0x220/0x274) from [<800bb464>] (vfs_ioctl+0x28/0x3c)
> |[<800bb464>] (vfs_ioctl+0x28/0x3c) from [<800bbfa8>] (do_vfs_ioctl+0x53c/0x590)
> |[<800bbfa8>] (do_vfs_ioctl+0x53c/0x590) from [<800bc030>] (SyS_ioctl+0x34/0x58)
> |[<800bc030>] (SyS_ioctl+0x34/0x58) from [<8000e660>] (ret_fast_syscall+0x0/0x30)
> |Code: f57ff04e e3a01000 e5820000 f57ff04e (e5820004) 
> |---[ end trace 49ef25cc4eb56f2d ]---
> |Kernel panic - not syncing: Fatal exception
> |CPU1: stopping
> |CPU: 1 PID: 0 Comm: swapper/1 Tainted: G      D      3.12.0-rc1-next-20130919-karo+ #91
> |[<8001416c>] (unwind_backtrace+0x0/0x11c) from [<800112f0>] (show_stack+0x10/0x14)
> |[<800112f0>] (show_stack+0x10/0x14) from [<803de8a4>] (dump_stack+0x68/0x84)
> |[<803de8a4>] (dump_stack+0x68/0x84) from [<80012fe8>] (handle_IPI+0xc0/0x124)
> |[<80012fe8>] (handle_IPI+0xc0/0x124) from [<80008530>] (gic_handle_irq+0x58/0x60)
> |[<80008530>] (gic_handle_irq+0x58/0x60) from [<80011d80>] (__irq_svc+0x40/0x50)
> |Exception stack(0xbe8b7f58 to 0xbe8b7fa0)
> |7f40:                                                       be8b7fa0 00000024
> |7f60: 7aa8d094 00000024 7a434975 00000024 80e73d50 00000001 805cff2c 412fc09a
> |7f80: 805cff88 00000000 00000005 be8b7fa0 80057dc0 80328354 60000013 ffffffff
> |[<80011d80>] (__irq_svc+0x40/0x50) from [<80328354>] (cpuidle_enter_state+0x54/0xf0)
> |[<80328354>] (cpuidle_enter_state+0x54/0xf0) from [<803284cc>] (cpuidle_idle_call+0xdc/0x144)
> |[<803284cc>] (cpuidle_idle_call+0xdc/0x144) from [<8000f1f4>] (arch_cpu_idle+0x8/0x38)
> |[<8000f1f4>] (arch_cpu_idle+0x8/0x38) from [<80050dbc>] (cpu_startup_entry+0xb0/0x114)
> |[<80050dbc>] (cpu_startup_entry+0xb0/0x114) from [<100085c4>] (0x100085c4)
> |CPU0: stopping
> |CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D      3.12.0-rc1-next-20130919-karo+ #91
> |[<8001416c>] (unwind_backtrace+0x0/0x11c) from [<800112f0>] (show_stack+0x10/0x14)
> |[<800112f0>] (show_stack+0x10/0x14) from [<803de8a4>] (dump_stack+0x68/0x84)
> |[<803de8a4>] (dump_stack+0x68/0x84) from [<80012fe8>] (handle_IPI+0xc0/0x124)
> |[<80012fe8>] (handle_IPI+0xc0/0x124) from [<80008530>] (gic_handle_irq+0x58/0x60)
> |[<80008530>] (gic_handle_irq+0x58/0x60) from [<80011d80>] (__irq_svc+0x40/0x50)
> |Exception stack(0x805c5f28 to 0x805c5f70)
> |5f20:                   805c5f70 00000024 7aa8d168 00000024 7a43d873 00000024
> |5f40: 80e6bd50 00000001 805cff2c 412fc09a 805cff88 00000000 00000005 805c5f70
> |5f60: 80057dc0 80328354 60000013 ffffffff
> |[<80011d80>] (__irq_svc+0x40/0x50) from [<80328354>] (cpuidle_enter_state+0x54/0xf0)
> |[<80328354>] (cpuidle_enter_state+0x54/0xf0) from [<803284cc>] (cpuidle_idle_call+0xdc/0x144)
> |[<803284cc>] (cpuidle_idle_call+0xdc/0x144) from [<8000f1f4>] (arch_cpu_idle+0x8/0x38)
> |[<8000f1f4>] (arch_cpu_idle+0x8/0x38) from [<80050dbc>] (cpu_startup_entry+0xb0/0x114)
> |[<80050dbc>] (cpu_startup_entry+0xb0/0x114) from [<805909d8>] (start_kernel+0x268/0x2ac)
> |CPU3: stopping
> |CPU: 3 PID: 0 Comm: swapper/3 Tainted: G      D      3.12.0-rc1-next-20130919-karo+ #91
> |[<8001416c>] (unwind_backtrace+0x0/0x11c) from [<800112f0>] (show_stack+0x10/0x14)
> |[<800112f0>] (show_stack+0x10/0x14) from [<803de8a4>] (dump_stack+0x68/0x84)
> |[<803de8a4>] (dump_stack+0x68/0x84) from [<80012fe8>] (handle_IPI+0xc0/0x124)
> |[<80012fe8>] (handle_IPI+0xc0/0x124) from [<80008530>] (gic_handle_irq+0x58/0x60)
> |[<80008530>] (gic_handle_irq+0x58/0x60) from [<80011d80>] (__irq_svc+0x40/0x50)
> |Exception stack(0xbe8bbf58 to 0xbe8bbfa0)
> |bf40:                                                       be8bbfa0 00000024
> |bf60: 7aa8d039 00000024 787aa018 00000024 80e83d50 00000000 805cff2c 412fc09a
> |bf80: 805cff3c 00000000 00000005 be8bbfa0 80057dc0 80328354 60000013 ffffffff
> |[<80011d80>] (__irq_svc+0x40/0x50) from [<80328354>] (cpuidle_enter_state+0x54/0xf0)
> |[<80328354>] (cpuidle_enter_state+0x54/0xf0) from [<803284cc>] (cpuidle_idle_call+0xdc/0x144)
> |[<803284cc>] (cpuidle_idle_call+0xdc/0x144) from [<8000f1f4>] (arch_cpu_idle+0x8/0x38)
> |[<8000f1f4>] (arch_cpu_idle+0x8/0x38) from [<80050dbc>] (cpu_startup_entry+0xb0/0x114)
> |[<80050dbc>] (cpu_startup_entry+0xb0/0x114) from [<100085c4>] (0x100085c4)
> 
> The same kernel/driver works perfectly well on an i.MX53 based board.

Just to be sure, can you boot with one CPU only.

> The data abort happens upon writing to can_ctrl in the second run of
> this loop in flexcan_chip_start():
> |	for (i = 0; i < ARRAY_SIZE(regs->cantxfg); i++) {
> |		flexcan_write(0, &regs->cantxfg[i].can_ctrl);
> ----------------^ crashes here with i = 1

Can you instrument flexcan_write().

> |
> |		flexcan_write(0, &regs->cantxfg[i].can_id);
> |		flexcan_write(0, &regs->cantxfg[i].data[0]);
> |		flexcan_write(0, &regs->cantxfg[i].data[1]);
> |
> |		/* put MB into rx queue */
> |		flexcan_write(FLEXCAN_MB_CNT_CODE(0x4),
> |			&regs->cantxfg[i].can_ctrl);
> |	}
> 
> Does anyone have any clue how this can happen?
> 
> Can anyone reproduce this on another machine?
> 
> The same hardware works well with a 3.0.35 Freescale kernel.

Marc
-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Russell King - ARM Linux]

On Thu, Sep 26, 2013 at 05:42:53PM +0200, Marc Kleine-Budde wrote:
> On 09/26/2013 04:01 PM, =?utf-8?Q?Lothar_Wa=C3=9Fmann?= wrote:
> > Hi,
> > 
> > when enabling the can interface with 'ifconfig can0 up' (after
> > configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
> > getting the following kernel dump:
> > 
> > |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
> > |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
> > |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
> > |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
> > |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc
> 
> Looks like a NULL pointer deref to me. But it doesn't make any sense,
> because the offset is way beyond the length of the struct flexcan_regs.

NULL pointer derefs don't produce imprecise external aborts.  I believe
that the FAR is unpredictable for such aborts as well, which means the
"at xxxx" should be ignored.

Bearing in mind that it is imprecise, that means that the abort happened
sometime in the past, so the PC isn't reliable either.

The code here is:

   0:	f57ff04e 	dsb	st
   4:	e3a01000 	mov	r1, #0
   8:	e5820000 	str	r0, [r2]
   c:	f57ff04e 	dsb	st
  10:	e5820004 	str	r0, [r2, #4]

Given the dsbs there, it's probably the str r0, [r2] which provoked it,
and we can see from the register dump, r2 is not NULL.

It's probably complaining that a clock necessary for the peripherals
registers to be accessible is not running.

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Santosh Shilimkar]

On Thursday 26 September 2013 11:54 AM, Russell King - ARM Linux wrote:
> On Thu, Sep 26, 2013 at 05:42:53PM +0200, Marc Kleine-Budde wrote:
>> On 09/26/2013 04:01 PM, =?utf-8?Q?Lothar_Wa=C3=9Fmann?= wrote:
>>> Hi,
>>>
>>> when enabling the can interface with 'ifconfig can0 up' (after
>>> configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
>>> getting the following kernel dump:
>>>
>>> |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
>>> |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
>>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
>>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
>>> |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc
>>
>> Looks like a NULL pointer deref to me. But it doesn't make any sense,
>> because the offset is way beyond the length of the struct flexcan_regs.
> 
> NULL pointer derefs don't produce imprecise external aborts.  I believe
> that the FAR is unpredictable for such aborts as well, which means the
> "at xxxx" should be ignored.
> 
yep .. FAR is never reliable for imprecise external aborts.

> Bearing in mind that it is imprecise, that means that the abort happened
> sometime in the past, so the PC isn't reliable either.
> 
> The code here is:
> 
>    0:	f57ff04e 	dsb	st
>    4:	e3a01000 	mov	r1, #0
>    8:	e5820000 	str	r0, [r2]
>    c:	f57ff04e 	dsb	st
>   10:	e5820004 	str	r0, [r2, #4]
> 
> Given the dsbs there, it's probably the str r0, [r2] which provoked it,
> and we can see from the register dump, r2 is not NULL.
> 
> It's probably complaining that a clock necessary for the peripherals
> registers to be accessible is not running.
> 
Not sure if it helps but almost 90% of the imprecise external aborts
cases I have seen on OMAP attributed to clock not being available
at the IP register accesses race conditions either at SW or
hardware(clock settling time). And rest of the 10 % majority falling
into interconnect synchronization issues.

Regards,
Santosh

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Hi,

Santosh Shilimkar writes:
> On Thursday 26 September 2013 11:54 AM, Russell King - ARM Linux wrote:
> > On Thu, Sep 26, 2013 at 05:42:53PM +0200, Marc Kleine-Budde wrote:
> >> On 09/26/2013 04:01 PM, Lothar Waßmann wrote:
> >>> Hi,
> >>>
> >>> when enabling the can interface with 'ifconfig can0 up' (after
> >>> configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
> >>> getting the following kernel dump:
> >>>
> >>> |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
> >>> |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
> >>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
> >>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
> >>> |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc
> >>
> >> Looks like a NULL pointer deref to me. But it doesn't make any sense,
> >> because the offset is way beyond the length of the struct flexcan_regs.
> > 
> > NULL pointer derefs don't produce imprecise external aborts.  I believe
> > that the FAR is unpredictable for such aborts as well, which means the
> > "at xxxx" should be ignored.
> > 
> yep .. FAR is never reliable for imprecise external aborts.
> 
> > Bearing in mind that it is imprecise, that means that the abort happened
> > sometime in the past, so the PC isn't reliable either.
> > 
> > The code here is:
> > 
> >    0:	f57ff04e 	dsb	st
> >    4:	e3a01000 	mov	r1, #0
> >    8:	e5820000 	str	r0, [r2]
> >    c:	f57ff04e 	dsb	st
> >   10:	e5820004 	str	r0, [r2, #4]
> > 
> > Given the dsbs there, it's probably the str r0, [r2] which provoked it,
> > and we can see from the register dump, r2 is not NULL.
> > 
> > It's probably complaining that a clock necessary for the peripherals
> > registers to be accessible is not running.
> > 
> Not sure if it helps but almost 90% of the imprecise external aborts
> cases I have seen on OMAP attributed to clock not being available
> at the IP register accesses race conditions either at SW or
> hardware(clock settling time). And rest of the 10 % majority falling
> into interconnect synchronization issues.
> 
I already checked this by enabling all clocks with devmem.
If a clock would be missing I would expect all register wriites to
fail not the register write to the _second_ mailbox (after the first
mailbox has been successfully initialized).


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 2985 bytes --]

On 09/27/2013 11:23 AM, Lothar Waßmann wrote:
> Hi,
> 
> Santosh Shilimkar writes:
>> On Thursday 26 September 2013 11:54 AM, Russell King - ARM Linux wrote:
>>> On Thu, Sep 26, 2013 at 05:42:53PM +0200, Marc Kleine-Budde wrote:
>>>> On 09/26/2013 04:01 PM, Lothar Waßmann wrote:
>>>>> Hi,
>>>>>
>>>>> when enabling the can interface with 'ifconfig can0 up' (after
>>>>> configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
>>>>> getting the following kernel dump:
>>>>>
>>>>> |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
>>>>> |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
>>>>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
>>>>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
>>>>> |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc
>>>>
>>>> Looks like a NULL pointer deref to me. But it doesn't make any sense,
>>>> because the offset is way beyond the length of the struct flexcan_regs.
>>>
>>> NULL pointer derefs don't produce imprecise external aborts.  I believe
>>> that the FAR is unpredictable for such aborts as well, which means the
>>> "at xxxx" should be ignored.
>>>
>> yep .. FAR is never reliable for imprecise external aborts.
>>
>>> Bearing in mind that it is imprecise, that means that the abort happened
>>> sometime in the past, so the PC isn't reliable either.
>>>
>>> The code here is:
>>>
>>>    0:	f57ff04e 	dsb	st
>>>    4:	e3a01000 	mov	r1, #0
>>>    8:	e5820000 	str	r0, [r2]
>>>    c:	f57ff04e 	dsb	st
>>>   10:	e5820004 	str	r0, [r2, #4]
>>>
>>> Given the dsbs there, it's probably the str r0, [r2] which provoked it,
>>> and we can see from the register dump, r2 is not NULL.
>>>
>>> It's probably complaining that a clock necessary for the peripherals
>>> registers to be accessible is not running.
>>>
>> Not sure if it helps but almost 90% of the imprecise external aborts
>> cases I have seen on OMAP attributed to clock not being available
>> at the IP register accesses race conditions either at SW or
>> hardware(clock settling time). And rest of the 10 % majority falling
>> into interconnect synchronization issues.
>>
> I already checked this by enabling all clocks with devmem.
> If a clock would be missing I would expect all register wriites to
> fail not the register write to the _second_ mailbox (after the first
> mailbox has been successfully initialized).

They layout of the flexcan varies in the different imx chips. I'm
comparing the datasheets at the moment. As Russell pointed out the area
staring at offset 0x80 is by the reception FIFO engine.

Marc
-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Hi,

Marc Kleine-Budde writes:
> On 09/27/2013 11:23 AM, Lothar Waßmann wrote:
> > Hi,
> > 
> > Santosh Shilimkar writes:
> >> On Thursday 26 September 2013 11:54 AM, Russell King - ARM Linux wrote:
> >>> On Thu, Sep 26, 2013 at 05:42:53PM +0200, Marc Kleine-Budde wrote:
> >>>> On 09/26/2013 04:01 PM, Lothar Waßmann wrote:
> >>>>> Hi,
> >>>>>
> >>>>> when enabling the can interface with 'ifconfig can0 up' (after
> >>>>> configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
> >>>>> getting the following kernel dump:
> >>>>>
> >>>>> |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
> >>>>> |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
> >>>>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
> >>>>> |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
> >>>>> |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc
> >>>>
> >>>> Looks like a NULL pointer deref to me. But it doesn't make any sense,
> >>>> because the offset is way beyond the length of the struct flexcan_regs.
> >>>
> >>> NULL pointer derefs don't produce imprecise external aborts.  I believe
> >>> that the FAR is unpredictable for such aborts as well, which means the
> >>> "at xxxx" should be ignored.
> >>>
> >> yep .. FAR is never reliable for imprecise external aborts.
> >>
> >>> Bearing in mind that it is imprecise, that means that the abort happened
> >>> sometime in the past, so the PC isn't reliable either.
> >>>
> >>> The code here is:
> >>>
> >>>    0:	f57ff04e 	dsb	st
> >>>    4:	e3a01000 	mov	r1, #0
> >>>    8:	e5820000 	str	r0, [r2]
> >>>    c:	f57ff04e 	dsb	st
> >>>   10:	e5820004 	str	r0, [r2, #4]
> >>>
> >>> Given the dsbs there, it's probably the str r0, [r2] which provoked it,
> >>> and we can see from the register dump, r2 is not NULL.
> >>>
> >>> It's probably complaining that a clock necessary for the peripherals
> >>> registers to be accessible is not running.
> >>>
> >> Not sure if it helps but almost 90% of the imprecise external aborts
> >> cases I have seen on OMAP attributed to clock not being available
> >> at the IP register accesses race conditions either at SW or
> >> hardware(clock settling time). And rest of the 10 % majority falling
> >> into interconnect synchronization issues.
> >>
> > I already checked this by enabling all clocks with devmem.
> > If a clock would be missing I would expect all register wriites to
> > fail not the register write to the _second_ mailbox (after the first
> > mailbox has been successfully initialized).
> 
> They layout of the flexcan varies in the different imx chips. I'm
> comparing the datasheets at the moment. As Russell pointed out the area
> staring at offset 0x80 is by the reception FIFO engine.
> 
The layout is correct when the Rx FIFO is disabled!

removing 'FLEXCAN_MCR_FEN' from the MCR setting makes the driver work
as expected (just without receive fifo).

For the use case with the FIFO enabled, the layout has to be changed
obviously.


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 1935 bytes --]

On 09/27/2013 12:04 PM, Lothar Waßmann wrote:

>> They layout of the flexcan varies in the different imx chips. I'm
>> comparing the datasheets at the moment. As Russell pointed out the area
>> staring at offset 0x80 is by the reception FIFO engine.
>>
> The layout is correct when the Rx FIFO is disabled!
> 
> removing 'FLEXCAN_MCR_FEN' from the MCR setting makes the driver work
> as expected (just without receive fifo).
> 
> For the use case with the FIFO enabled, the layout has to be changed
> obviously.

Can you just remove the loop completely? I think it's not needed. It
was in the original driver, when I picked it up.

diff --git a/drivers/net/can/flexcan.c b/drivers/net/can/flexcan.c
index e1ac75d..57ee3df 100644
--- a/drivers/net/can/flexcan.c
+++ b/drivers/net/can/flexcan.c
@@ -778,17 +778,6 @@ static int flexcan_chip_start(struct net_device *dev)
        netdev_dbg(dev, "%s: writing ctrl=0x%08x", __func__, reg_ctrl);
        flexcan_write(reg_ctrl, &regs->ctrl);

-       for (i = 0; i < ARRAY_SIZE(regs->cantxfg); i++) {
-               flexcan_write(0, &regs->cantxfg[i].can_ctrl);
-               flexcan_write(0, &regs->cantxfg[i].can_id);
-               flexcan_write(0, &regs->cantxfg[i].data[0]);
-               flexcan_write(0, &regs->cantxfg[i].data[1]);
-
-               /* put MB into rx queue */
-               flexcan_write(FLEXCAN_MB_CNT_CODE(0x4),
-                       &regs->cantxfg[i].can_ctrl);
-       }
-
        /* acceptance mask/acceptance code (accept everything) */
        flexcan_write(0x0, &regs->rxgmask);
        flexcan_write(0x0, &regs->rx14mask);

Marc

-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Hi,

Marc Kleine-Budde writes:
> On 09/27/2013 12:04 PM, Lothar Waßmann wrote:
> 
> >> They layout of the flexcan varies in the different imx chips. I'm
> >> comparing the datasheets at the moment. As Russell pointed out the area
> >> staring at offset 0x80 is by the reception FIFO engine.
> >>
> > The layout is correct when the Rx FIFO is disabled!
> > 
> > removing 'FLEXCAN_MCR_FEN' from the MCR setting makes the driver work
> > as expected (just without receive fifo).
> > 
> > For the use case with the FIFO enabled, the layout has to be changed
> > obviously.
> 
> Can you just remove the loop completely? I think it's not needed. It
> was in the original driver, when I picked it up.
> 
> diff --git a/drivers/net/can/flexcan.c b/drivers/net/can/flexcan.c
> index e1ac75d..57ee3df 100644
> --- a/drivers/net/can/flexcan.c
> +++ b/drivers/net/can/flexcan.c
> @@ -778,17 +778,6 @@ static int flexcan_chip_start(struct net_device *dev)
>         netdev_dbg(dev, "%s: writing ctrl=0x%08x", __func__, reg_ctrl);
>         flexcan_write(reg_ctrl, &regs->ctrl);
> 
> -       for (i = 0; i < ARRAY_SIZE(regs->cantxfg); i++) {
> -               flexcan_write(0, &regs->cantxfg[i].can_ctrl);
> -               flexcan_write(0, &regs->cantxfg[i].can_id);
> -               flexcan_write(0, &regs->cantxfg[i].data[0]);
> -               flexcan_write(0, &regs->cantxfg[i].data[1]);
> -
> -               /* put MB into rx queue */
> -               flexcan_write(FLEXCAN_MB_CNT_CODE(0x4),
> -                       &regs->cantxfg[i].can_ctrl);
> -       }
> -
>         /* acceptance mask/acceptance code (accept everything) */
>         flexcan_write(0x0, &regs->rxgmask);
>         flexcan_write(0x0, &regs->rx14mask);
> 
OK, done that. Driver works perfectly well now on i.MX6Q. I'll retest
on i.MX53 later to make sure it's still working there too.


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Matt Sealey]

Hi Russel,

On Thu, Sep 26, 2013 at 10:54 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
>
> The code here is:
>
>    0:   f57ff04e        dsb     st
>    4:   e3a01000        mov     r1, #0
>    8:   e5820000        str     r0, [r2]
>    c:   f57ff04e        dsb     st
>   10:   e5820004        str     r0, [r2, #4]
>
> Given the dsbs there, it's probably the str r0, [r2] which provoked it,
> and we can see from the register dump, r2 is not NULL.

For reference, that was:

|r7 : 04000000  r6 : c0a18088  r5 : be168000  r4 : c0a18000
|r3 : 00000001  r2 : c0a18090  r1 : 00000000  r0 : 00000000

If r2 is indeed the address of the access, then this is entirely down
to a misuse of the hardware; from the manual:

~
When the FEN bit is set in the MCR, the memory area from 0x80 to 0xFF (which is
normally occupied by MB0 to MB7) is used by the reception FIFO engine.
Table 34-6
shows the Rx FIFO data structure. The region 0x80-0x8F contains an
message buffer
structure which is the port through which the ARM core reads data from
the FIFO (the
oldest frame received and not read yet). The region 0x90-0xDF is
reserved for internal
use of the FIFO engine.
^^^^^^^^^^^^^^^^^^^^
~

In this case, yes, we're trying to write to 0x90. And in
flexcan_chip_start, above the location where it dumps the array to
hardware, FEN is set; this is reflected by the debug (mcr=0x79a2020f).

I'm surprised this ever worked.. if the area 0x90-0xDF is being used
by the FIFO internally then the difference between i.MX53 ('working')
and i.MX6Q ('imprecise abort') is that the i.MX6Q is causing the
proper bus response. We can only hope that the i.MX53 is ONLY not
causing the proper bus response and not actually letting you write
into a FIFO internal data area..

--
Matt Sealey <neko@bakuhatsu.net>

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Hi,

Matt Sealey writes:
> Hi Russel,
> 
> On Thu, Sep 26, 2013 at 10:54 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> >
> > The code here is:
> >
> >    0:   f57ff04e        dsb     st
> >    4:   e3a01000        mov     r1, #0
> >    8:   e5820000        str     r0, [r2]
> >    c:   f57ff04e        dsb     st
> >   10:   e5820004        str     r0, [r2, #4]
> >
> > Given the dsbs there, it's probably the str r0, [r2] which provoked it,
> > and we can see from the register dump, r2 is not NULL.
> 
> For reference, that was:
> 
> |r7 : 04000000  r6 : c0a18088  r5 : be168000  r4 : c0a18000
> |r3 : 00000001  r2 : c0a18090  r1 : 00000000  r0 : 00000000
> 
> If r2 is indeed the address of the access, then this is entirely down
> to a misuse of the hardware; from the manual:
> 
> ~
> When the FEN bit is set in the MCR, the memory area from 0x80 to 0xFF (which is
> normally occupied by MB0 to MB7) is used by the reception FIFO engine.
> Table 34-6
> shows the Rx FIFO data structure. The region 0x80-0x8F contains an
> message buffer
>
What manual are you referring to? In my "i.MX 6Dual/6Quad Applications
Processor Reference Manual, Rev. 1, 04/2013" flexcan is chapter 26 and
Table 34-6 is: Single-Ended Source Termination Resistance Settings
of the HDMI interface.

> structure which is the port through which the ARM core reads data from
> the FIFO (the
> oldest frame received and not read yet). The region 0x90-0xDF is
> reserved for internal
> use of the FIFO engine.
> ^^^^^^^^^^^^^^^^^^^^
> ~
> 
> In this case, yes, we're trying to write to 0x90. And in
> flexcan_chip_start, above the location where it dumps the array to
> hardware, FEN is set; this is reflected by the debug (mcr=0x79a2020f).
> 
> I'm surprised this ever worked.. if the area 0x90-0xDF is being used
> by the FIFO internally then the difference between i.MX53 ('working')
> and i.MX6Q ('imprecise abort') is that the i.MX6Q is causing the
> proper bus response. We can only hope that the i.MX53 is ONLY not
> causing the proper bus response and not actually letting you write
> into a FIFO internal data area..
> 
This seems a plausible explanation.

The definition:
|struct flexcan_mb {
|	u32 can_ctrl;
|	u32 can_id;
|	u32 data[2];
|};
|
|/* Structure of the hardware registers */
|struct flexcan_regs {
|	u32 mcr;		/* 0x00 */
[...]
|	struct flexcan_mb cantxfg[64];
|	                          ^^
has obviously nothing to do with reality!


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Matt Sealey]

On Fri, Sep 27, 2013 at 4:41 AM, Lothar Waßmann <LW@karo-electronics.de> wrote:
> Hi,
>
> Matt Sealey writes:
>> Hi Russel,
>>
>> If r2 is indeed the address of the access, then this is entirely down
>> to a misuse of the hardware; from the manual:
>>
>> ~
>> When the FEN bit is set in the MCR, the memory area from 0x80 to 0xFF (which is
>> normally occupied by MB0 to MB7) is used by the reception FIFO engine.
>> Table 34-6
>> shows the Rx FIFO data structure. The region 0x80-0x8F contains an
>> message buffer
>
> What manual are you referring to? In my "i.MX 6Dual/6Quad Applications
> Processor Reference Manual, Rev. 1, 04/2013" flexcan is chapter 26 and
> Table 34-6 is: Single-Ended Source Termination Resistance Settings
> of the HDMI interface.

Actually that was from the i.MX53 manual.. it's table 26-7 in the MX6
manual and the description is different, but the hardware operation is
*identical*.

>> I'm surprised this ever worked.. if the area 0x90-0xDF is being used
>> by the FIFO internally then the difference between i.MX53 ('working')
>> and i.MX6Q ('imprecise abort') is that the i.MX6Q is causing the
>> proper bus response. We can only hope that the i.MX53 is ONLY not
>> causing the proper bus response and not actually letting you write
>> into a FIFO internal data area..
>
> This seems a plausible explanation.
>
> The definition:
> |struct flexcan_mb {
> |       u32 can_ctrl;
> |       u32 can_id;
> |       u32 data[2];
> |};
> |
> |/* Structure of the hardware registers */
> |struct flexcan_regs {
> |       u32 mcr;                /* 0x00 */
> [...]
> |       struct flexcan_mb cantxfg[64];
> |                                 ^^
> has obviously nothing to do with reality!

It does, as you found later, reflect reality as it is where the
message buffers 0-5 go when the FIFO is disabled. After that there are
filter registers.. some unions or egregious casting is required, but
the structure is semi-correct.

This is defined in the i.MX6Q manual at least in section 26.4:

~~~
26.4 Message Buffer Structure
Message Buffer Address: Base + 0x0080-0x047C
~~~

But when FEN is set, this is internal FIFO data after entry 0, as
defined in section 26.5:

~~~
26.5 Rx FIFO Structure
When the MCR[RFEN] bit is set, the memory area from $80 to $DC (which
is normally
occupied by MBs 0 to 5) is used by the reception FIFO engine.

The region 0x80-0x8C contains the output of the FIFO which must be
read by the CPU as
a Message Buffer. This output contains the oldest message received and
not read yet. The
region 0x90-0xDC is reserved for internal use of the FIFO engine.
~~~

Marc - I don't think FLEXCAN has changed layout at all in these areas.
The hardware always worked this way..

The only difference here  is that the i.MX53 is doing something weird
on the bus. The i.MX6Q is giving the absolutely correct BRESP for the
transfer to the peripheral (in effect, a "go away, this is my data"
failure, which the CPU turns into an imprecise abort since it no idea
which transaction it committed aeons ago caused it).

Writes to 0x90 to 0xDF while MCR[FEN] is set *should* cause a data
abort.. because it's not even a read-only region, it *should* be
totally inaccessible to the CPU.

The question I have is, when MCR[FEN] is set on i.MX53, does reading
from those reserved registers give anything but 0's or garbage? I'm
curious, that's all, it doesn't really matter ;)

Ta,
Matt <neko@bakuhatsu.net>

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 4564 bytes --]

On 09/27/2013 07:24 PM, Matt Sealey wrote:
> On Fri, Sep 27, 2013 at 4:41 AM, Lothar Waßmann <LW@karo-electronics.de> wrote:
>>> If r2 is indeed the address of the access, then this is entirely down
>>> to a misuse of the hardware; from the manual:
>>>
>>> ~
>>> When the FEN bit is set in the MCR, the memory area from 0x80 to 0xFF (which is
>>> normally occupied by MB0 to MB7) is used by the reception FIFO engine.
>>> Table 34-6
>>> shows the Rx FIFO data structure. The region 0x80-0x8F contains an
>>> message buffer
>>
>> What manual are you referring to? In my "i.MX 6Dual/6Quad Applications
>> Processor Reference Manual, Rev. 1, 04/2013" flexcan is chapter 26 and
>> Table 34-6 is: Single-Ended Source Termination Resistance Settings
>> of the HDMI interface.
> 
> Actually that was from the i.MX53 manual.. it's table 26-7 in the MX6
> manual and the description is different, but the hardware operation is
> *identical*.

Yes, mx25, mx35, mx53 and mx28 seem to be identical when it comes to the
message buffers. The flexcan on mx6 has an additional acceptance filter
functionality.

>>> I'm surprised this ever worked.. if the area 0x90-0xDF is being used
>>> by the FIFO internally then the difference between i.MX53 ('working')
>>> and i.MX6Q ('imprecise abort') is that the i.MX6Q is causing the
>>> proper bus response. We can only hope that the i.MX53 is ONLY not
>>> causing the proper bus response and not actually letting you write
>>> into a FIFO internal data area..
>>
>> This seems a plausible explanation.
>>
>> The definition:
>> |struct flexcan_mb {
>> |       u32 can_ctrl;
>> |       u32 can_id;
>> |       u32 data[2];
>> |};
>> |
>> |/* Structure of the hardware registers */
>> |struct flexcan_regs {
>> |       u32 mcr;                /* 0x00 */
>> [...]
>> |       struct flexcan_mb cantxfg[64];
>> |                                 ^^
>> has obviously nothing to do with reality!
> 
> It does, as you found later, reflect reality as it is where the
> message buffers 0-5 go when the FIFO is disabled. After that there are
> filter registers.. some unions or egregious casting is required, but
> the structure is semi-correct.
> 
> This is defined in the i.MX6Q manual at least in section 26.4:
> 
> ~~~
> 26.4 Message Buffer Structure
> Message Buffer Address: Base + 0x0080-0x047C
> ~~~
> 
> But when FEN is set, this is internal FIFO data after entry 0, as
> defined in section 26.5:
> 
> ~~~
> 26.5 Rx FIFO Structure
> When the MCR[RFEN] bit is set, the memory area from $80 to $DC (which
> is normally
> occupied by MBs 0 to 5) is used by the reception FIFO engine.
> 
> The region 0x80-0x8C contains the output of the FIFO which must be
> read by the CPU as
> a Message Buffer. This output contains the oldest message received and
> not read yet. The
> region 0x90-0xDC is reserved for internal use of the FIFO engine.
> ~~~
> 
> Marc - I don't think FLEXCAN has changed layout at all in these areas.
> The hardware always worked this way..

All flexcan cores support the RX FIFO, the mx6 supports an additional
mode for different acceptance filters.

> The only difference here  is that the i.MX53 is doing something weird
> on the bus. The i.MX6Q is giving the absolutely correct BRESP for the
> transfer to the peripheral (in effect, a "go away, this is my data"
> failure, which the CPU turns into an imprecise abort since it no idea
> which transaction it committed aeons ago caused it).
> 
> Writes to 0x90 to 0xDF while MCR[FEN] is set *should* cause a data
> abort.. because it's not even a read-only region, it *should* be
> totally inaccessible to the CPU.
> 
> The question I have is, when MCR[FEN] is set on i.MX53, does reading
> from those reserved registers give anything but 0's or garbage? I'm
> curious, that's all, it doesn't really matter ;)

The driver only read from message buffer 0, and uses buffer 8 for tx.
The others are not accessed, unless in that chip start routine. We don't
need this loop, it comes from the original driver. I think no one has
ever noticed that bug, because all other CPUs have not complained.

The driver without the loop is working on mx6 and Lothar is testing it
on mx53.

regards,
Marc

-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Hi,

Marc Kleine-Budde writes:
> On 09/27/2013 07:24 PM, Matt Sealey wrote:
> > Marc - I don't think FLEXCAN has changed layout at all in these areas.
> > The hardware always worked this way..
> 
> All flexcan cores support the RX FIFO, the mx6 supports an additional
> mode for different acceptance filters.
> 
> > The only difference here  is that the i.MX53 is doing something weird
> > on the bus. The i.MX6Q is giving the absolutely correct BRESP for the
> > transfer to the peripheral (in effect, a "go away, this is my data"
> > failure, which the CPU turns into an imprecise abort since it no idea
> > which transaction it committed aeons ago caused it).
> > 
> > Writes to 0x90 to 0xDF while MCR[FEN] is set *should* cause a data
> > abort.. because it's not even a read-only region, it *should* be
> > totally inaccessible to the CPU.
> > 
> > The question I have is, when MCR[FEN] is set on i.MX53, does reading
> > from those reserved registers give anything but 0's or garbage? I'm
> > curious, that's all, it doesn't really matter ;)
> 
> The driver only read from message buffer 0, and uses buffer 8 for tx.
> The others are not accessed, unless in that chip start routine. We don't
> need this loop, it comes from the original driver. I think no one has
> ever noticed that bug, because all other CPUs have not complained.
> 
> The driver without the loop is working on mx6 and Lothar is testing it
> on mx53.
> 
I can confirm, that the driver still works on i.MX53 as before the
patch.
Is someone going to prepare a patch, or should I do it, as I was the
one who first brought up this issue?


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 2083 bytes --]

On 09/30/2013 01:06 PM, Lothar Waßmann wrote:
> Hi,
> 
> Marc Kleine-Budde writes:
>> On 09/27/2013 07:24 PM, Matt Sealey wrote:
>>> Marc - I don't think FLEXCAN has changed layout at all in these areas.
>>> The hardware always worked this way..
>>
>> All flexcan cores support the RX FIFO, the mx6 supports an additional
>> mode for different acceptance filters.
>>
>>> The only difference here  is that the i.MX53 is doing something weird
>>> on the bus. The i.MX6Q is giving the absolutely correct BRESP for the
>>> transfer to the peripheral (in effect, a "go away, this is my data"
>>> failure, which the CPU turns into an imprecise abort since it no idea
>>> which transaction it committed aeons ago caused it).
>>>
>>> Writes to 0x90 to 0xDF while MCR[FEN] is set *should* cause a data
>>> abort.. because it's not even a read-only region, it *should* be
>>> totally inaccessible to the CPU.
>>>
>>> The question I have is, when MCR[FEN] is set on i.MX53, does reading
>>> from those reserved registers give anything but 0's or garbage? I'm
>>> curious, that's all, it doesn't really matter ;)
>>
>> The driver only read from message buffer 0, and uses buffer 8 for tx.
>> The others are not accessed, unless in that chip start routine. We don't
>> need this loop, it comes from the original driver. I think no one has
>> ever noticed that bug, because all other CPUs have not complained.
>>
>> The driver without the loop is working on mx6 and Lothar is testing it
>> on mx53.
>>
> I can confirm, that the driver still works on i.MX53 as before the
> patch.
> Is someone going to prepare a patch, or should I do it, as I was the
> one who first brought up this issue?

I have a patch ready, it's basically what I send you. Can I add your
Tested-by?

Marc

-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Hi,

Marc Kleine-Budde writes:
> On 09/30/2013 01:06 PM, Lothar Waßmann wrote:
> > Hi,
> > 
> > Marc Kleine-Budde writes:
> >> On 09/27/2013 07:24 PM, Matt Sealey wrote:
> >>> Marc - I don't think FLEXCAN has changed layout at all in these areas.
> >>> The hardware always worked this way..
> >>
> >> All flexcan cores support the RX FIFO, the mx6 supports an additional
> >> mode for different acceptance filters.
> >>
> >>> The only difference here  is that the i.MX53 is doing something weird
> >>> on the bus. The i.MX6Q is giving the absolutely correct BRESP for the
> >>> transfer to the peripheral (in effect, a "go away, this is my data"
> >>> failure, which the CPU turns into an imprecise abort since it no idea
> >>> which transaction it committed aeons ago caused it).
> >>>
> >>> Writes to 0x90 to 0xDF while MCR[FEN] is set *should* cause a data
> >>> abort.. because it's not even a read-only region, it *should* be
> >>> totally inaccessible to the CPU.
> >>>
> >>> The question I have is, when MCR[FEN] is set on i.MX53, does reading
> >>> from those reserved registers give anything but 0's or garbage? I'm
> >>> curious, that's all, it doesn't really matter ;)
> >>
> >> The driver only read from message buffer 0, and uses buffer 8 for tx.
> >> The others are not accessed, unless in that chip start routine. We don't
> >> need this loop, it comes from the original driver. I think no one has
> >> ever noticed that bug, because all other CPUs have not complained.
> >>
> >> The driver without the loop is working on mx6 and Lothar is testing it
> >> on mx53.
> >>
> > I can confirm, that the driver still works on i.MX53 as before the
> > patch.
> > Is someone going to prepare a patch, or should I do it, as I was the
> > one who first brought up this issue?
> 
> I have a patch ready, it's basically what I send you. Can I add your
> Tested-by?
> 
Yes, of course.


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 2282 bytes --]

On 09/30/2013 01:37 PM, Lothar Waßmann wrote:
> Hi,
> 
> Marc Kleine-Budde writes:
>> On 09/30/2013 01:06 PM, Lothar Waßmann wrote:
>>> Hi,
>>>
>>> Marc Kleine-Budde writes:
>>>> On 09/27/2013 07:24 PM, Matt Sealey wrote:
>>>>> Marc - I don't think FLEXCAN has changed layout at all in these areas.
>>>>> The hardware always worked this way..
>>>>
>>>> All flexcan cores support the RX FIFO, the mx6 supports an additional
>>>> mode for different acceptance filters.
>>>>
>>>>> The only difference here  is that the i.MX53 is doing something weird
>>>>> on the bus. The i.MX6Q is giving the absolutely correct BRESP for the
>>>>> transfer to the peripheral (in effect, a "go away, this is my data"
>>>>> failure, which the CPU turns into an imprecise abort since it no idea
>>>>> which transaction it committed aeons ago caused it).
>>>>>
>>>>> Writes to 0x90 to 0xDF while MCR[FEN] is set *should* cause a data
>>>>> abort.. because it's not even a read-only region, it *should* be
>>>>> totally inaccessible to the CPU.
>>>>>
>>>>> The question I have is, when MCR[FEN] is set on i.MX53, does reading
>>>>> from those reserved registers give anything but 0's or garbage? I'm
>>>>> curious, that's all, it doesn't really matter ;)
>>>>
>>>> The driver only read from message buffer 0, and uses buffer 8 for tx.
>>>> The others are not accessed, unless in that chip start routine. We don't
>>>> need this loop, it comes from the original driver. I think no one has
>>>> ever noticed that bug, because all other CPUs have not complained.
>>>>
>>>> The driver without the loop is working on mx6 and Lothar is testing it
>>>> on mx53.
>>>>
>>> I can confirm, that the driver still works on i.MX53 as before the
>>> patch.
>>> Is someone going to prepare a patch, or should I do it, as I was the
>>> one who first brought up this issue?
>>
>> I have a patch ready, it's basically what I send you. Can I add your
>> Tested-by?
>>
> Yes, of course.

Tnx,
Marc

-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: imprecise external abort using the flexcan driver on i.MX6Q

[Lothar Waßmann]

Marc Kleine-Budde writes:
> On 09/26/2013 04:01 PM, Lothar Waßmann wrote:
> > Hi,
> > 
> > when enabling the can interface with 'ifconfig can0 up' (after
> > configuring the bitrate with canconfig) on an i.MX6Q board (TX6) I'm
> > getting the following kernel dump:
> > 
> > |flexcan 2094000.flexcan can0: writing ctrl=0x0a212003
> > |flexcan 2094000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
> > |flexcan 2094000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
> > |flexcan 2094000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
> > |Unhandled fault: imprecise external abort (0x1c06) at 0x00057adc
> 
> Looks like a NULL pointer deref to me. But it doesn't make any sense,
> because the offset is way beyond the length of the struct flexcan_regs.
> 
Since it is an 'imprecise' external abort the access is obviously not
directly related to a CPU write, but rather caused by DMA access (by
the flexcan controller). But I cannot imagine why the flexcan
controller would want to access memory when clearing the control word
of the second mailbox.

> > The same kernel/driver works perfectly well on an i.MX53 based board.
> 
> Just to be sure, can you boot with one CPU only.
> 
same result.

> > The data abort happens upon writing to can_ctrl in the second run of
> > this loop in flexcan_chip_start():
> > |	for (i = 0; i < ARRAY_SIZE(regs->cantxfg); i++) {
> > |		flexcan_write(0, &regs->cantxfg[i].can_ctrl);
> > ----------------^ crashes here with i = 1
> 
> Can you instrument flexcan_write().
>
I applied the following patch:
--- .pc/flexcan-debug2/drivers/net/can/flexcan.c	2013-09-27 10:48:23.940490850 +0200
+++ drivers/net/can/flexcan.c	2013-09-27 10:56:17.108489543 +0200
@@ -248,8 +248,28 @@ static inline void flexcan_write(u32 val
 	out_be32(addr, val);
 }
 #else
-static inline u32 flexcan_read(void __iomem *addr)
+#ifdef DEBUG
+#define LINE_OFFS 21
+#define flexcan_read(a)	flexcan_dbg_read(a, __func__, __LINE__)
+static inline u32 flexcan_dbg_read(void __iomem *addr,
+				const char *fn, int ln)
 {
+	u32 val = readl(addr);
+	printk("%s@%d: read %08x from %p\n", fn, ln - LINE_OFFS, val, addr);
+	return val;
+}
+
+#define flexcan_write(v, a)	flexcan_dbg_write(v, a, __func__, __LINE__)
+static inline void flexcan_dbg_write(u32 val, void __iomem *addr,
+				const char *fn, int ln)
+{
+	printk("%s@%d: writing %08x to %p\n", fn, ln - LINE_OFFS, val, addr);
+	writel(val, addr);
+}
+#else
+static inline u32 flexcan_dbg_read(void __iomem *addr)
+{
+	u32 val = readl(addr);
 	return readl(addr);
 }
 
@@ -257,6 +277,7 @@ static inline void flexcan_write(u32 val
 {
 	writel(val, addr);
 }
+#endif /* DEBUG */
 #endif
 
 static inline int flexcan_has_and_handle_berr(const struct flexcan_priv *priv,

This now leads to a hang or a: "BUG: recent printk recursion!"
(obviously because the data abort is induced while the printk() is
being executed).

The line numbers printed are adjusted to match the original kernel
source:
|CAN device driver interface
|flexcan_chip_disable@286: read 5980000f from c0ae8000
|flexcan_chip_disable@288: writing d980000f to c0ae8000
|register_flexcandev@952: read 00000000 from c0ae8004
|register_flexcandev@954: writing 00002000 to c0ae8004
|flexcan_chip_enable@274: read d890000f from c0ae8000
|flexcan_chip_enable@276: writing 5890000f to c0ae8000
|register_flexcandev@959: read 5980000f from c0ae8000
|register_flexcandev@962: writing 7980000f to c0ae8000
|register_flexcandev@969: read 7980000f from c0ae8000
|flexcan_get_berr_counter@296: read 00000000 from c0ae801c
|flexcan_chip_disable@286: read 7980000f from c0ae8000
|flexcan_chip_disable@288: writing f980000f to c0ae8000
|flexcan 2090000.flexcan: device registered (reg_base=c0ae8000, irq=142)
|flexcan_chip_disable@286: read 5980000f from c0af0000
|flexcan_chip_disable@288: writing d980000f to c0af0000
|register_flexcandev@952: read 00000000 from c0af0004
|register_flexcandev@954: writing 00002000 to c0af0004
|flexcan_chip_enable@274: read d890000f from c0af0000
|flexcan_chip_enable@276: writing 5890000f to c0af0000
|register_flexcandev@959: read 5980000f from c0af0000
|register_flexcandev@962: writing 7980000f to c0af0000
|register_flexcandev@969: read 7980000f from c0af0000
|flexcan_get_berr_counter@296: read 00000000 from c0af001c
|flexcan_chip_disable@286: read 7980000f from c0af0000
|flexcan_chip_disable@288: writing f980000f to c0af0000
|flexcan 2094000.flexcan: device registered (reg_base=c0af0000, irq=143)
|flexcan_get_berr_counter@296: read 00000000 from c0ae801c
|flexcan_get_berr_counter@296: read 00000000 from c0af001c
|flexcan_get_berr_counter@296: read 00000000 from c0ae801c
|flexcan_get_berr_counter@296: read 00000000 from c0af001c
|flexcan_chip_enable@274: read f890000f from c0ae8000
|flexcan_chip_enable@276: writing 7890000f to c0ae8000
|flexcan_chip_start@713: writing 02000000 to c0ae8000
|flexcan_chip_start@716: read 5980000f from c0ae8000
|flexcan_set_bittiming@664: read 00002000 from c0ae8004
|flexcan 2090000.flexcan can0: writing ctrl=0x0a212003
|flexcan_set_bittiming@688: writing 0a212003 to c0ae8004
|flexcan_set_bittiming@692: read 5980000f from c0ae8000
|flexcan_set_bittiming@692: read 0a212003 from c0ae8004
|flexcan 2090000.flexcan can0: flexcan_set_bittiming: mcr=0x5980000f ctrl=0x0a212003
|flexcan_chip_start@738: read 5980000f from c0ae8000
|flexcan 2090000.flexcan can0: flexcan_chip_start: writing mcr=0x79a2020f
|flexcan_chip_start@743: writing 79a2020f to c0ae8000
|flexcan_chip_start@757: read 0a212003 from c0ae8004
|flexcan 2090000.flexcan can0: flexcan_chip_start: writing ctrl=0x0a21ac53
|flexcan_chip_start@773: writing 0a21ac53 to c0ae8004
|flexcan_chip_start@776: writing 00000000 to c0ae8080
|flexcan_chip_start@777: writing 00000000 to c0ae8084
|flexcan_chip_start@778: writing 00000000 to c0ae8088
|flexcan_chip_start@779: writing 00000000 to c0ae808c
|flexcan_chip_start@783: writing 04000000 to c0ae8080
|flexcan_chip_start@776: writing 00000000 to c0ae8090

here the CPU either hangs or continues with:

|BUG: recent printk recursion!
|Internal error: : 1c06 [#1] SMP ARM
|Modules linked in: flexcan can_dev snd_soc_imx_sgtl5000 snd_soc_fsl_ssi imx_pcm_dma snd_soc_imx_audmux snd_soc_s
|gtl5000 snd_soc_core regmap_spi snd_pcm_dmaengine snd_pcm snd_timer snd_page_alloc evdev snd_compress snd regmap
|_i2c edt_ft5x06
|CPU: 0 PID: 1271 Comm: ifconfig Not tainted 3.12.0-rc1-next-20130919-karo+ #95
|task: beb83500 ti: be178000 task.ti: be178000
|PC is at vprintk_emit+0x11c/0x4c4
|LR is at vprintk_emit+0x108/0x4c4
|pc : [<8004f908>]    lr : [<8004f8f4>]    psr: 60000093
|sp : be179db8  ip : 00000000  fp : 00000000
|r10: c0ae8004  r9 : 00000000  r8 : 00000000
|r7 : 00000000  r6 : ffffffff  r5 : 8061b930  r4 : 00000000
|r3 : 805f47a4  r2 : 000010ab  r1 : 00000000  r0 : 00000000
|Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
|Control: 10c5387d  Table: 4e1a404a  DAC: 00000015
|Process ifconfig (pid: 1271, stack limit = 0xbe178240)
|Stack: (0xbe179db8 to 0xbe17a000)
|9da0:                                                       00000000 00000000
|9dc0: 00000000 00000000 8061c230 00000034 7f0c1158 be179dcc 60000013 00000000
|9de0: 8051f255 c0ae8000 bebef800 00000000 c0ae8090 00000001 c0ae808c c0ae8004
|9e00: c0ae8088 803f4360 7f0c103e be179e24 00000309 be179e24 c0ae8000 7f0bf82c
|9e20: 7f0c103e 7f0c0b23 00000309 00000000 c0ae8094 0a212003 be8c1400 bebef800
|9e40: 00000000 be81e480 bebef82c 7eb89c08 00008914 00000000 00000001 7f0c08fc
|9e60: bebef800 bebef800 bebef800 000000c1 7f0c0e68 8036f358 8036f2dc bebef800
|9e80: 000000c1 00000001 00040080 8036f590 beb83500 bebef800 00040080 beb37600
|9ea0: bebef800 8036f690 00000000 beb3760c beb37600 803b9658 00000000 01b89c08
|9ec0: 306e6163 00000000 00000000 00000000 000000c1 76fa44d0 7eb89ed5 000566ce
|9ee0: 00000080 00008914 be047b40 7eb89c08 00008914 00000003 be178000 7eb89c08
|9f00: bd844920 8035ae40 7eb89c08 be047b40 00000003 800bb464 7eb89c08 800bbfa8
|9f20: be166a80 00000003 804379c0 800add4c 00000020 00000003 805eeff8 be179f60
|9f40: 00000003 800ade08 805e7bb8 805eeff8 bd844900 00000000 00000002 8035a5fc
|9f60: be8ce3d0 7eb89c08 be047b40 00000000 00008914 00000003 be178000 00000000
|9f80: 00000000 800bc030 00000003 00000000 0005868f 00000004 00054f14 00000036
|9fa0: 8000e7e4 8000e660 0005868f 00000004 00000003 00008914 7eb89c08 0005868f
|9fc0: 0005868f 00000004 00054f14 00000036 00000000 00000000 7eb89ee3 00000000
|9fe0: 00000000 7eb89bf0 0000cad4 76e6f87c 20000010 00000003 4eff1811 4eff1c11
|[<8004f908>] (vprintk_emit+0x11c/0x4c4) from [<803f4360>] (printk+0x2c/0x3c)
|[<803f4360>] (printk+0x2c/0x3c) from [<7f0bf82c>] (flexcan_chip_start+0x358/0x620 [flexcan])
|[<7f0bf82c>] (flexcan_chip_start+0x358/0x620 [flexcan]) from [<7f0c08fc>] (flexcan_open+0x74/0x118 [flexcan])
|[<7f0c08fc>] (flexcan_open+0x74/0x118 [flexcan]) from [<8036f358>] (__dev_open+0x7c/0xfc)
|[<8036f358>] (__dev_open+0x7c/0xfc) from [<8036f590>] (__dev_change_flags+0x8c/0x118)
|[<8036f590>] (__dev_change_flags+0x8c/0x118) from [<8036f690>] (dev_change_flags+0x10/0x44)
|[<8036f690>] (dev_change_flags+0x10/0x44) from [<803b9658>] (devinet_ioctl+0x2a4/0x62c)
|[<803b9658>] (devinet_ioctl+0x2a4/0x62c) from [<8035ae40>] (sock_ioctl+0x220/0x274)
|[<8035ae40>] (sock_ioctl+0x220/0x274) from [<800bb464>] (vfs_ioctl+0x28/0x3c)
|[<800bb464>] (vfs_ioctl+0x28/0x3c) from [<800bbfa8>] (do_vfs_ioctl+0x53c/0x590)
|[<800bbfa8>] (do_vfs_ioctl+0x53c/0x590) from [<800bc030>] (SyS_ioctl+0x34/0x58)
|[<800bc030>] (SyS_ioctl+0x34/0x58) from [<8000e660>] (ret_fast_syscall+0x0/0x30)
|Code: e59f338c e3a00000 e3540000 e583705c (0a00000c) 
|---[ end trace 16ce38ae8b7c1165 ]---
|Kernel panic - not syncing: Fatal exception


Lothar Waßmann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstraße 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Geschäftsführer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info@karo-electronics.de
___________________________________________________________