Re: GPMI iMX6ull timeout on DMA

Greg Ungerer
2019-07-29T12:19:29Z

Hi Michael,

On 29/7/19 6:42 pm, Michael Nazzareno Trimarchi wrote:
> Hi all
>
> On Mon, Jul 29, 2019 at 10:36 AM Miquel Raynal
> <miquel.raynal@bootlin.com> wrote:
>>
>> Hi Greg,
>>
>> One question below.
>>
>> +Michael
>> +Sascha
>>
>> Hello Michael, here is a similar issue to yours, I know you did not
>> have enough time to share your solution but here we have someone else
>> reproducing the issue, would you mind sharing a branch or a patch, even
>> a WIP one, just to help debugging?
>>
>
> I have patches reverted as I mention in the email. The step to
> reproduce is simple.
>
> Just reboot every successful boot.

Testing like this how often does it occur?

Regards
Greg

> Michael
>
>> Greg Ungerer <gerg@kernel.org> wrote on Mon, 29 Jul 2019 16:41:51 +1000:
>>
>>> Hi Miquel,
>>>
>>> I am experiencing a problem with NAND flash DMA timeouts on
>>> iMX6ull based boards. The problem is very similar to that
>>> described in:
>>>
>>> https://linux-mtd.infradead.narkive.com/JIUulfFB/gpmi-imx6ull-timeout-on-dma
>>>
>>> That didn't come to any specific resolution that I could see
>>> in that thread.
>>>
>>> The boot trace on the console for me looks like this:
>>>
>>> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xda
>>> nand: Micron MT29F2G08ABAEAWP
>>> nand: 256 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
>>> gpmi-nand 1806000.gpmi-nand: DMA timeout, last DMA
>>> gpmi-nand 1806000.gpmi-nand: Show GPMI registers :
>>> gpmi-nand 1806000.gpmi-nand: offset 0x000 : 0x20830002
>>> gpmi-nand 1806000.gpmi-nand: offset 0x010 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x020 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x030 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x040 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x050 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x060 : 0x01c6800c
>>> gpmi-nand 1806000.gpmi-nand: offset 0x070 : 0x00010101
>>> gpmi-nand 1806000.gpmi-nand: offset 0x080 : 0xe0000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x090 : 0x23023336
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0a0 : 0x000001ee
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0b0 : 0xff000001
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0c0 : 0x00000001
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0d0 : 0x05020000
>>> gpmi-nand 1806000.gpmi-nand: Show BCH registers :
>>> gpmi-nand 1806000.gpmi-nand: offset 0x000 : 0x00000100
>>> gpmi-nand 1806000.gpmi-nand: offset 0x010 : 0x00000010
>>> gpmi-nand 1806000.gpmi-nand: offset 0x020 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x030 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x040 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x050 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x060 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x070 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x080 : 0x030a2080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x090 : 0x083e2080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0a0 : 0x070a4080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0b0 : 0x10da4080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0c0 : 0x070a4080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0d0 : 0x10da4080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0e0 : 0x070a4080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x0f0 : 0x10da4080
>>> gpmi-nand 1806000.gpmi-nand: offset 0x100 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x110 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x120 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x130 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x140 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x150 : 0x20484342
>>> gpmi-nand 1806000.gpmi-nand: offset 0x160 : 0x01000000
>>> gpmi-nand 1806000.gpmi-nand: offset 0x170 : 0x00000000
>>> gpmi-nand 1806000.gpmi-nand: BCH Geometry :
>>> GF length : 13
>>> ECC Strength : 8
>>> Page Size in Bytes : 2110
>>> Metadata Size in Bytes : 10
>>> ECC Chunk0 Size in Bytes: 512
>>> ECC Chunkn Size in Bytes: 512
>>> ECC Chunk Count : 4
>>> Payload Size in Bytes : 2048
>>> Auxiliary Size in Bytes: 16
>>> Auxiliary Status Offset: 12
>>> Block Mark Byte Offset : 1999
>>> Block Mark Bit Offset : 0
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -110
>>> nand: timing mode 5 not acknowledged by the NAND chip
>>
>> What is the final timing mode used? Most of us tested in mode 5 I
>> guess, maybe mode 4 is broken (don't know if this is the one used here,
>> neither why mode 5 is refused). Can you please try by limiting the mode
>> to 0, 1, 2... until, hopefully, we narrow down to the failing mode.
>>
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> Scanning device for bad blocks
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> ....
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> gpmi-nand 1806000.gpmi-nand: Chip: 0, Error -22
>>> 5 fixed-partitions partitions found on MTD device gpmi-nand
>>> Creating 5 MTD partitions on "gpmi-nand":
>>> 0x000000000000-0x000000500000 : "u-boot"
>>> 0x000000500000-0x000000600000 : "u-boot-env"
>>> 0x000000600000-0x000000800000 : "log"
>>> 0x000000800000-0x000010000000 : "flash"
>>> 0x000000000000-0x000010000000 : "all"
>>> gpmi-nand 1806000.gpmi-nand: driver registered.
>>>
>>>
>>> This is using a linux kernel v5.1.14. I have seen this happen on
>>> a number of boards I have here - but it is only occasional. It
>>> only happens once in a while on boot, maybe 1 in 40 or more times.
>>> So it can take quite a while to reproduce (using a boot loop setup).
>>
>> That's strange... I don't get what would produce such unstable issue.
>>
>>>
>>> As per the email thread I pointed to above I looked at reverting
>>> those patches, but that was not at all easy given how much the gpmi
>>> driver code had moved. So instead I modified the code with this:
>>>
>>> --- a/linux/drivers/mtd/nand/raw/gpmi-nand/gpmi-lib.c
>>> +++ b/linux/drivers/mtd/nand/raw/gpmi-nand/gpmi-lib.c
>>> @@ -481,6 +481,7 @@ static void gpmi_nfc_compute_timings(struct gpmi_nand_data *this,
>>> void gpmi_nfc_apply_timings(struct gpmi_nand_data *this)
>>> {
>>> +#if 0
>>> struct gpmi_nfc_hardware_timing *hw = &this->hw;
>>> struct resources *r = &this->resources;
>>> void __iomem *gpmi_regs = r->gpmi_regs;
>>> @@ -505,6 +512,7 @@ void gpmi_nfc_apply_timings(struct gpmi_nand_data *this)
>>> /* Wait for the DLL to settle. */
>>> udelay(dll_wait_time_us);
>>> +#endif
>>> }
>>> int gpmi_setup_data_interface(struct nand_chip *chip, int chipnr,
>>>
>>> So far after a couple of days of testing with this I no longer
>>> see the DMA timeout.
>>>
>>> Any thoughts?
>>>
>>> Regards
>>> Greg
>>>
>>
>> Thanks,
>> Miquèl