I2C boot issue with RPI3B

I2C boot issue with RPI3B

[Sergey Suloev]

Greetings,

I am dealing with a custom hardware based on RPI3B running kernel 4.19 
(or 4.20).
My device has 2 slaves on I2C  bus: codec WM8731 (on address 0x1a) and 
clock generator si5351a (on 0x60).

It has been a lot of time for me fighting with an I2C issue which does 
not allow the two I2C devices work correctly together: I2C bus hangs up 
at kernel boot. The issue happens in 50-70% boot cases and the fail 
percentage noticeably depends on  the bus clock set in DTS: the more 
frequency set the less fail percentage.
The kernel itself loads successfully, I can ssh to the RPI in order to 
issue "i2cdetect -y 0" and make sure that I2C bus is locked completely 
and not responding.

So far I have found the only way to make it work :  to compile si5351a 
driver into kernel and use wm8731 driver as a module.
This probably makes the clock driver load 100% before the codec and I2C 
bus does not hang up.
Since it can be fixed this way the problem really looks like a "race 
condition" in I2C bus/driver.

Would be really nice to know your opinion and possible ways to fix this.


Best regards,

Sergey


_______________________________________________
linux-rpi-kernel mailing list
linux-rpi-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rpi-kernel

Re: I2C boot issue with RPI3B

[Stefan Wahren]

Hi Sergey,

[add Noralf and drop the Broadcom guys from CC since this specific to
the RPi]

Am 07.12.18 um 14:36 schrieb Sergey Suloev:
> Greetings,
>
> I am dealing with a custom hardware based on RPI3B running kernel 4.19
> (or 4.20).
> My device has 2 slaves on I2C  bus: codec WM8731 (on address 0x1a) and
> clock generator si5351a (on 0x60).
>
> It has been a lot of time for me fighting with an I2C issue which does
> not allow the two I2C devices work correctly together: I2C bus hangs
> up at kernel boot.
Could you give some kernel message or some measurements of the I2C line
states?
> The issue happens in 50-70% boot cases and the fail percentage
> noticeably depends on  the bus clock set in DTS: the more frequency
> set the less fail percentage.
I assume you don't use the downstream cpufreq driver, which could
generate interferences on the clock.
> The kernel itself loads successfully, I can ssh to the RPI in order to
> issue "i2cdetect -y 0" and make sure that I2C bus is locked completely
> and not responding.
>
AFAIK this I2C0 is shared with the VideoCore. Did you already tried I2C1
which should be exclusive for the ARM core(s)?

Stefan

> So far I have found the only way to make it work :  to compile si5351a
> driver into kernel and use wm8731 driver as a module.
> This probably makes the clock driver load 100% before the codec and
> I2C bus does not hang up.
> Since it can be fixed this way the problem really looks like a "race
> condition" in I2C bus/driver.
>
> Would be really nice to know your opinion and possible ways to fix this.
>
>
> Best regards,
>
> Sergey
>

_______________________________________________
linux-rpi-kernel mailing list
linux-rpi-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rpi-kernel

Re: I2C boot issue with RPI3B

[Sergey Suloev]

Hi, Stefan

On 12/7/18 6:42 PM, Stefan Wahren wrote:
> Hi Sergey,
>
> [add Noralf and drop the Broadcom guys from CC since this specific to
> the RPi]
>
> Am 07.12.18 um 14:36 schrieb Sergey Suloev:
>> Greetings,
>>
>> I am dealing with a custom hardware based on RPI3B running kernel 4.19
>> (or 4.20).
>> My device has 2 slaves on I2C  bus: codec WM8731 (on address 0x1a) and
>> clock generator si5351a (on 0x60).
>>
>> It has been a lot of time for me fighting with an I2C issue which does
>> not allow the two I2C devices work correctly together: I2C bus hangs
>> up at kernel boot.
> Could you give some kernel message or some measurements of the I2C line
> states?


I will provide dmesg as soon as I get to my machine.
And what measurements are you talking about ? I have a logical analyzer 
here and theoretically can get some bus snapshots, but I don't know what 
I should look for.


>> The issue happens in 50-70% boot cases and the fail percentage
>> noticeably depends on  the bus clock set in DTS: the more frequency
>> set the less fail percentage.
> I assume you don't use the downstream cpufreq driver, which could
> generate interferences on the clock.


No, it is 100% disabled as you mentioned this problem before in github 
thread.


>> The kernel itself loads successfully, I can ssh to the RPI in order to
>> issue "i2cdetect -y 0" and make sure that I2C bus is locked completely
>> and not responding.
>>
> AFAIK this I2C0 is shared with the VideoCore. Did you already tried I2C1
> which should be exclusive for the ARM core(s)?

I am aware of that. In my system 0 means i2c1.
Hard to say what exactly in device tree is managing this but I can't 
access i2c0 with i2cdetect, i.e. it is kinda disabled. I can access two 
i2c busses (probably i2c1 and i2c2) and 0 is assigned to the one 
connected to GPIO 2 & 3.

> Stefan
>
>> So far I have found the only way to make it work :  to compile si5351a
>> driver into kernel and use wm8731 driver as a module.
>> This probably makes the clock driver load 100% before the codec and
>> I2C bus does not hang up.
>> Since it can be fixed this way the problem really looks like a "race
>> condition" in I2C bus/driver.
>>
>> Would be really nice to know your opinion and possible ways to fix this.
>>
>>
>> Best regards,
>>
>> Sergey
>>

_______________________________________________
linux-rpi-kernel mailing list
linux-rpi-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rpi-kernel

Re: I2C boot issue with RPI3B

[Sergey Suloev]

Hi, Stefan,

today I made a stupid patch for testing only: I instructed the wm8731 
codec's probe function to wait for si5351's clock output and return 
EPROBE_DEFER until it is ready. This delays any I2C bus access from 
wm8731 driver until si5351 initialization is complete. It worked like a 
charm.

Now I have no idea what type of issue we are dealing with. I can't 
believe that I2C bus should allow only one device at a time.

Needless to say if only one of the drivers is loaded(either wm8731 or 
si5351) then  I2C bus works with no issue.

Sergey


On 12/7/18 6:42 PM, Stefan Wahren wrote:
> Hi Sergey,
>
> [add Noralf and drop the Broadcom guys from CC since this specific to
> the RPi]
>
> Am 07.12.18 um 14:36 schrieb Sergey Suloev:
>> Greetings,
>>
>> I am dealing with a custom hardware based on RPI3B running kernel 4.19
>> (or 4.20).
>> My device has 2 slaves on I2C  bus: codec WM8731 (on address 0x1a) and
>> clock generator si5351a (on 0x60).
>>
>> It has been a lot of time for me fighting with an I2C issue which does
>> not allow the two I2C devices work correctly together: I2C bus hangs
>> up at kernel boot.
> Could you give some kernel message or some measurements of the I2C line
> states?
>> The issue happens in 50-70% boot cases and the fail percentage
>> noticeably depends on  the bus clock set in DTS: the more frequency
>> set the less fail percentage.
> I assume you don't use the downstream cpufreq driver, which could
> generate interferences on the clock.
>> The kernel itself loads successfully, I can ssh to the RPI in order to
>> issue "i2cdetect -y 0" and make sure that I2C bus is locked completely
>> and not responding.
>>
> AFAIK this I2C0 is shared with the VideoCore. Did you already tried I2C1
> which should be exclusive for the ARM core(s)?
>
> Stefan
>
>> So far I have found the only way to make it work :  to compile si5351a
>> driver into kernel and use wm8731 driver as a module.
>> This probably makes the clock driver load 100% before the codec and
>> I2C bus does not hang up.
>> Since it can be fixed this way the problem really looks like a "race
>> condition" in I2C bus/driver.
>>
>> Would be really nice to know your opinion and possible ways to fix this.
>>
>>
>> Best regards,
>>
>> Sergey
>>

_______________________________________________
linux-rpi-kernel mailing list
linux-rpi-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rpi-kernel