Re: USB network gadget / DWC3 issue

[Ferry Toth <fntoth@gmail.com>]

Hi,

Op 02-04-2021 om 22:16 schreef Thinh Nguyen:
> Ferry Toth wrote:
>> Hi
>>
>> Op 30-03-2021 om 23:57 schreef Ferry Toth:
>>> Hi
>>>
>>> Op 30-03-2021 om 22:26 schreef Ferry Toth:
>>>> Hi,
>>>>
>>>> Op 30-03-2021 om 18:17 schreef Felipe Balbi:
>>>>> Hi,
>>>>>
>>>>> Andy Shevchenko <andy.shevchenko@gmail.com> writes:
>>>>>> Hi!
>>>>>>
>>>>>> I have a platform with DWC3 in Dual Role mode. Currently I'm
>>>>>> experimenting on v5.12-rc5 with a few patches (mostly configuration)
>>>>>> applied [1]. I'm using Debian Unstable on the host machine and
>>>>>> BuildRoot with the above mentioned kernel on the target.
>>>>>>
>>>>>> **So, scenario 0:
>>>>>> 1. Run iperf3 -s on target
>>>>>> 2. Run iperf3 -c ... -t 0 on the host
>>>>>> 3.  0.00-10.36  sec   237 MBytes   192 Mbits/sec
>>>>>> receiver
>>>>>>
>>>>>> **Scenario 1:
>>>>>> 1. Now, detach USB cable, wait for several seconds, attach it back,
>>>>>> repeat above:
>>>>>> 0.00-9.94   sec   209 MBytes   176 Mbits/sec receiver
>>>>>>
>>>>>> Note the bandwidth drop (177 vs. 192).
>>>>>>
>>>>>> (Repeating scenario 1 will give now the same result)
>>>>>>
>>>>>> **Scenario 2.
>>>>>> 1. Detach USB cable, attach a device, for example USB stick,
>>>>>> 2. See it being enumerated and detach it.
>>>>>> 3. Attach cable from host
>>>>>> 4 .   0.00-19.36  sec   315 MBytes   136 Mbits/sec
>>>>>> receiver
>>>>>>
>>>>>> Note even more bandwidth drop!
>>>>>>
>>>>>> (Repeating scenario 1 keeps the same lower bandwidth)
>>>>>>
>>>>>> NOTE, sometimes on this scenario after several seconds the target
>>>>>> simply reboots (w/o any logs [from kernel] printed)!
>>>>>>
>>>>>> So, any pointers on how to debug and what can be a smoking gun here?
>>>>>>
>>>>>> Ferry reported this in [2]. There are different kernel versions and
>>>>>> tools to establish the connection (like connman vs. none in my case).
>>>>>>
>>>>>> [1]:
>>>>>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEppG6qq-d$
>>>>>> [2]:
>>>>>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/issues/31__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEptMCrp-F$
>>>>>
>>>>> dwc3 tracepoints should give some initial hints. Look at packets sizes
>>>>> and period of transmission. From dwc3 side, I can't think of
>>>>> anything we
>>>>> would do to throttle the transmission, but tracepoints should tell a
>>>>> clearer story.
>>>>>
>>>> My testing (but yes, with difference kernel and network managed by
>>>> connman) shows:
>>>>
>>>> 1) on cold boot eem network gadget works fine
>>>>
>>>> 2) after unplug or warm reboot (which is also an unplug) it's broken,
>>>> speed is lost (|12.0 Mbits/sec from 200Mb/s normally)|, packets lost,
>>>> no configuration received from dhcp, occasional reboot, only way to
>>>> fix is cold boot
>>>>
>>>> 3) if before unplug `connmanctl disable gadget`, on replugging and
>>>> enabling it works fine
>>>>
>>>> My theory is that some HW register is disturbed on a surprise unplug,
>>>> but not reset on plug or warm boot. But on cold boot is cleared.
>>>> Maybe that can help to narrow down tracepoints?
>>>>
>>> I captured a plug after warm and after cold boot. This includes
>>> network setup (dhcp). You can find it in [2] or directly link here:
>>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/files/6232410/boot.zip__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEpjhhrwa-$
>>
>>
>> While the above traces in boot.zip allow compare which regs not
>> correctly initialized on warm boot, I have now captured traces of
>> unplug/plug.
>>
>> Here kernel is 5.10.27 (LTS), cold booted with USB cable plugged and the
>> eem gadget network setup (dhcp). Then trace unplug. Then trace plug.
>>
>> After plug the eem connection is again broken.
>>
>> This might allow figuring out what goes wrong on unplug. Traces here:
>> https://urldefense.com/v3/__https://github.com/andy-shev/linux/files/6250924/plug-unplug.zip__;!!A4F2R9G_pg!KpQnudHIK6XgK6HbPaqtbVgipDmkNBWewo-euAIuBlGdtSiaQiJ8jLn9OoMEpgii82NS$
>> **
>>
> 
> Hi,
> 
> Were you able to narrow down the issue to only DWC3 device? (i.e. you
> tested with different hosts and different device controllers to confirm
> this)

I haven't tried with other devices. I have been forced to replace my 
host mobo and nothing changed. But I didn't pay attention to the 
particular host controller.

> Did you see this issue previously? If not, is it possible to do git
> bisection?

This is with Intel Edison where main line usb gadget support appeared 
around 4.19 iirc. I believed the problem appeared between 5.4 and 5.7 
and tried to bisect but failed.

I realize only now that I failed because:
1) 5.4 already has this issue as I recently retested
2) I didn't use a reproducible criterion. After warm reboot the eem 
gadget fails, but you can flip the host/gadget switch back and forth and 
have the illusion that the connection restored.

The scenario described here is reproducible: leaving the switch in 
gadget mode eem works after cold boot only. And it likely breaks on unplug.

A 2nd hint is that disabling gadget (I used `connmanctl disable gadget` 
but I believe that has the same effect as `iw link set dev usb0 down`) 
before unplug prevents messing up the driver, so you can replug and 
enable again.

> BR,
> Thinh
>