Re: Receiving report multiple times when changing cpu

[Thilo Roerig <thilo.roerig@googlemail.com>]

> On Fri, Oct 14, 2022 at 01:18:55PM +0200, Thilo Roerig wrote:
> > > > I have a problem with a custom usb device on a raspberry pi running a
> > > > buildroot 64bit linux with a v5.19.14 kernel.
> > > > I ran into a problem reading data from a custom usb hid device using
> > > > interrupt transfers. I'm using
> > > > the python hidapi library (https://pypi.org/project/hidapi/) with the
> > > > libusb backend. After sending
> > > > a custom command to the device, the driver changes from usbhid to usbfs.
> > > > The device starts sending
> > > > reports of size 1023 bytes to endpoint 0x83. `usb-devices` yields the
> > > > following output:
> > >
> > > USB devices do not "send" anything, they require a request from the host
> > > to be able to transfer any data.  So you have to ask the device "do you
> > > have any data?" before it can respond with anything.
> > >
> > > So just wait, don't ask the device if you are not ready for the data to
> > > be recieved, do not allow the device to continue to flood the data to
> > > you if you can not recieve it all.
> >
> > Thank you for pointing this out. I looked into the implementation of the hidapi
> > library. The library immediately starts submitting transfer when opening the
> > device. So to get a better grip on the issue, I'm now using libusb and in
> > this case I only get the reports, when I'm asking for it.
> >
> > > > T:  Bus=01 Lev=03 Prnt=13 Port=00 Cnt=01 Dev#= 14 Spd=480 MxC> > > h= 0
> > > > D:  Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
> > > > C:  #Ifs= 1 Cfg#= 1 Atr=80 MxPwr=500mA
> > > > I:  If#= 0 Alt= 0 #EPs= 1 Cls=03(HID  ) Sub=00 Prot=00 Driver> > > =usbfs
> > > > E:  Ad=83(I) Atr=03(Int.) MxPS=1024 Ivl=125us
> > > >
> > > > The reports are numbered by an increasing report id and are available via
> > > > interrupt transfer approx. every
> > > > 230 microseconds (1.85 microframes). The device does not retain any old
> > > > reports.
> > >
> > > So is the buffer in the device overflowing because the host is not
> > > asking for enough data from it, or is the data overflowing in the host
> > > because you are not reading the buffers fast enough?
> >
> > The buffer on the device does not buffer reports, that are as old as the
> > reports I am receiving via libusb. Interestingly, the speed that I can receive
> > the reports in userspace increases, when the wrong pattern starts to occur.
> > Initially, I receive ~4350 reports per second (with correct report id and
> > increasing order), but once the wrong pattern occurs, the speed goes up to
> > ~7650 reports per second, which is approx. 1 microframe. Without having found
> > any reference in the code, the pattern in the data looks as if some dequeue
> > pointer is "overtaking" the enqueue pointer in a ring buffer - but this is just
> > a wild guess. It would maybe explain the pattern in the data and maybe also the
> > speedup once the ring buffer is not preventing the dequeue from reading once
> > the enqueue pointer is reached.
> 
> Are you sure that the firmware in the device itself is not just broken
> when you ramp up the speed?  It is the thing that is filling the buffers
> to the host, when it has it ready, so it is the thing that is in control
> of how fast it responds.
> 
> If all works well at slower speeds, try just doing that and not letting
> the device itself work too hard.
> 
> What does other operating systems do with this device, how do they
> control it?  With a in-kernel driver or with userspace code that takes a
> long time between requests?

Based on your (and Alans) replies, I could narrow down the potential reasons
for the duplicate reports and it turned out to be a problem with a buffer on 
the device when reports were not read fast enough.

Together with the device provider we could adapt the firmware on the device to
avoid the buffer overflow that caused the duplicate reports. Additionally, I
moved some of the code processing the reports from python to C, so that the HID
report queue used by hidapi does not overflow, when reading to slowly from
userspace. 

I did not dig deeper into how windows handles the device, but this is something
I will investigate, if I continue to have issues with the reports.

Thank you very much again for your great help.

Best
Thilo Roerig