Re: [PATCH v7 08/10] accel/rocket: add RK3576 NPU (RKNN) support

[Jiaxing Hu <gahing@gahingwoo.com>]

Hi Igor,

Thank you. This is the column I cannot produce, and it settles the
question it was aimed at.

First, what you were comparing against has moved. The RK3576 collapse
from operator 4 with maxdiff 255 is gone; four Mesa faults came out
between that mail and yours. Against the same simulation RK3576 now
reads 21/64 md 13, 18/128 md 14, 9/128 md 13, 4/128 md 23 and 7/128
md 10 at operators 3 to 7, which is the simulation exactly, and
MobileNet end to end is 1000 of 1001 outputs within one count of the
CPU. So the thing your run was built to characterise no longer exists,
and your table is measuring something else.

That something is worth a note, and I do not have an explanation for it.
From operator 4 on RK3588 sits below the simulation where RK3576 now
sits on it, 8 of 128 against 18 at operator 4 and 20 of 256 against 36
at operator 8, with a few hundred elements per surface already off by 1
to 4 at operators 0 to 3.

My first thought was that this is my Mesa tree rather than your silicon,
since you ran upstream and four fixes are not posted yet. Checking it
before writing it: three of those four are not gated on the SoC, but
none of them bite at MobileNet's channel counts.

  the CBUF row cost fires when ceil(ic/16) is 3 modulo 4, which is 33 to
  48 or 97 to 112 input channels. MobileNet has 3, 32, 64, 128, 256,
  512 and 1024, so 1, 2, 4, 8, 16, 32 and 64 atoms. Never 3.

  the output channel pair rounding fires on an odd count, and the
  coefficient operand alignment on a count that is not a multiple of 8.
  In this model that is only operator 28, the 1001 channel classifier,
  which is past the end of your table.

  the fourth, the tiled 1x1 weight layout, is inside the RK3576 path.

So my hypothesis does not survive its own arithmetic and I am not going
to send it as one. Operators 4 to 26 on your board are unexplained by
anything I have, which means either RK3588's own path carries a rounding
difference RK3576 does not, or my simulation is closer to RK3576 than to
the hardware in general because that is the chip I tuned it against. The
second is the more likely and the less flattering.

If you ever want to close it, the cheapest probe is operator 28 rather
than any of the ones you ran: 1001 output channels is odd AND not a
multiple of 8, so it is the one operator in this model where two of my
unposted fixes would change anything on RK3588. On RK3576 without them
it came back an empty convolution.

One more thing, since it cuts against my own earlier note: operator 3
scoring identically on both chips retires the suspicion in my round 104
write-up that its asymmetric stride 2 padding was the first RK3576
divergence. It was not, on either SoC.

Jiaxing