Re: [PATCH v7 08/10] accel/rocket: add RK3576 NPU (RKNN) support

[Igor Paunovic <royalnet026@gmail.com>]

Hi Jiaxing,

Here is operator 28 on RK3588, and it changes the picture: it is not an
empty convolution here, and once the reference is right it is the
cleanest operator in the whole model.

Setup as before - same board, same upstream Mesa at bf70ab68a21, none of
your four unposted fixes, ROCKET_SEED=7, model truncated the same way as
the table (subgraph output repointed to the operator's output tensor,
here the 1x1x1001 BiasAdd). One change to disclose: the kernel moved
from 7.2.0-rc6 to 7.2.0-rc7 as my daily since the table. The rc6 run also
had my devfreq patches in the rocket driver; this rc7 rocket has no
devfreq node registered, so it runs like yours does. Before running
operator 28 I re-ran operators 0-4 as a control and all five reproduce
the published column exactly (32/32 md 1, 28/32 md 3, 22/64 md 6, 21/64
md 13, 8/128 md 15), so neither the kernel change nor the devfreq
difference moves these numbers.

Scored exactly the way perch.py scores, operator 28 reads:

  674/1001 channels match, maxdiff 29
  NOT empty: only 36 of 1001 channels sit at the output zero point,
  71 distinct output values, min 37, max 117 (zp = 66)

But that maxdiff 29 is the reference, not the hardware. perch.py scores
against max(cpu, zp), which is right for operators 0-26 with their fused
ReLU6 - the hardware really does floor those at the zero point. Operator
28 has no fused activation (it is the logits BiasAdd), and RK3588 does
not floor it: the raw CPU output has 362 channels below the zero point
(down to 37) and the NPU follows them down instead of clamping.

Against the raw, unclamped CPU output:

  seed 7: 1001/1001 channels within 1, maxdiff 1 (708 exact, 293 off
          by one)
  seed 8: 1001/1001 within 1, maxdiff 1
  three seed-7 runs byte-identical

So on RK3588 with upstream Mesa, 1001 output channels - odd AND not a
multiple of 8 - come out clean without either of your unposted fixes.
Whatever those two fixes change, this SoC's path does not need it for
correctness at this operator. It may be worth re-scoring your RK3576
operator 28 against the unclamped CPU as well before reading its "empty
convolution" - if RK3576 also skips the floor on a no-activation conv,
the max(cpu, zp) reference alone moves several hundred channels.

Two small notes for reproduction: perch.py prints the headline and then
crashes on this operator at its spatial-profile section (got[1:-1,1:-1]
on a 1x1 surface is empty) - the numbers above it are unaffected. And I
find it a nice detail that my board reads md 23 at operator 26, yet
after the CPU average-pool the classifier lands within 1 of the CPU -
the pooling averages the deep-layer noise away, which fits your 1000 of
1001 end-to-end result.

This reply was prepared with the help of Claude (Anthropic). The board,
the runs and the numbers are mine, and each number above comes from a
run on this machine today.

Best regards,
Igor

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip