Re: [PATCH v7 08/10] accel/rocket: add RK3576 NPU (RKNN) support

[Jiaxing Hu <gahing@gahingwoo.com>]

Hi Igor,

I owe you a correction, and it is on the clamp.

I wrote that since the vendor userspace on the same silicon does not clamp, the
clamp had to be mine and there was no hardware behaviour left to appeal to. The
first half holds and the conclusion does not. The output stage is

    byte = clamp(max(requant + L, 0) + offset, -128, 127) + 128

with the floor applied before the offset, so a negative requant is gone before
the offset can do anything about it. That is the silicon, and this driver was
shipping L of 0.

L is the output zero point, which is the smallest lift that clears the floor,
and with it the expression collapses to clamp(requant + out_zp, 0, 255) with the
offset a constant -128. On hardware conv2d-cal went from 0 of 128 channels
against the unclamped output to 128 of 128, and every model whose zero point is
zero came back byte identical to emitting no lift at all, which is the control I
would want to see if someone told me this.

The reason I am not claiming to understand it, and it is a real tension rather
than a detail. If the vendor compensated the way I now do, its output offset
would be a constant -128. It is not. Across six compiled .rknn, DPU 0x40ac
reads -2, +9, +17, 0, -128 and +10, which is out_zp - 0x80 exactly, the same
expression this driver used before the fix. A floor that is unconditional in the
silicon should therefore pin the vendor's output too, and measured on the same
board it does not. So either the compensation is somewhere I cannot see, which
would be A, since librknnrt builds the coefficient buffer at load time and it is
not in the .rknn, or the floor has an enable I have not found. My model fits
everything I can measure on my own stack and does not explain that row.

If you can read A on RK3588, or force an output below its zero point there, that
would separate the two.

Two things I should not repeat at you. The 88 and 120 sweep went out on the
17th, and the reference argument is older than that. I lost track of both and
nearly sent you a mail announcing them as news.

One thing that column earned. Scoring each MobileNet operator on the board
against exact arithmetic rather than against tflite puts every one of them
between 99.93 and 99.99 percent of pixels identical, and the per operator
figures chainmodel.py predicts, 4 of 128 at operator 6 and 34 of 256 at
operator 8, came back exactly on hardware. Your reading rule was right.

What is still genuinely open on my side and unchanged. RGP_CNTER 8, OW_SRC 1 and
the two in the reserved range have never been varied one at a time. I said that
would happen before the Mesa series went out and then opened it anyway, which
was the wrong way round. And of the discriminating counts, 20, 50, 60, 90 and
114 remain unrun on either SoC.

v9 is ready. I am holding it to let v8 collect more review rather than resetting
the thread, and I will say so here when it goes.

Regards,
Jiaxing