Re: [PATCH v13 03/14] accel/rocket: wait for a running IRQ handler before resetting a core

[Igor Paunovic <royalnet026@gmail.com>]

Hi Jiaxing,

On Sat, Sep 19, 2026 at 09:17:25PM +1200, Jiaxing Hu wrote:
> I read the tally as 8 of the 126 scored inferences missing on all 48
> channels across the three runs, and 5 of those, in the two runs you
> traced, sitting on the 5 -ECANCELED completions.

Nested that way, yes. One correction: six runs, three per arm. Five
of the eight zeros are on 2+3 and three on 2+3+4. In the two traced
runs the -ECANCELED completions sit on exactly the inferences that
scored 0/48 there: rounds 1, 4, 8 and the one after suspend/resume
on 2+3, round 7 on 2+3+4. The other three zeros are in runs without
the kprobe.

After that mail, on the evening of 16 September, I reached the other
path. A 1x1 convolution gets there when the input, not the weights,
overflows the CBUF: 80x80x64 in, 48 out, so rkt_split_tasks() splits
on input rows and rkt_ml_subgraph_invoke() submits one job with
task_count 2 (rows 0-63 and 64-79). A graph whose weights do not fit
goes the other way: reuse_weights_cbuf is false and each task becomes
its own one-task job, back on the scheduler thread.

Same board and config (JOB_TIMEOUT_MS=2, PROVE_LOCKING,
DEBUG_ATOMIC_SLEEP), the induced-reset protocol at 60 inferences a
run. On 2+3: five runs, 310 inferences, 180 timeouts, one of the runs
with a 32-output variant that splits the same way. On 2+3+4: three
runs, 186 inferences, 181 timeouts. In the three traced runs on 2+3,
24 of 110 timeouts cut into a job after the IRQ thread had submitted
its next task (9, 9 and 6); in the one traced run on 2+3+4, 11 of
61. No lockdep report, warning or MMU fault in either boot, and
debug_locks 1 at the end of every run.

So the lock scope ran with the IRQ thread submitting tasks, but this
does not show the race is closed: the window in hw_submit() is
microseconds, one reset began while the IRQ thread was in it (in the
32-output run), and waiting on job_lock itself was not instrumented.

One result bears on 4/14. Without it the core stayed active after
every cancelled job (113 of 113) and no cancelled job was followed by
another (0 of 109). With it the core suspended after each (122 of
122) and 84 of 120 cancelled jobs were followed by another, the next
job waiting for resume against the same 2 ms. That is the timeout
feeding itself, not a defect, and it is why 181 of 186 inferences
timed out on that arm against 180 of 310. Core 0, one client. If
4/14 changes shape in v14 I will re-run that arm, with the two-task
job in it.

Scripts, counting and this draft were prepared with an LLM assistant;
I ran the tests, and every number came from the raw files of each run.

Regards,
Igor