[PATCH 1/4] remoteproc: core: Reset freed vring entries to FW_RSC_ADDR_ANY

[Yonghao Zhang <hyz3367@gmail.com>]

rproc_free_vring() resets a vring entry in the resource table as it
releases it.  Resetting da to 0 leaves the entry looking like a vring
at device address 0; write FW_RSC_ADDR_ANY instead, the marker for an
unallocated address, which is what the entry means from then on.

That matters on the error paths of rproc_start() and rproc_attach(),
where rproc_free_vring() runs while the remote processor is already
running: the failure originates in rp_find_vq(), called through
virtio_find_vqs() from e.g. rpmsg_probe(), which the core only
reaches through rproc_start_subdevices(), after ops->start() or
ops->attach() have completed.  At that point table_ptr is the
resource table installed in remote processor memory, and whatever
the reset writes is visible to the remote side.  Whether and when a
running processor reads the entry cannot be known, so the value must
be safe for one: 0 is a plausible device address, FW_RSC_ADDR_ANY is
not.  On the teardown paths, stop and detach, the write lands in the
cached table or in a copy of the installed one and reaches no one.

The comment replaced along with it was written for a call graph that
no longer exists.  It described the teardown callers only:
rproc_stop() has run, table_ptr points at the cached table, and that
table is NULL for a processor started by another entity, so there is
nothing to clear.  Two facts have overtaken it:

- rp_find_vq() has been calling rproc_free_vring() when
  vring_new_virtqueue() fails since commit 6db20ea8d850 ("remoteproc:
  allocate vrings on demand, free when not needed"), and that call
  was present, unchanged, when commit 9dc9507f1880 ("remoteproc:
  Properly deal with the resource table when detaching") landed: it
  runs from rproc_start_subdevices(), after ops->start() or
  ops->attach() have completed, with table_ptr at the table installed
  in remote processor memory and the processor running.  The comment
  never described this caller, not on the day it was written.

- commit 9dc9507f1880 ("remoteproc: Properly deal with the resource
  table when detaching") and commit 8088dd4d9316 ("remoteproc: Properly
  deal with the resource table when stopping") later taught the
  detach/stop paths to take a kmemdup() copy of the installed table for
  processors started by another entity, so the NULL case the guard was
  deciding between is gone: table_ptr is always valid where the teardown
  callers run.  (Those callers have since moved to the rproc-virtio
  platform driver, 9c31255ce5fe.)

Fixes: c0d631570ad5 ("remoteproc: set vring addresses in resource table")
Fixes: 4d3ebb3b9990 ("remoteproc: Refactor function rproc_free_vring()")
Signed-off-by: Yonghao Zhang <hyz3367@gmail.com>
---
 drivers/remoteproc/remoteproc_core.c | 40 +++++++++++++++++++---------
 1 file changed, 28 insertions(+), 12 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 263e12f022ea..123aadb467a0 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -403,6 +403,33 @@ rproc_parse_vring(struct rproc_vdev *rvdev, struct fw_rsc_vdev *rsc, int i)
 	return 0;
 }
 
+/*
+ * rproc_free_vring() runs while a remote processor is being torn down
+ * (rproc_stop()/rproc_detach() paths) or while a boot or attach attempt
+ * is failing.  Whether the reset below can reach the remote processor
+ * depends on what rproc->table_ptr refers to at that point:
+ *
+ * - teardown paths: the call comes from the rvdev cleanup in
+ *   rproc_resource_cleanup(), by which time rproc_stop()/__rproc_detach()
+ *   have switched table_ptr to the table the core was booted with, or
+ *   to a copy of the installed table when the remote processor was
+ *   started by another entity (rproc_reset_rsc_table_on_{stop,detach}()),
+ *   so the write cannot reach the remote processor.
+ *
+ * - error paths of rproc_start()/rproc_attach(): the failure originates
+ *   in rp_find_vq() (virtio_find_vqs() failing in e.g. rpmsg_probe()),
+ *   which rproc_start_subdevices() calls only once ops->start() or
+ *   ops->attach() have completed.  table_ptr still points at the
+ *   resource table installed in remote processor memory and the remote
+ *   processor is already running.
+ *
+ * Especially in the last case the reset has to tell the remote
+ * processor that the vring entry is not usable: with da set to the
+ * invalid FW_RSC_ADDR_ANY a running remote processor sees the address
+ * as unallocated, and notifyid is invalidated along with it.
+ *
+ * Reset the virtio device section only if there is a table to work with.
+ */
 void rproc_free_vring(struct rproc_vring *rvring)
 {
 	struct rproc *rproc = rvring->rvdev->rproc;
@@ -411,20 +438,9 @@ void rproc_free_vring(struct rproc_vring *rvring)
 
 	idr_remove(&rproc->notifyids, rvring->notifyid);
 
-	/*
-	 * At this point rproc_stop() has been called and the installed resource
-	 * table in the remote processor memory may no longer be accessible. As
-	 * such and as per rproc_stop(), rproc->table_ptr points to the cached
-	 * resource table (rproc->cached_table).  The cached resource table is
-	 * only available when a remote processor has been booted by the
-	 * remoteproc core, otherwise it is NULL.
-	 *
-	 * Based on the above, reset the virtio device section in the cached
-	 * resource table only if there is one to work with.
-	 */
 	if (rproc->table_ptr) {
 		rsc = (void *)rproc->table_ptr + rvring->rvdev->rsc_offset;
-		rsc->vring[idx].da = 0;
+		rsc->vring[idx].da = FW_RSC_ADDR_ANY;
 		rsc->vring[idx].notifyid = -1;
 	}
 }
-- 
2.34.1