Re: [RFC PATCH net-next 00/10] Use robust notifiers in DSA

[Vladimir Oltean <olteanv@gmail.com>]

On Thu, Aug 18, 2022 at 11:49:24PM +0200, Andrew Lunn wrote:
> I would split it into two classes of errors:
> 
> Bus transactions fail. This very likely means the hardware design is
> bad, connectors are loose, etc. There is not much we can do about
> this, bad things are going to happen no what.
> 
> We have consumed all of some sort of resource. Out of memory, the ATU
> is full, too many LAGs, etc. We try to roll back in order to get out
> of this resource problem.
> 
> So i would say -EIO, -ETIMEDOUT, we don't care about too
> much. -ENOMEM, -ENOBUF, -EOPNOTSUPP or whatever, we should try to do a
> robust rollback.
> 
> The original design of switchdev was two phase:
> 
> 1) Allocate whatever resources are needed, can fail
> 2) Put those resources into use, must not fail
> 
> At some point that all got thrown away.

So you think that rollback at the cross-chip notifier layer is a new
problem we need to tackle, because we don't have enough transactional
layering in the code?

In case you don't remember how that utopia dug itself into a hole in practice:
nobody (not even DSA) used the switchdev transactional item queue (which
passed allocated resources between the prepare and the commit phase)
from its introduction in 2015 until it was deleted in 2019, and then
drivers were left unable to reclaim the memory they allocated during
preparation, if the code path never came to the commit stage. There was
nothing left to do except to throw it away.

To discover whether the ATU is full, you either need to reserve space
for static entries beforehand, which is inefficient, or just try to add
what you want and see if you could. Which inevitably leads to encouraging
the strategy of doing the work in the preparation phase and nothing in
the commit phase.

"Too many X" where the resource limitation is known beforehand is about
the only case where a prepare/commit phase could avoid useless rollback.
It's also a case which could also be solved by being upfront about the
limitation to your higher layer, then it would not even try at all.
And do note that "less useless rollback" is different than "code gives
more guarantees that system will remain in a known state".

Sadly reality is much more dynamic than "allocate" -> can fail / "use" ->
must not fail. I think when the model fails to describe reality, you
change the model, not reality.