Re: xarray, fault injection and syzkaller

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Nov 03, 2022 at 08:00:25PM +0000, Matthew Wilcox wrote:
> On Thu, Nov 03, 2022 at 04:09:04PM -0300, Jason Gunthorpe wrote:
> > Hi All,
> > 
> > I wonder if anyone has some thoughts on this - I have spent some time
> > setting up syzkaller for a new subsystem and I've noticed that nth
> > fault injection does not reliably cause things like xa_store() to
> > fail.
> > 
> > It seems the basic reason is that xarray will usually do two
> > allocations, one in an atomic context which fault injection does
> > reliably fail, but then it almost always follows up with a second
> > allocation in a non-atomic context that doesn't fail because nth has
> > become 0.
> 
> Hahaha.  I didn't intentionally set out to thwart memory allocation
> fault injection.  Realistically, do we want it to fail though?
> GFP_KERNEL allocations of small sizes are supposed to never fail.
> (for those not aware, node allocations are 576 bytes; typically the slab
> allocator bundles 28 of them into an order-2 allocation).

I don't know, I have code to handle these failures, I want to test it
:)

> I think a simple solution if we really do want to make allocations fail
> is to switch error injection from "fail one allocation per N" to "fail
> M allocations per N".  eg, 7 allocations succeed, 3 allocations fail,
> 7 succeed, 3 fail, ...  It's more realistic because you do tend to see
> memory allocation failures come in bursts.

The systemic testing I've setup just walks nth through the entire
range until it no longer triggers. This hits every injection point and
checks the failure path of it. This is also what syzkaller does
automatically from what I can tell

If we make it probabilistic it is harder to reliably trigger these
fault points.

Jason