Re: [PATCH] block: Add config option to not allow writing to mounted devices

[Dave Chinner <david@fromorbit.com>]

On Wed, Jun 14, 2023 at 02:27:46PM +0200, Dmitry Vyukov wrote:
> On Wed, 14 Jun 2023 at 04:04, Darrick J. Wong <djwong@kernel.org> wrote:
> > On Tue, Jun 13, 2023 at 08:49:38AM +0200, Dmitry Vyukov wrote:
> > > On Mon, 12 Jun 2023 at 18:16, Jan Kara <jack@suse.cz> wrote:
> > > CONFIG_INSECURE description can say something along the lines of "this
> > > kernel includes subsystems with known bugs that may cause security and
> > > data integrity issues". When a subsystem adds "depends on INSECURE",
> > > the commit should list some of the known issues.
> > >
> > > Then I see how trading disabling things on syzbot in exchange for
> > > "depends on INSECURE" becomes reasonable and satisfies all parties to
> > > some degree.
> >
> > Well in that case, post a patchset adding "depends on INSECURE" for
> > every subsystem that syzbot files bugs against, if the maintainers do
> > not immediately drop what they're doing to resolve the bug.
> 
> Hi Darrick,
> 
> Open unfixed bugs are fine (for some definition of fine).
> What's discussed here is different. It's not having any filed bugs at
> all due to not testing a thing and then not having any visibility into
> the state of things.

Just because syzbot doesn't test something, it does not mean the
code is not tested, nor does it mean the developers who are
responsible for the code have no visibility into the state of their
code.

The reason they want to avoid this sort of corruption injection
testing in syzbot is that it *does not provide a net benefit* to
anyone. The number (and value) of real bugs it might find are vastly
outweighed by the cost of filtering out the many, many false
positives the testing methodology raises.

Keep in mind that syzbot does not provide useful unit and functional
test coverage. We have to run tests suites like fstests to provide
this sort of functionality and visibility into the *correct
operation of the code*.

However, alongside all the unit/functional tests in fstests, we also
have non-deterministic stress and fuzzer tests that are similar in
nature to syzbot. They often flush out weird integration level bugs
before we even get to merging the code. These non-deterministic
stress tests in fstests have found *hundreds* of bugs over the
*couple of decades* we have been running them, and they also have a
history of uncovering entire new classes of bugs we've had to
address.

At this point, syzbot is yet to do prove it is more than a one-trick
pony - it typically only finds a single class of filesystem bug.
That is, it only finds bugs that are related to undetected physical
structure corruption of the filesystem that result in macro level
failures (crash, warn, hang).

Syzbot does nothing to ensure correct behaviour is occuring, that
data integrity is maintained by the filesystem, that crash recovery
after failures works correctly, etc. These things are *by far* the
most important things we have to ensure during filesystem
development.

IOWs, the sorts of problems that syzbot finds in filesystems are way
down the list of important things we need to validate.  Yes,
structural validation testing is something we should be
running, and it's clear that is does get run (both from fstests and
syzbot).

Hence the claim that "because syzbot doesn't run we don't have
visibility of code bugs" is naive, conceited, incredibly
narcissistic and demonstratable false. It also indicates a very
poor understanding of where syzbot actually fits into the overall
engineering processes.

-Dave.
-- 
Dave Chinner
david@fromorbit.com