Re: [PATCH 5.20 1/4] block: add bio_rewind() API

[Mike Snitzer <snitzer@redhat.com>]

On Tue, Jun 28 2022 at  1:52P -0400,
Kent Overstreet <kent.overstreet@gmail.com> wrote:

> On Tue, Jun 28, 2022 at 01:41:12PM -0400, Mike Snitzer wrote:
> > On Tue, Jun 28 2022 at 12:36P -0400,
> > Kent Overstreet <kent.overstreet@gmail.com> wrote:
> > 
> > > On Tue, Jun 28, 2022 at 03:49:28PM +0800, Ming Lei wrote:
> > > > On Tue, Jun 28, 2022 at 12:26:10AM -0400, Kent Overstreet wrote:
> > > > > On Mon, Jun 27, 2022 at 03:36:22PM +0800, Ming Lei wrote:
> > > > > > Not mention bio_iter, bvec_iter has been 32 bytes, which is too big to
> > > > > > hold in per-io data structure. With this patch, 8bytes is enough
> > > > > > to rewind one bio if the end sector is fixed.
> > > > > 
> > > > > And with rewind, you're making an assumption about the state the iterator is
> > > > > going to be in when the IO has completed.
> > > > > 
> > > > > What if the iterator was never advanced?
> > > > 
> > > > bio_rewind() works as expected if the iterator doesn't advance, since bytes
> > > > between the recorded position and the end position isn't changed, same
> > > > with the end position.
> > > > 
> > > > > 
> > > > > So say you check for that by saving some other part of the iterator - but that
> > > > > may have legitimately changed too, if the bio was redirected (bi_sector changes)
> > > > > or trimmed (bi_size changes)
> > > > > 
> > > > > I still think this is an inherently buggy interface, the way it's being proposed
> > > > > to be used.
> > > > 
> > > > The patch did mention that the interface should be for situation in which end
> > > > sector of bio won't change.
> > > 
> > > But that's an assumption that you simply can't make!
> > 
> > Why not?  There is clear use-case for this API, as demonstrated in the
> > patchset: DM can/will make use of it for the purposes of enhancing its
> > more unlikely bio requeuing work (that is needed to make the more
> > likely bio splitting scenario more efficient).
> > 
> > > We allow block device drivers to be stacked in _any_ combination. After a bio is
> > > completed it may have been partially advanced, fully advanced, trimmed, not
> > > trimmed, anything - and bi_sector and thus also bio_end_sector() may have
> > > changed, and will have if there's partition tables involved.
> > 
> > We don't _need_ this API to cure cancer for all hypothetical block
> > drivers.
> > 
> > If consumers of the API follow the rule that end sector of the
> > _original bio_ isn't changed then it is all fine.  It is that simple.
> > 
> > Stacked drivers will work just fine.  The lower layers will be
> > modifying their bios as needed. Because for DM those bios happen to
> > be clones, so there is isolation to the broader design flaw you are
> > trying to say is a major problem. As this patchset demonstrates.
> > 
> > I do concede that policing who can use an API is hard.  But if some
> > consumer of an API tries something that invalidates rules of the API
> > they get to keep the N pieces when it breaks.
> 
> Mike, keep in mind that when bio_rewind() was originally introduced, it
> immediately grew users that were _inherently buggy_ (of the "users can break
> this trivially") variety, and the whole thing had to be reverted, and I was
> really annoyed - mostly at myself, because I would have caught it if I'd been
> paying attention to the mailing list more.
> 
> And I _guarantee_ you that if this makes it in, we'll have the same thing
> happening all over again - we have a lot of different block drivers being
> written by a lot of different people, and most of them do not understand all the
> subtleties of the block layer and the ways different things can interact, and so
> the onus is on us to not add tools that they aren't going to immediately turn
> around and slice themselves with.
> 
> The 32 bytes you're trying to save is meaningless. Think instead about the weeks
> of engineer time that get wasted by bugs like this - chasing the bugs,
> babysitting the patches in to fix it, then the _endless_ -stable backports.

While you or others did something approximating all that, really
doesn't mean it applicable here.

> _Please_ try to think more defensively when you're writing code.

Please try to dial down the hyperbole and judgment. Ming wrote this
code. And you haven't been able to point out anything _actually_ wrong
with it (yet).

This patch's header does need editing for clarity, but we can help
improve it and the documentation above bio_rewind() in the code.

> So, and I'm sorry I have to be the killjoy here, but hard NACK on this patchset.
> Hard, hard NACK.

<insert tom-delonge-wtf.gif>

You see this bio_rewind() as history repeating itself, but it isn't
like what you ranted about in the past:
https://marc.info/?l=linux-block&m=153549921116441&w=2

I can certainly see why you think it similar at first glance. But this
patchset shows how bio_rewind() must be used, and how DM benefits from
using it safely (with no impact to struct bio or DM's per-bio-data).

bio_rewind() usage will be as niche as DM's use-case for it. If other
code respects the documented constraint, that the original bio's end
sector be preserved, then they can use it too.

The key is for a driver to maintain enough state to allow this fixed
end be effectively immutable. (DM happens to get this state "for free"
simply because it was already established for its IO accounting of
split bios).

The Linux codebase requires precision. This isn't new.

> I'll be happy to assist in coming up with alternate, less dangerous solutions
> though (and I think introducing a real bio_iter is overdue, so that's probably
> the first thing we should look at).

It isn't dangerous. It is an interface whose constraint needs to be
respected. Just like is documented for a myriad other kernel
interfaces.

Factoring out a bio_iter will bloat struct bio for functionality most
consumers don't need. And gating DM's ability to achieve this
patchset's functionality with some overdue refactoring is really _not_
acceptable.

Mike