Re: [PATCH v19 00/40] DEPT(DEPendency Tracker)

[NeilBrown <neilb@ownmail.net>]

On Tue, 25 Aug 2026, Matthew Wilcox wrote:
> On Tue, Aug 25, 2026 at 09:37:34AM +1000, NeilBrown wrote:
> > > For readahead, I think that queues / lru are not involved. We submit the I/O,
> > > and once the I/O is done, we unlock the folio from interrupt context.
> > 
> > "submit the I/O" means "attach the page to a "struct bio" (or similar)
> > and attach the struct bio to a transmit queue for the device (or
> > similar).  So there really is a queue.
> > 
> > > 
> > > end_buffer_async_read() / iomap_finish_folio_read() end up calling
> > > folio_end_read(), where we do the magic
> > > 
> > > 	folio_wake_bit(folio, PG_locked);
> > 
> > Exactly where the lock ownership should be reclaimed is not immediately
> > clear to me.  bio_endio() might be early enough but there are probably
> > better points.
> > 
> > A small difficulty here is that a bio has multiple folios and they are
> > all locked.  I cannot see that lockdep has a concept of holding an
> > arbitrarily large set of related locks.
> > We could just tell lockdep 
> >   "I have some folios locked" 
> > or maybe enhance lockdep to allow
> >   "I have N folios locked"
> > or even
> >   "I have N folios in address-space A with the highest offset being O".
> > 
> > This would allow lockdep to check the validity of locking another folio
> > - only allowed if the address space is the same and the offset is larger
> > than the previous largest.
> 
> It's more complex than that; see my other emails on the subject.

It always is more complex that you think, isn't it...

though I found
 https://lore.kernel.org/all/akvueAxPl8aoLvMR@casper.infradead.org/

and think that you are saying the same thing as me, though I'm probably
not saying it as clearly.
To say what I think you were saying, but with possibly different words:

  We don't want to tell lockdep that we are locking a "folio".  The
  particular folio is irrelevant.  Rather we want to tell lockdep that
  we are locking something at a particular offset of some array of
  folios.

The vfs_lock_two_folios example is much like lock_two_nondirectories().
We use the subclass mechanism to tell lockdep "trust me, this is OK".

> 
> I think we need the ability for lockdep to call a function which says "I
> hold this lock, is that lock OK to acquire".  But the devil is in the
> details.

I doubt we need arbitrary callbacks.  lockdep doesn't try to prove the
locking is correct, it just tries to prove that the information we have
given it is consistent.  So if you say:

  spin_lock(some_lock);
  spin_lock_nested(other_lock, 1);

and later say

  spin_lock(other_lock);
  spin_lock_nested(some_lock, 1);

lockdep will say "fine, that look good", while you or I would say "hang
on, that looks weird".

Similarly for folios we just need to decide what information we want to
give lockdep, and how it should manipulate that information.

I think that locks on ranges of an address-space is the sort of
abstraction that would be most useful.  But if you thought a different
abstraction would be more useful I would certainly be open to that.

As an aside, d_walk() needs to lock a dentry, then lock a child while
holding the parent lock, then drop the original lock and move down the
tree. i.e. it wants an arbitrarily lock sequence of overlapping locks.
lockdep doesn't have any direct way to express this.  It has subclasses
so that it is OK to lock the child while holding a lock on the parent,
but there is a smallish limit to the number of subclasses.
So d_walk() tells lockdep it is dropping the subclass lock on a dentry,
then taking (try_lock) a primary lock on the same dentry.  This is a
"white lie" that leaves lockdep with almost full knowledge but without
confusing it.
This example highlights the idea that lockdep doesn't need to know
exactly what is going on, it just needs to know about the stuff we
want it to check.


Thanks,
NeilBrown