Re: Report 2 in ext4 and journal based on v5.17-rc1

Byungchul Park
2022-03-07T02:44:01Z

On Sat, Mar 05, 2022 at 03:05:23PM +0000, Joel Fernandes wrote:
> On Sat, Mar 05, 2022 at 11:15:38PM +0900, Byungchul Park wrote:
> > Almost all you've been blaming at Dept are totally non-sense. Based on
> > what you're saying, I'm conviced that you don't understand how Dept
> > works even 1%. You don't even try to understand it before blame.
> >
> > You don't have to understand and support it. But I can't response to you
> > if you keep saying silly things that way.
>
> Byungchul, other than ext4 have there been any DEPT reports that other
> subsystem maintainers' agree were valid usecases?

Not yet.

> Regarding false-positives, just to note lockdep is not without its share of
> false-positives. Just that (as you know), the signal-to-noise ratio should be
> high for it to be useful. I've put up with lockdep's false positives just
> because it occasionally saves me from catastrophe.

I love your insight. Agree. A tool would be useful only when it's
*actually* helpful. I hope Dept would be so.

> > > In any case, if DEPT is going to report these "circular dependencies
> > > as bugs that MUST be fixed", it's going to be pure noise and I will
> > > ignore all DEPT reports, and will push back on having Lockdep replaced
> >
> > Dept is going to be improved so that what you are concerning about won't
> > be reported.
>
> Yeah I am looking forward to learning more about it however I was wondering
> about the following: lockdep can already be used for modeling "resource
> acquire/release" and "resource wait" semantics that are unrelated to locks,
> like we do in mm reclaim. I am wondering why we cannot just use those existing
> lockdep mechanisms for the wait/wake usecases (Assuming that we can agree

1. Lockdep can't work with general waits/events happening across
contexts basically. To get over this, manual tagging of
acquire/release can be used at each section that we suspect. But
unfortunately, we cannot use the method if we cannot simply identify
the sections. Furthermore, it's inevitable to miss sections that
shouldn't get missed.

2. Some cases should be correctly tracked via wait/event model, not
acquisition order model. For example, read-lock in rwlock should be
defined as a waiter waiting for write-unlock, write-lock in rwlock
as a waiter waiting for either read-unlock or write-unlock.
Otherwise, if we try to track those cases using acquisition order,
it cannot completely work. Don't you think it looks werid?

3. Tracking what we didn't track before means both stronger detection
and new emergence of false positives, exactly same as Lockdep at its
beginning when it started to track what we hadn't tracked before.
Even though the emergence was allowed at that time, now that Locdkep
got stable enough, folks would be more strict on new emergences. It's
gonna get even worse if valid reports are getting prevented by false
positives.

For that reason, multi reporting functionality is essential. I was
thinking to improve Lockdep to allow multi reporting. But it might be
needed to change more than developing a new tool from scratch. Plus
it might be even more difficult cuz Lockdep already works not badly.
So even for Lockdep, I thought the new thing should be developed
independently leaving Lockdep as it is.

4. (minor reason) The concept and name of acquisition and release is not
for general wait/event. The design and implementation are not,
either. I wanted to address the issue as soon as possible before we
squeeze out Lockdep to use for general wait/event more and the kernel
code gets weird. Of course, it doesn't mean Dept is more stable than
Lockdep. However, I can tell Dept works what a dependency tool should
do and we need to make the code go right.

> that circular dependencies on related to wait/wake is a bad thing. Or perhaps
> there's a reason why Peter Zijlstra did not use lockdep for wait/wake
> dependencies (such as multiple wake sources) considering he wrote a lot of
> that code.
>
> Keep kicking ass brother, you're doing great.

Thank you! I'll go through this in a right way so as not to disappoint
you!

Thanks,
Byungchul