Re: [PATCH] nvmet: add missing locks around nvmet_ns_revalidate

[Niels Dossche <dossche.niels@gmail.com>]

On 10/03/2022 00:12, Bart Van Assche wrote:
> On 3/9/22 14:30, Niels Dossche wrote:
>> On 09/03/2022 23:27, Bart Van Assche wrote:
>>> On 3/9/22 12:34, Niels Dossche wrote:
>>>> nvmet_ns_changed states via lockdep that the ns->subsys->lock
>>>> must be held. The only caller of nvmet_ns_changed which does not
>>>> acquire that lock is nvmet_ns_revalidate. The only 2 callers of
>>>> nvmet_ns_revalidate which do not acquire that lock are
>>>> nvmet_execute_identify_cns_cs_ns and nvmet_execute_identify_ns.
>>>> Add a lock for around the call to nvmet_ns_revalidate in those 2
>>>> functions.
>>>>
>>>> Both of those identify functions are called from a common
>>>> function nvmet_execute_identify, which itself is called
>>>> indirectly via the req->execute function pointer.
>>>
>>> Please mention in the patch description whether this has been
>>> discovered by studying the source code or by software (static
>>> source code analyzer? runtime data race detector?).
>>
>> This was discovered by first using a static analyzer and then
>> verifying it by manual inspection of the source code.
> 
> Hi Niels,
> 
> Are there any plans to make that static analyzer available to other kernel developers?
> 
> Is the static analyzer more powerful than clang thread safety annotations? If it is more powerful, is it possible to integrate the static analyzer in clang?
> 
> See also:
> * "C/C++ Thread Safety Analysis" (https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf).
> * "Thread Safety Annotations for Clang" (https://llvm.org/devmtg/2011-11/Hutchins_ThreadSafety.pdf).
> 
> Thanks,
> 
> Bart.

Hi Bart,

I'm currently developing this static analyzer as part of my master's thesis in order to obtain my master's degree.
The analyzer is currently still a work-in-progress.
I plan on making the source code of this analyzer available when my thesis is finished, which should be sometime in June.

The main focus of the analyzer is not on lockdep assertions actually.
It works roughly in the following way:
1) The analyzer searches for *_lock and *_unlock calls in order to know which fields are locks.
2) It searches wrappers for those lock and unlock calls (e.g. task_lock locks task_struct->alloc_lock)
3) It determines which field accesses of the same struct type occur guarded by a lock (e.g. A->field guarded by A->lock). This is used to (try to) determine which fields need to be locked by which lock.
4) It searches for violations by counting for each field access how many paths are guarded by the lock and how many are not. If the count of unguarded is way smaller than the count of guarded, then it is reported as a possible violation.

The analysis works interprocedurally. It also uses Multi-Layer Type Analysis of K. Lu et al. in order to improve the global call graph with respect to indirect calls.

The aforementioned methodology tries to obtain knowledge about which field requires which lock, without using assertions or annotations. I manually inspect the cases this analyzer outputs in order to determine whether the reported case is a false positive or a true positive. I have submitted patches for some of the true positives over the past few weeks, of which some patches have already been accepted into the mainline or linux-next.

I recently submitted a patch for a case which was missing a lock, but had a lockdep assertions. So that got me wondering whether there are more cases that have a lockdep assertion but don't actually have the expected lock acquired. I added the ability for the analyzer to leverage information about the lockdep assertions. The patch in this email thread was to fix a case that it found. I checked manually too to be extra certain.

The lockdep part of the analyzer is not more powerful than the clang assertions. To be honest, I'm not sure that it can be easily integrated into Clang itself. The analyzer currently uses LLVM bitcode files as an input.

Thanks,
Niels