[PATCH v3 0/2] arm64: Fix kcsan test_barrier fail and panic

[PATCH v3 0/2] arm64: Fix kcsan test_barrier fail and panic

[Kefeng Wang]

Fix selftest and kcsan_test() module fail when KCSAN_STRICT
and KCSAN_WEAK_MEMORY enabled on ARM64.

v3:
- update dma_mb()'s description and add the generic definition,
  also asm-generic change is moved into patch1, suggested by Marco.
v2:
- Add documents about dma_mb(), suggested by Mike and Will.
- drop Fixes tag and update changlog, suggested by Mike.

Kefeng Wang (2):
  asm-generic: Add memory barrier dma_mb()
  arm64: kcsan: Support detecting more missing memory barriers

 Documentation/memory-barriers.txt | 11 ++++++-----
 arch/arm64/include/asm/barrier.h  | 12 ++++++------
 include/asm-generic/barrier.h     |  8 ++++++++
 3 files changed, 20 insertions(+), 11 deletions(-)

-- 
2.35.3

[PATCH v3 1/2] asm-generic: Add memory barrier dma_mb()

[Kefeng Wang]

The memory barrier dma_mb() is introduced by commit a76a37777f2c
("iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer"),
which is used to ensure that prior (both reads and writes) accesses
to memory by a CPU are ordered w.r.t. a subsequent MMIO write, this
is only defined on arm64, but it is a generic memory barrier, let's
add dma_mb() into documentation and include/asm-generic/barrier.h.

Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
 Documentation/memory-barriers.txt | 11 ++++++-----
 include/asm-generic/barrier.h     |  8 ++++++++
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
index b12df9137e1c..07a8b8e1b12a 100644
--- a/Documentation/memory-barriers.txt
+++ b/Documentation/memory-barriers.txt
@@ -1894,6 +1894,7 @@ There are some more advanced barrier functions:
 
  (*) dma_wmb();
  (*) dma_rmb();
+ (*) dma_mb();
 
      These are for use with consistent memory to guarantee the ordering
      of writes or reads of shared memory accessible to both the CPU and a
@@ -1925,11 +1926,11 @@ There are some more advanced barrier functions:
      The dma_rmb() allows us guarantee the device has released ownership
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
-     can see it now has ownership.  Note that, when using writel(), a prior
-     wmb() is not needed to guarantee that the cache coherent memory writes
-     have completed before writing to the MMIO region.  The cheaper
-     writel_relaxed() does not provide this guarantee and must not be used
-     here.
+     can see it now has ownership. The dma_mb() implies both a dma_rmb() and
+     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
+     to guarantee that the cache coherent memory writes have completed before
+     writing to the MMIO region. The cheaper writel_relaxed() does not provide
+     this guarantee and must not be used here.
 
      See the subsection "Kernel I/O barrier effects" for more information on
      relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index fd7e8fbaeef1..961f4d88f9ef 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -38,6 +38,10 @@
 #define wmb()	do { kcsan_wmb(); __wmb(); } while (0)
 #endif
 
+#ifdef __dma_mb
+#define dma_mb()	do { kcsan_mb(); __dma_mb(); } while (0)
+#endif
+
 #ifdef __dma_rmb
 #define dma_rmb()	do { kcsan_rmb(); __dma_rmb(); } while (0)
 #endif
@@ -65,6 +69,10 @@
 #define wmb()	mb()
 #endif
 
+#ifndef dma_mb
+#define dma_mb()	mb()
+#endif
+
 #ifndef dma_rmb
 #define dma_rmb()	rmb()
 #endif
-- 
2.35.3

Re: [PATCH v3 1/2] asm-generic: Add memory barrier dma_mb()

[Marco Elver]

On Mon, 23 May 2022 at 03:50, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
> The memory barrier dma_mb() is introduced by commit a76a37777f2c
> ("iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer"),
> which is used to ensure that prior (both reads and writes) accesses
> to memory by a CPU are ordered w.r.t. a subsequent MMIO write, this
> is only defined on arm64, but it is a generic memory barrier, let's
> add dma_mb() into documentation and include/asm-generic/barrier.h.
>
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> ---
>  Documentation/memory-barriers.txt | 11 ++++++-----
>  include/asm-generic/barrier.h     |  8 ++++++++
>  2 files changed, 14 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
> index b12df9137e1c..07a8b8e1b12a 100644
> --- a/Documentation/memory-barriers.txt
> +++ b/Documentation/memory-barriers.txt
> @@ -1894,6 +1894,7 @@ There are some more advanced barrier functions:
>
>   (*) dma_wmb();
>   (*) dma_rmb();
> + (*) dma_mb();
>
>       These are for use with consistent memory to guarantee the ordering
>       of writes or reads of shared memory accessible to both the CPU and a
> @@ -1925,11 +1926,11 @@ There are some more advanced barrier functions:
>       The dma_rmb() allows us guarantee the device has released ownership
>       before we read the data from the descriptor, and the dma_wmb() allows
>       us to guarantee the data is written to the descriptor before the device
> -     can see it now has ownership.  Note that, when using writel(), a prior
> -     wmb() is not needed to guarantee that the cache coherent memory writes
> -     have completed before writing to the MMIO region.  The cheaper
> -     writel_relaxed() does not provide this guarantee and must not be used
> -     here.
> +     can see it now has ownership. The dma_mb() implies both a dma_rmb() and
> +     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
> +     to guarantee that the cache coherent memory writes have completed before
> +     writing to the MMIO region. The cheaper writel_relaxed() does not provide
> +     this guarantee and must not be used here.

It seems you've changed that spacing. This document uses 2 spaces
after a sentence-ending '.'. (My original suggestion included the 2
spaces after dots.)

Otherwise it all looks fine to me.

Thanks,
-- Marco

Re: [PATCH v3 1/2] asm-generic: Add memory barrier dma_mb()

[Kefeng Wang]

On 2022/5/23 16:22, Marco Elver wrote:
> On Mon, 23 May 2022 at 03:50, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>> The memory barrier dma_mb() is introduced by commit a76a37777f2c
>> ("iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer"),
>> which is used to ensure that prior (both reads and writes) accesses
>> to memory by a CPU are ordered w.r.t. a subsequent MMIO write, this
>> is only defined on arm64, but it is a generic memory barrier, let's
>> add dma_mb() into documentation and include/asm-generic/barrier.h.
>>
>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>> ---
>>   Documentation/memory-barriers.txt | 11 ++++++-----
>>   include/asm-generic/barrier.h     |  8 ++++++++
>>   2 files changed, 14 insertions(+), 5 deletions(-)
>>
>> diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
>> index b12df9137e1c..07a8b8e1b12a 100644
>> --- a/Documentation/memory-barriers.txt
>> +++ b/Documentation/memory-barriers.txt
>> @@ -1894,6 +1894,7 @@ There are some more advanced barrier functions:
>>
>>    (*) dma_wmb();
>>    (*) dma_rmb();
>> + (*) dma_mb();
>>
>>        These are for use with consistent memory to guarantee the ordering
>>        of writes or reads of shared memory accessible to both the CPU and a
>> @@ -1925,11 +1926,11 @@ There are some more advanced barrier functions:
>>        The dma_rmb() allows us guarantee the device has released ownership
>>        before we read the data from the descriptor, and the dma_wmb() allows
>>        us to guarantee the data is written to the descriptor before the device
>> -     can see it now has ownership.  Note that, when using writel(), a prior
>> -     wmb() is not needed to guarantee that the cache coherent memory writes
>> -     have completed before writing to the MMIO region.  The cheaper
>> -     writel_relaxed() does not provide this guarantee and must not be used
>> -     here.
>> +     can see it now has ownership. The dma_mb() implies both a dma_rmb() and
>> +     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
>> +     to guarantee that the cache coherent memory writes have completed before
>> +     writing to the MMIO region. The cheaper writel_relaxed() does not provide
>> +     this guarantee and must not be used here.
> It seems you've changed that spacing. This document uses 2 spaces
> after a sentence-ending '.'. (My original suggestion included the 2
> spaces after dots.)

I don't know the rules, it seems that some uses 1 spaces, others are 2 
spaces, but most

uses 2 spaces, will update.

> Otherwise it all looks fine to me.
>
> Thanks,
> -- Marco
> .

Re: [PATCH v3 1/2] asm-generic: Add memory barrier dma_mb()

[Arnd Bergmann]

On Mon, May 23, 2022 at 4:00 AM Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
> The memory barrier dma_mb() is introduced by commit a76a37777f2c
> ("iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer"),
> which is used to ensure that prior (both reads and writes) accesses
> to memory by a CPU are ordered w.r.t. a subsequent MMIO write, this
> is only defined on arm64, but it is a generic memory barrier, let's
> add dma_mb() into documentation and include/asm-generic/barrier.h.
>
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> ---

I assume you plan this to get merged through the arm64 tree.

Reviewed-by: Arnd Bergmann <arnd@arndb.de> # for asm-generic

[PATCH v3 2/2] arm64: kcsan: Support detecting more missing memory barriers

[Kefeng Wang]

As "kcsan: Support detecting a subset of missing memory barriers"[1]
introduced KCSAN_STRICT/KCSAN_WEAK_MEMORY which make kcsan detects
more missing memory barrier, but arm64 don't have KCSAN instrumentation
for barriers, so the new selftest test_barrier() and test cases for
memory barrier instrumentation in kcsan_test module will fail, even
panic on selftest.

Let's prefix all barriers with __ on arm64, as asm-generic/barriers.h
defined the final instrumented version of these barriers, which will
fix the above issues.

Note, barrier instrumentation can be disabled via __no_kcsan with
appropriate compiler-support (and not just with objtool help), see
commit bd3d5bd1a0ad ("kcsan: Support WEAK_MEMORY with Clang where
no objtool support exists"), it adds disable_sanitizer_instrumentation
to __no_kcsan attribute for Clang which will remove all sanitizer
instrumentation fully (with Clang 14.0). Meanwhile, GCC does the
same thing with no_sanitize.

[1] https://lore.kernel.org/linux-mm/20211130114433.2580590-1-elver@google.com/
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
 arch/arm64/include/asm/barrier.h | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
index 9f3e2c3d2ca0..2cfc4245d2e2 100644
--- a/arch/arm64/include/asm/barrier.h
+++ b/arch/arm64/include/asm/barrier.h
@@ -50,13 +50,13 @@
 #define pmr_sync()	do {} while (0)
 #endif
 
-#define mb()		dsb(sy)
-#define rmb()		dsb(ld)
-#define wmb()		dsb(st)
+#define __mb()		dsb(sy)
+#define __rmb()		dsb(ld)
+#define __wmb()		dsb(st)
 
-#define dma_mb()	dmb(osh)
-#define dma_rmb()	dmb(oshld)
-#define dma_wmb()	dmb(oshst)
+#define __dma_mb()	dmb(osh)
+#define __dma_rmb()	dmb(oshld)
+#define __dma_wmb()	dmb(oshst)
 
 #define io_stop_wc()	dgh()
 
-- 
2.35.3

Re: [PATCH v3 2/2] arm64: kcsan: Support detecting more missing memory barriers

[Marco Elver]

On Mon, 23 May 2022 at 03:50, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
> As "kcsan: Support detecting a subset of missing memory barriers"[1]
> introduced KCSAN_STRICT/KCSAN_WEAK_MEMORY which make kcsan detects
> more missing memory barrier, but arm64 don't have KCSAN instrumentation
> for barriers, so the new selftest test_barrier() and test cases for
> memory barrier instrumentation in kcsan_test module will fail, even
> panic on selftest.
>
> Let's prefix all barriers with __ on arm64, as asm-generic/barriers.h
> defined the final instrumented version of these barriers, which will
> fix the above issues.
>
> Note, barrier instrumentation can be disabled via __no_kcsan with
> appropriate compiler-support (and not just with objtool help), see
> commit bd3d5bd1a0ad ("kcsan: Support WEAK_MEMORY with Clang where
> no objtool support exists"), it adds disable_sanitizer_instrumentation
> to __no_kcsan attribute for Clang which will remove all sanitizer
> instrumentation fully (with Clang 14.0). Meanwhile, GCC does the
> same thing with no_sanitize.
>
> [1] https://lore.kernel.org/linux-mm/20211130114433.2580590-1-elver@google.com/
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>

Acked-by: Marco Elver <elver@google.com>

> ---
>  arch/arm64/include/asm/barrier.h | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
> index 9f3e2c3d2ca0..2cfc4245d2e2 100644
> --- a/arch/arm64/include/asm/barrier.h
> +++ b/arch/arm64/include/asm/barrier.h
> @@ -50,13 +50,13 @@
>  #define pmr_sync()     do {} while (0)
>  #endif
>
> -#define mb()           dsb(sy)
> -#define rmb()          dsb(ld)
> -#define wmb()          dsb(st)
> +#define __mb()         dsb(sy)
> +#define __rmb()                dsb(ld)
> +#define __wmb()                dsb(st)
>
> -#define dma_mb()       dmb(osh)
> -#define dma_rmb()      dmb(oshld)
> -#define dma_wmb()      dmb(oshst)
> +#define __dma_mb()     dmb(osh)
> +#define __dma_rmb()    dmb(oshld)
> +#define __dma_wmb()    dmb(oshst)
>
>  #define io_stop_wc()   dgh()
>
> --
> 2.35.3
>