public inbox for netdev@vger.kernel.org
 help / color / mirror / Atom feed
From: Peter Oberparleiter <oberpar@linux.ibm.com>
To: Nathan Chancellor <nathan@kernel.org>,
	Konstantin Khorenko <khorenko@virtuozzo.com>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: "Mikhail Zaslonko" <zaslonko@linux.ibm.com>,
	"Nicolas Schier" <nsc@kernel.org>,
	"Masahiro Yamada" <masahiroy@kernel.org>,
	"Thomas Weißschuh" <linux@weissschuh.net>,
	"Arnd Bergmann" <arnd@arndb.de>,
	"Steffen Klassert" <steffen.klassert@secunet.com>,
	"Herbert Xu" <herbert@gondor.apana.org.au>,
	linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
	netdev@vger.kernel.org,
	"Pavel Tikhomirov" <ptikhomirov@virtuozzo.com>,
	"Vasileios Almpanis" <vasileios.almpanis@virtuozzo.com>,
	"Jakub Kicinski" <kuba@kernel.org>
Subject: Re: [PATCH] gcov: use atomic counter updates to fix concurrent access crashes
Date: Thu, 9 Apr 2026 10:11:24 +0200	[thread overview]
Message-ID: <9fba075d-9388-483f-818e-6ee3b168f18d@linux.ibm.com> (raw)
In-Reply-To: <20260406193707.GB1319599@ax162>

On 06.04.2026 21:37, Nathan Chancellor wrote:
> On Thu, Apr 02, 2026 at 05:18:31PM +0300, Konstantin Khorenko wrote:
>> GCC's GCOV instrumentation can merge global branch counters with loop
>> induction variables as an optimization.  In inflate_fast(), the inner
>> copy loops get transformed so that the GCOV counter value is loaded
>> multiple times to compute the loop base address, start index, and end
>> bound.  Since GCOV counters are global (not per-CPU), concurrent
>> execution on different CPUs causes the counter to change between loads,
>> producing inconsistent values and out-of-bounds memory writes.
>>
>> The crash manifests during IPComp (IP Payload Compression) processing
>> when inflate_fast() runs concurrently on multiple CPUs:
>>
>>   BUG: unable to handle page fault for address: ffffd0a3c0902ffa
>>   RIP: inflate_fast+1431
>>   Call Trace:
>>    zlib_inflate
>>    __deflate_decompress
>>    crypto_comp_decompress
>>    ipcomp_decompress [xfrm_ipcomp]
>>    ipcomp_input [xfrm_ipcomp]
>>    xfrm_input
>>
>> At the crash point, the compiler generated three loads from the same
>> global GCOV counter (__gcov0.inflate_fast+216) to compute base, start,
>> and end for an indexed loop.  Another CPU modified the counter between
>> loads, making the values inconsistent — the write went 3.4 MB past a
>> 65 KB buffer.
>>
>> Add -fprofile-update=atomic to CFLAGS_GCOV at the global level in the
>> top-level Makefile.  This tells GCC that GCOV counters may be
>> concurrently accessed, causing counter updates to use atomic
>> instructions (lock addq) instead of plain load/store.  This prevents
>> the compiler from merging counters with loop induction variables.
>>
>> Applying this globally rather than per-subsystem not only addresses the
>> observed crash in zlib but makes GCOV coverage data more consistent
>> overall, preventing similar issues in any kernel code path that may
>> execute concurrently.
>>
>> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
>> Tested-by: Peter Oberparleiter <oberpar@linux.ibm.com>
>> Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
> 
> While this is obviously a fix, what are the chances of regressions from
> this change? As this should only impact GCOV, this could go via whatever
> tree carries GCOV patches. If Kbuild is to take this change, my vote
> would be to defer it to 7.2 at this point in the development cycle so
> that it can have most of a cycle to sit in -next.

Adding Andrew since he typically integrates GCOV patches via his tree,
and for input on how to handle this patch.

To summarize the situation, this patch:
- is only effective with GCC + GCOV profiling enabled
- fixes a run-time crash
- improves overall GCOV coverage data consistency
- triggers a number of build errors due to side-effects on GCC constant
  folding and therefore depends on the associated series [1] that fixes
  these build-errors
- has a non-zero chance to trigger additional build-time errors, e.g.
  in similar macros guarded by arch/config symbols not covered by
  current testing

Given the last point, I agree with Nathan that this patch would benefit
from additional test coverage to minimize regression risks, e.g. via a
cycle in -next.

[1]
https://lore.kernel.org/lkml/20260402140558.1437002-1-khorenko@virtuozzo.com/

>> ---
>>  Makefile | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 6b1d9fb1a6b4..a55ad668d6ba 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -806,7 +806,7 @@ all: vmlinux
>>  
>>  CFLAGS_GCOV	:= -fprofile-arcs -ftest-coverage
>>  ifdef CONFIG_CC_IS_GCC
>> -CFLAGS_GCOV	+= -fno-tree-loop-im
>> +CFLAGS_GCOV	+= -fno-tree-loop-im -fprofile-update=atomic
>>  endif
>>  export CFLAGS_GCOV
>>  
>> -- 
>> 2.43.5
>>


-- 
Peter Oberparleiter
Linux on IBM Z Development - IBM Germany R&D

      reply	other threads:[~2026-04-09  8:12 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-02 14:18 [PATCH 0/1] gcov: add -fprofile-update=atomic to fix concurrent access crashes Konstantin Khorenko
2026-04-02 14:18 ` [PATCH] gcov: use atomic counter updates " Konstantin Khorenko
2026-04-06 19:37   ` Nathan Chancellor
2026-04-09  8:11     ` Peter Oberparleiter [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9fba075d-9388-483f-818e-6ee3b168f18d@linux.ibm.com \
    --to=oberpar@linux.ibm.com \
    --cc=akpm@linux-foundation.org \
    --cc=arnd@arndb.de \
    --cc=herbert@gondor.apana.org.au \
    --cc=khorenko@virtuozzo.com \
    --cc=kuba@kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@weissschuh.net \
    --cc=masahiroy@kernel.org \
    --cc=nathan@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=nsc@kernel.org \
    --cc=ptikhomirov@virtuozzo.com \
    --cc=steffen.klassert@secunet.com \
    --cc=vasileios.almpanis@virtuozzo.com \
    --cc=zaslonko@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox