From: Peter Oberparleiter <oberpar@linux.ibm.com>
To: Nathan Chancellor <nathan@kernel.org>,
Konstantin Khorenko <khorenko@virtuozzo.com>,
Andrew Morton <akpm@linux-foundation.org>
Cc: "Mikhail Zaslonko" <zaslonko@linux.ibm.com>,
"Nicolas Schier" <nsc@kernel.org>,
"Masahiro Yamada" <masahiroy@kernel.org>,
"Thomas Weißschuh" <linux@weissschuh.net>,
"Arnd Bergmann" <arnd@arndb.de>,
"Steffen Klassert" <steffen.klassert@secunet.com>,
"Herbert Xu" <herbert@gondor.apana.org.au>,
linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org,
"Pavel Tikhomirov" <ptikhomirov@virtuozzo.com>,
"Vasileios Almpanis" <vasileios.almpanis@virtuozzo.com>,
"Jakub Kicinski" <kuba@kernel.org>
Subject: Re: [PATCH] gcov: use atomic counter updates to fix concurrent access crashes
Date: Thu, 9 Apr 2026 10:11:24 +0200 [thread overview]
Message-ID: <9fba075d-9388-483f-818e-6ee3b168f18d@linux.ibm.com> (raw)
In-Reply-To: <20260406193707.GB1319599@ax162>
On 06.04.2026 21:37, Nathan Chancellor wrote:
> On Thu, Apr 02, 2026 at 05:18:31PM +0300, Konstantin Khorenko wrote:
>> GCC's GCOV instrumentation can merge global branch counters with loop
>> induction variables as an optimization. In inflate_fast(), the inner
>> copy loops get transformed so that the GCOV counter value is loaded
>> multiple times to compute the loop base address, start index, and end
>> bound. Since GCOV counters are global (not per-CPU), concurrent
>> execution on different CPUs causes the counter to change between loads,
>> producing inconsistent values and out-of-bounds memory writes.
>>
>> The crash manifests during IPComp (IP Payload Compression) processing
>> when inflate_fast() runs concurrently on multiple CPUs:
>>
>> BUG: unable to handle page fault for address: ffffd0a3c0902ffa
>> RIP: inflate_fast+1431
>> Call Trace:
>> zlib_inflate
>> __deflate_decompress
>> crypto_comp_decompress
>> ipcomp_decompress [xfrm_ipcomp]
>> ipcomp_input [xfrm_ipcomp]
>> xfrm_input
>>
>> At the crash point, the compiler generated three loads from the same
>> global GCOV counter (__gcov0.inflate_fast+216) to compute base, start,
>> and end for an indexed loop. Another CPU modified the counter between
>> loads, making the values inconsistent — the write went 3.4 MB past a
>> 65 KB buffer.
>>
>> Add -fprofile-update=atomic to CFLAGS_GCOV at the global level in the
>> top-level Makefile. This tells GCC that GCOV counters may be
>> concurrently accessed, causing counter updates to use atomic
>> instructions (lock addq) instead of plain load/store. This prevents
>> the compiler from merging counters with loop induction variables.
>>
>> Applying this globally rather than per-subsystem not only addresses the
>> observed crash in zlib but makes GCOV coverage data more consistent
>> overall, preventing similar issues in any kernel code path that may
>> execute concurrently.
>>
>> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
>> Tested-by: Peter Oberparleiter <oberpar@linux.ibm.com>
>> Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
>
> While this is obviously a fix, what are the chances of regressions from
> this change? As this should only impact GCOV, this could go via whatever
> tree carries GCOV patches. If Kbuild is to take this change, my vote
> would be to defer it to 7.2 at this point in the development cycle so
> that it can have most of a cycle to sit in -next.
Adding Andrew since he typically integrates GCOV patches via his tree,
and for input on how to handle this patch.
To summarize the situation, this patch:
- is only effective with GCC + GCOV profiling enabled
- fixes a run-time crash
- improves overall GCOV coverage data consistency
- triggers a number of build errors due to side-effects on GCC constant
folding and therefore depends on the associated series [1] that fixes
these build-errors
- has a non-zero chance to trigger additional build-time errors, e.g.
in similar macros guarded by arch/config symbols not covered by
current testing
Given the last point, I agree with Nathan that this patch would benefit
from additional test coverage to minimize regression risks, e.g. via a
cycle in -next.
[1]
https://lore.kernel.org/lkml/20260402140558.1437002-1-khorenko@virtuozzo.com/
>> ---
>> Makefile | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 6b1d9fb1a6b4..a55ad668d6ba 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -806,7 +806,7 @@ all: vmlinux
>>
>> CFLAGS_GCOV := -fprofile-arcs -ftest-coverage
>> ifdef CONFIG_CC_IS_GCC
>> -CFLAGS_GCOV += -fno-tree-loop-im
>> +CFLAGS_GCOV += -fno-tree-loop-im -fprofile-update=atomic
>> endif
>> export CFLAGS_GCOV
>>
>> --
>> 2.43.5
>>
--
Peter Oberparleiter
Linux on IBM Z Development - IBM Germany R&D
prev parent reply other threads:[~2026-04-09 8:12 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-02 14:18 [PATCH 0/1] gcov: add -fprofile-update=atomic to fix concurrent access crashes Konstantin Khorenko
2026-04-02 14:18 ` [PATCH] gcov: use atomic counter updates " Konstantin Khorenko
2026-04-06 19:37 ` Nathan Chancellor
2026-04-09 8:11 ` Peter Oberparleiter [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9fba075d-9388-483f-818e-6ee3b168f18d@linux.ibm.com \
--to=oberpar@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=herbert@gondor.apana.org.au \
--cc=khorenko@virtuozzo.com \
--cc=kuba@kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@weissschuh.net \
--cc=masahiroy@kernel.org \
--cc=nathan@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=nsc@kernel.org \
--cc=ptikhomirov@virtuozzo.com \
--cc=steffen.klassert@secunet.com \
--cc=vasileios.almpanis@virtuozzo.com \
--cc=zaslonko@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox