From: Peter Oberparleiter <oberpar@linux.ibm.com>
To: Nathan Chancellor <nathan@kernel.org>,
Konstantin Khorenko <khorenko@virtuozzo.com>,
Andrew Morton <akpm@linux-foundation.org>
Cc: "Mikhail Zaslonko" <zaslonko@linux.ibm.com>,
"Nicolas Schier" <nsc@kernel.org>,
"Masahiro Yamada" <masahiroy@kernel.org>,
"Thomas Weißschuh" <linux@weissschuh.net>,
"Arnd Bergmann" <arnd@arndb.de>,
"Steffen Klassert" <steffen.klassert@secunet.com>,
"Herbert Xu" <herbert@gondor.apana.org.au>,
linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org,
"Pavel Tikhomirov" <ptikhomirov@virtuozzo.com>,
"Vasileios Almpanis" <vasileios.almpanis@virtuozzo.com>,
"Jakub Kicinski" <kuba@kernel.org>
Subject: Re: [PATCH] gcov: use atomic counter updates to fix concurrent access crashes
Date: Thu, 9 Apr 2026 10:11:24 +0200 [thread overview]
Message-ID: <9fba075d-9388-483f-818e-6ee3b168f18d@linux.ibm.com> (raw)
In-Reply-To: <20260406193707.GB1319599@ax162>
On 06.04.2026 21:37, Nathan Chancellor wrote:
> On Thu, Apr 02, 2026 at 05:18:31PM +0300, Konstantin Khorenko wrote:
>> GCC's GCOV instrumentation can merge global branch counters with loop
>> induction variables as an optimization. In inflate_fast(), the inner
>> copy loops get transformed so that the GCOV counter value is loaded
>> multiple times to compute the loop base address, start index, and end
>> bound. Since GCOV counters are global (not per-CPU), concurrent
>> execution on different CPUs causes the counter to change between loads,
>> producing inconsistent values and out-of-bounds memory writes.
>>
>> The crash manifests during IPComp (IP Payload Compression) processing
>> when inflate_fast() runs concurrently on multiple CPUs:
>>
>> BUG: unable to handle page fault for address: ffffd0a3c0902ffa
>> RIP: inflate_fast+1431
>> Call Trace:
>> zlib_inflate
>> __deflate_decompress
>> crypto_comp_decompress
>> ipcomp_decompress [xfrm_ipcomp]
>> ipcomp_input [xfrm_ipcomp]
>> xfrm_input
>>
>> At the crash point, the compiler generated three loads from the same
>> global GCOV counter (__gcov0.inflate_fast+216) to compute base, start,
>> and end for an indexed loop. Another CPU modified the counter between
>> loads, making the values inconsistent — the write went 3.4 MB past a
>> 65 KB buffer.
>>
>> Add -fprofile-update=atomic to CFLAGS_GCOV at the global level in the
>> top-level Makefile. This tells GCC that GCOV counters may be
>> concurrently accessed, causing counter updates to use atomic
>> instructions (lock addq) instead of plain load/store. This prevents
>> the compiler from merging counters with loop induction variables.
>>
>> Applying this globally rather than per-subsystem not only addresses the
>> observed crash in zlib but makes GCOV coverage data more consistent
>> overall, preventing similar issues in any kernel code path that may
>> execute concurrently.
>>
>> Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
>> Tested-by: Peter Oberparleiter <oberpar@linux.ibm.com>
>> Reviewed-by: Peter Oberparleiter <oberpar@linux.ibm.com>
>
> While this is obviously a fix, what are the chances of regressions from
> this change? As this should only impact GCOV, this could go via whatever
> tree carries GCOV patches. If Kbuild is to take this change, my vote
> would be to defer it to 7.2 at this point in the development cycle so
> that it can have most of a cycle to sit in -next.
Adding Andrew since he typically integrates GCOV patches via his tree,
and for input on how to handle this patch.
To summarize the situation, this patch:
- is only effective with GCC + GCOV profiling enabled
- fixes a run-time crash
- improves overall GCOV coverage data consistency
- triggers a number of build errors due to side-effects on GCC constant
folding and therefore depends on the associated series [1] that fixes
these build-errors
- has a non-zero chance to trigger additional build-time errors, e.g.
in similar macros guarded by arch/config symbols not covered by
current testing
Given the last point, I agree with Nathan that this patch would benefit
from additional test coverage to minimize regression risks, e.g. via a
cycle in -next.
[1]
https://lore.kernel.org/lkml/20260402140558.1437002-1-khorenko@virtuozzo.com/
>> ---
>> Makefile | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/Makefile b/Makefile
>> index 6b1d9fb1a6b4..a55ad668d6ba 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -806,7 +806,7 @@ all: vmlinux
>>
>> CFLAGS_GCOV := -fprofile-arcs -ftest-coverage
>> ifdef CONFIG_CC_IS_GCC
>> -CFLAGS_GCOV += -fno-tree-loop-im
>> +CFLAGS_GCOV += -fno-tree-loop-im -fprofile-update=atomic
>> endif
>> export CFLAGS_GCOV
>>
>> --
>> 2.43.5
>>
--
Peter Oberparleiter
Linux on IBM Z Development - IBM Germany R&D
next prev parent reply other threads:[~2026-04-09 8:12 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-02 14:18 [PATCH 0/1] gcov: add -fprofile-update=atomic to fix concurrent access crashes Konstantin Khorenko
2026-04-02 14:18 ` [PATCH] gcov: use atomic counter updates " Konstantin Khorenko
2026-04-06 19:37 ` Nathan Chancellor
2026-04-09 8:11 ` Peter Oberparleiter [this message]
2026-04-15 10:19 ` Andrew Morton
2026-04-11 14:17 ` kernel test robot
-- strict thread matches above, loose matches on Subject: below --
2026-04-11 11:17 kernel test robot
2026-04-17 11:28 kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9fba075d-9388-483f-818e-6ee3b168f18d@linux.ibm.com \
--to=oberpar@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=herbert@gondor.apana.org.au \
--cc=khorenko@virtuozzo.com \
--cc=kuba@kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@weissschuh.net \
--cc=masahiroy@kernel.org \
--cc=nathan@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=nsc@kernel.org \
--cc=ptikhomirov@virtuozzo.com \
--cc=steffen.klassert@secunet.com \
--cc=vasileios.almpanis@virtuozzo.com \
--cc=zaslonko@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.