From: Mateusz Guzik <mjguzik@gmail.com>
To: peterz@infradead.org
Cc: linux-kernel@vger.kernel.org, Mateusz Guzik <mjguzik@gmail.com>
Subject: [PATCH] seqlock: annotate spinning as unlikely() in __read_seqcount_begin
Date: Thu, 27 Jul 2023 20:03:55 +0200 [thread overview]
Message-ID: <20230727180355.813995-1-mjguzik@gmail.com> (raw)
Annotation already used to be there, but got lost in 52ac39e5db5148f7
("seqlock: seqcount_t: Implement all read APIs as statement expressions").
Does not look like it was intentional.
Without it gcc 12 decides to compile the following in path_init:
nd->m_seq = __read_seqcount_begin(&mount_lock.seqcount);
nd->r_seq = __read_seqcount_begin(&rename_lock.seqcount);
into 2 cases of conditional jumps forward if the value is even, aka
branch prediction miss by default in the common case on x86-64.
With the patch jumps are only for odd values.
before:
[snip]
mov 0x104fe96(%rip),%eax # 0xffffffff82409680 <mount_lock>
test $0x1,%al
je 0xffffffff813b97fa <path_init+122>
pause
mov 0x104fe8a(%rip),%eax # 0xffffffff82409680 <mount_lock>
test $0x1,%al
jne 0xffffffff813b97ee <path_init+110>
mov %eax,0x48(%rbx)
mov 0x104fdfd(%rip),%eax # 0xffffffff82409600 <rename_lock>
test $0x1,%al
je 0xffffffff813b9813 <path_init+147>
pause
mov 0x104fdf1(%rip),%eax # 0xffffffff82409600 <rename_lock>
test $0x1,%al
jne 0xffffffff813b9807 <path_init+135>
[/snip]
after:
[snip]
mov 0x104fec6(%rip),%eax # 0xffffffff82409680 <mount_lock>
test $0x1,%al
jne 0xffffffff813b99af <path_init+607>
mov %eax,0x48(%rbx)
mov 0x104fe35(%rip),%eax # 0xffffffff82409600 <rename_lock>
test $0x1,%al
jne 0xffffffff813b999d <path_init+589>
[/snip]
Interestingly .text gets slightly smaller (as reported by size(1)):
before: 20702563
after: 20702429
Signed-off-by: Mateusz Guzik <mjguzik@gmail.com>
---
include/linux/seqlock.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h
index 987a59d977c5..225eb1db35b8 100644
--- a/include/linux/seqlock.h
+++ b/include/linux/seqlock.h
@@ -326,7 +326,7 @@ SEQCOUNT_LOCKNAME(mutex, struct mutex, true, s->lock, mutex
({ \
unsigned __seq; \
\
- while ((__seq = seqprop_sequence(s)) & 1) \
+ while (unlikely((__seq = seqprop_sequence(s)) & 1)) \
cpu_relax(); \
\
kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); \
--
2.39.2
next reply other threads:[~2023-07-27 18:04 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-27 18:03 Mateusz Guzik [this message]
2024-11-20 11:11 ` [PATCH] seqlock: annotate spinning as unlikely() in __read_seqcount_begin Christian Brauner
2024-11-20 14:47 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230727180355.813995-1-mjguzik@gmail.com \
--to=mjguzik@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox