* [PATCH 1/2] futex: Use atomic64_inc_return() in get_inode_sequence_number()
@ 2024-10-10 7:10 Uros Bizjak
2024-10-10 7:10 ` [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() " Uros Bizjak
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Uros Bizjak @ 2024-10-10 7:10 UTC (permalink / raw)
To: linux-kernel
Cc: Uros Bizjak, Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Darren Hart, Davidlohr Bueso, André Almeida
Use atomic64_inc_return(&ref) instead of atomic64_add_return(1, &ref)
to use optimized implementation and ease register pressure around
the primitive for targets that implement optimized variant.
Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Darren Hart <dvhart@infradead.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: "André Almeida" <andrealmeid@igalia.com>
---
kernel/futex/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 136768ae2637..3146730e55f7 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -181,7 +181,7 @@ static u64 get_inode_sequence_number(struct inode *inode)
return old;
for (;;) {
- u64 new = atomic64_add_return(1, &i_seq);
+ u64 new = atomic64_inc_return(&i_seq);
if (WARN_ON_ONCE(!new))
continue;
--
2.46.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() in get_inode_sequence_number()
2024-10-10 7:10 [PATCH 1/2] futex: Use atomic64_inc_return() in get_inode_sequence_number() Uros Bizjak
@ 2024-10-10 7:10 ` Uros Bizjak
2024-10-10 18:06 ` André Almeida
2024-10-17 20:10 ` [tip: locking/core] " tip-bot2 for Uros Bizjak
2024-10-10 18:05 ` [PATCH 1/2] futex: Use atomic64_inc_return() " André Almeida
2024-10-17 20:10 ` [tip: locking/core] " tip-bot2 for Uros Bizjak
2 siblings, 2 replies; 6+ messages in thread
From: Uros Bizjak @ 2024-10-10 7:10 UTC (permalink / raw)
To: linux-kernel
Cc: Uros Bizjak, Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
Darren Hart, Davidlohr Bueso, André Almeida
Optimize get_inode_sequence_number() to use simpler and faster:
!atomic64_try_cmpxchg_relaxed(*ptr, &old, new)
instead of:
atomic64_cmpxchg relaxed(*ptr, old, new) != old
The x86 CMPXCHG instruction returns success in ZF flag, so
this change saves a compare after cmpxchg. The generated
code improves from:
3da: 31 c0 xor %eax,%eax
3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
3e3: 00 00
3e5: 48 85 c0 test %rax,%rax
3e8: 48 0f 44 c1 cmove %rcx,%rax
to:
3da: 31 c0 xor %eax,%eax
3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
3e3: 00 00
3e5: 48 0f 44 c1 cmove %rcx,%rax
Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Darren Hart <dvhart@infradead.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: "André Almeida" <andrealmeid@igalia.com>
---
kernel/futex/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 3146730e55f7..11795439efb7 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -185,8 +185,9 @@ static u64 get_inode_sequence_number(struct inode *inode)
if (WARN_ON_ONCE(!new))
continue;
- old = atomic64_cmpxchg_relaxed(&inode->i_sequence, 0, new);
- if (old)
+ old = 0;
+ if (!atomic64_try_cmpxchg_relaxed(&inode->i_sequence,
+ &old, new))
return old;
return new;
}
--
2.46.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] futex: Use atomic64_inc_return() in get_inode_sequence_number()
2024-10-10 7:10 [PATCH 1/2] futex: Use atomic64_inc_return() in get_inode_sequence_number() Uros Bizjak
2024-10-10 7:10 ` [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() " Uros Bizjak
@ 2024-10-10 18:05 ` André Almeida
2024-10-17 20:10 ` [tip: locking/core] " tip-bot2 for Uros Bizjak
2 siblings, 0 replies; 6+ messages in thread
From: André Almeida @ 2024-10-10 18:05 UTC (permalink / raw)
To: Uros Bizjak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
Davidlohr Bueso, linux-kernel
Em 10/10/2024 04:10, Uros Bizjak escreveu:
> Use atomic64_inc_return(&ref) instead of atomic64_add_return(1, &ref)
> to use optimized implementation and ease register pressure around
> the primitive for targets that implement optimized variant.
>
> Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Darren Hart <dvhart@infradead.org>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: "André Almeida" <andrealmeid@igalia.com>
> ---
Reviewed-by: André Almeida <andrealmeid@igalia.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() in get_inode_sequence_number()
2024-10-10 7:10 ` [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() " Uros Bizjak
@ 2024-10-10 18:06 ` André Almeida
2024-10-17 20:10 ` [tip: locking/core] " tip-bot2 for Uros Bizjak
1 sibling, 0 replies; 6+ messages in thread
From: André Almeida @ 2024-10-10 18:06 UTC (permalink / raw)
To: Uros Bizjak
Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
linux-kernel, Davidlohr Bueso
Em 10/10/2024 04:10, Uros Bizjak escreveu:
> Optimize get_inode_sequence_number() to use simpler and faster:
>
> !atomic64_try_cmpxchg_relaxed(*ptr, &old, new)
>
> instead of:
>
> atomic64_cmpxchg relaxed(*ptr, old, new) != old
>
> The x86 CMPXCHG instruction returns success in ZF flag, so
> this change saves a compare after cmpxchg. The generated
> code improves from:
>
> 3da: 31 c0 xor %eax,%eax
> 3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
> 3e3: 00 00
> 3e5: 48 85 c0 test %rax,%rax
> 3e8: 48 0f 44 c1 cmove %rcx,%rax
>
> to:
>
> 3da: 31 c0 xor %eax,%eax
> 3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
> 3e3: 00 00
> 3e5: 48 0f 44 c1 cmove %rcx,%rax
>
> Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Darren Hart <dvhart@infradead.org>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: "André Almeida" <andrealmeid@igalia.com>
Reviewed-by: André Almeida <andrealmeid@igalia.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [tip: locking/core] futex: Use atomic64_try_cmpxchg_relaxed() in get_inode_sequence_number()
2024-10-10 7:10 ` [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() " Uros Bizjak
2024-10-10 18:06 ` André Almeida
@ 2024-10-17 20:10 ` tip-bot2 for Uros Bizjak
1 sibling, 0 replies; 6+ messages in thread
From: tip-bot2 for Uros Bizjak @ 2024-10-17 20:10 UTC (permalink / raw)
To: linux-tip-commits
Cc: Uros Bizjak, Thomas Gleixner, andrealmeid, x86, linux-kernel
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 87347f148061b48c3495fb61dcbad384760da9cf
Gitweb: https://git.kernel.org/tip/87347f148061b48c3495fb61dcbad384760da9cf
Author: Uros Bizjak <ubizjak@gmail.com>
AuthorDate: Thu, 10 Oct 2024 09:10:05 +02:00
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Thu, 17 Oct 2024 22:02:27 +02:00
futex: Use atomic64_try_cmpxchg_relaxed() in get_inode_sequence_number()
Optimize get_inode_sequence_number() to use simpler and faster:
!atomic64_try_cmpxchg_relaxed(*ptr, &old, new)
instead of:
atomic64_cmpxchg relaxed(*ptr, old, new) != old
The x86 CMPXCHG instruction returns success in ZF flag, so
this change saves a compare after cmpxchg. The generated
code improves from:
3da: 31 c0 xor %eax,%eax
3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
3e3: 00 00
3e5: 48 85 c0 test %rax,%rax
3e8: 48 0f 44 c1 cmove %rcx,%rax
to:
3da: 31 c0 xor %eax,%eax
3dc: f0 48 0f b1 8a 38 01 lock cmpxchg %rcx,0x138(%rdx)
3e3: 00 00
3e5: 48 0f 44 c1 cmove %rcx,%rax
Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: André Almeida <andrealmeid@igalia.com>
Link: https://lore.kernel.org/all/20241010071023.21913-2-ubizjak@gmail.com
---
kernel/futex/core.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 3146730..692912b 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -185,8 +185,8 @@ static u64 get_inode_sequence_number(struct inode *inode)
if (WARN_ON_ONCE(!new))
continue;
- old = atomic64_cmpxchg_relaxed(&inode->i_sequence, 0, new);
- if (old)
+ old = 0;
+ if (!atomic64_try_cmpxchg_relaxed(&inode->i_sequence, &old, new))
return old;
return new;
}
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [tip: locking/core] futex: Use atomic64_inc_return() in get_inode_sequence_number()
2024-10-10 7:10 [PATCH 1/2] futex: Use atomic64_inc_return() in get_inode_sequence_number() Uros Bizjak
2024-10-10 7:10 ` [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() " Uros Bizjak
2024-10-10 18:05 ` [PATCH 1/2] futex: Use atomic64_inc_return() " André Almeida
@ 2024-10-17 20:10 ` tip-bot2 for Uros Bizjak
2 siblings, 0 replies; 6+ messages in thread
From: tip-bot2 for Uros Bizjak @ 2024-10-17 20:10 UTC (permalink / raw)
To: linux-tip-commits
Cc: Uros Bizjak, Thomas Gleixner, andrealmeid, x86, linux-kernel
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 19298f48694987fac843261c84e24834c255b451
Gitweb: https://git.kernel.org/tip/19298f48694987fac843261c84e24834c255b451
Author: Uros Bizjak <ubizjak@gmail.com>
AuthorDate: Thu, 10 Oct 2024 09:10:04 +02:00
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Thu, 17 Oct 2024 22:02:27 +02:00
futex: Use atomic64_inc_return() in get_inode_sequence_number()
Use atomic64_inc_return(&ref) instead of atomic64_add_return(1, &ref)
to use optimized implementation and ease register pressure around
the primitive for targets that implement optimized variant.
Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: André Almeida <andrealmeid@igalia.com>
Link: https://lore.kernel.org/all/20241010071023.21913-1-ubizjak@gmail.com
---
kernel/futex/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index 136768a..3146730 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -181,7 +181,7 @@ static u64 get_inode_sequence_number(struct inode *inode)
return old;
for (;;) {
- u64 new = atomic64_add_return(1, &i_seq);
+ u64 new = atomic64_inc_return(&i_seq);
if (WARN_ON_ONCE(!new))
continue;
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-10-17 20:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-10 7:10 [PATCH 1/2] futex: Use atomic64_inc_return() in get_inode_sequence_number() Uros Bizjak
2024-10-10 7:10 ` [PATCH 2/2] futex: Use atomic64_try_cmpxchg_relaxed() " Uros Bizjak
2024-10-10 18:06 ` André Almeida
2024-10-17 20:10 ` [tip: locking/core] " tip-bot2 for Uros Bizjak
2024-10-10 18:05 ` [PATCH 1/2] futex: Use atomic64_inc_return() " André Almeida
2024-10-17 20:10 ` [tip: locking/core] " tip-bot2 for Uros Bizjak
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox