[RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[Peter Zijlstra]

So I was looking at text_poke_bp() today and I couldn't make sense of
the barriers there.

How's for something like so?

---
 arch/x86/kernel/alternative.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 32e14d137416..3344d3382e91 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -742,7 +742,16 @@ static void *bp_int3_handler, *bp_int3_addr;
 
 int poke_int3_handler(struct pt_regs *regs)
 {
-	/* bp_patching_in_progress */
+	/*
+	 * Having observed our INT3 instruction, we now must observe
+	 * bp_patching_in_progress.
+	 *
+	 * 	in_progress = TRUE		INT3
+	 * 	WMB				RMB
+	 * 	write INT3			if (in_progress)
+	 *
+	 * Idem for bp_int3_handler.
+	 */
 	smp_rmb();
 
 	if (likely(!bp_patching_in_progress))
@@ -788,9 +797,8 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)
 	bp_int3_addr = (u8 *)addr + sizeof(int3);
 	bp_patching_in_progress = true;
 	/*
-	 * Corresponding read barrier in int3 notifier for
-	 * making sure the in_progress flags is correctly ordered wrt.
-	 * patching
+	 * Corresponding read barrier in int3 notifier for making sure the
+	 * in_progress and handler are correctly ordered wrt. patching.
 	 */
 	smp_wmb();
 
@@ -815,9 +823,11 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)
 	text_poke(addr, opcode, sizeof(int3));
 
 	on_each_cpu(do_sync_core, NULL, 1);
-
+	/*
+	 * sync_core() implies an smp_mb() and orders this store against
+	 * the writing of the new instruction.
+	 */
 	bp_patching_in_progress = false;
-	smp_wmb();
 
 	return addr;
 }

Re: [RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[Steven Rostedt]

On Mon, 31 Jul 2017 12:21:54 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> So I was looking at text_poke_bp() today and I couldn't make sense of
> the barriers there.
> 
> How's for something like so?
> 
> ---
>  arch/x86/kernel/alternative.c | 22 ++++++++++++++++------
>  1 file changed, 16 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
> index 32e14d137416..3344d3382e91 100644
> --- a/arch/x86/kernel/alternative.c
> +++ b/arch/x86/kernel/alternative.c
> @@ -742,7 +742,16 @@ static void *bp_int3_handler, *bp_int3_addr;
>  
>  int poke_int3_handler(struct pt_regs *regs)
>  {
> -	/* bp_patching_in_progress */
> +	/*
> +	 * Having observed our INT3 instruction, we now must observe
> +	 * bp_patching_in_progress.
> +	 *
> +	 * 	in_progress = TRUE		INT3
> +	 * 	WMB				RMB
> +	 * 	write INT3			if (in_progress)
> +	 *
> +	 * Idem for bp_int3_handler.
> +	 */

Looks correct.

>  	smp_rmb();
>  
>  	if (likely(!bp_patching_in_progress))
> @@ -788,9 +797,8 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)
>  	bp_int3_addr = (u8 *)addr + sizeof(int3);
>  	bp_patching_in_progress = true;
>  	/*
> -	 * Corresponding read barrier in int3 notifier for
> -	 * making sure the in_progress flags is correctly ordered wrt.
> -	 * patching
> +	 * Corresponding read barrier in int3 notifier for making sure the
> +	 * in_progress and handler are correctly ordered wrt. patching.
>  	 */

This looks correct as well.

>  	smp_wmb();
>  
> @@ -815,9 +823,11 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)
>  	text_poke(addr, opcode, sizeof(int3));
>  
>  	on_each_cpu(do_sync_core, NULL, 1);
> -
> +	/*
> +	 * sync_core() implies an smp_mb() and orders this store against
> +	 * the writing of the new instruction.
> +	 */

Yep.

>  	bp_patching_in_progress = false;
> -	smp_wmb();

Heh, I think this was a "lets not leak bp_patching_in_progress" out of
this function. But I don't see any harm if it happens.

As this function was a *very* slow path, that smp_wmb() was a "it's not
really needed, but it wont hurt anything to slap it in there just in
case".


Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

>  
>  	return addr;
>  }

Re: [RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[Peter Zijlstra]

On Mon, Jul 31, 2017 at 10:23:23AM -0400, Steven Rostedt wrote:
> > -	smp_wmb();
> 
> Heh, I think this was a "lets not leak bp_patching_in_progress" out of
> this function. But I don't see any harm if it happens.
> 
> As this function was a *very* slow path, that smp_wmb() was a "it's not
> really needed, but it wont hurt anything to slap it in there just in
> case".

Well, this is x86, its a NO-OP. The only reason to write barriers like
that is for documentation purposes and in that regard is confuses. IOW
has negative value.

Re: [RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[Steven Rostedt]

On Mon, 31 Jul 2017 17:04:11 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> Well, this is x86, its a NO-OP. The only reason to write barriers like

Is it really a nop?

> that is for documentation purposes and in that regard is confuses. IOW
> has negative value.

Not arguing. But a comment would have helped, even if it was

 /* This isn't really needed, but just being paranoid */

-- Steve

Re: [RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[Peter Zijlstra]

On Mon, Jul 31, 2017 at 11:16:48AM -0400, Steven Rostedt wrote:
> On Mon, 31 Jul 2017 17:04:11 +0200
> Peter Zijlstra <peterz@infradead.org> wrote:
> 
> > Well, this is x86, its a NO-OP. The only reason to write barriers like
> 
> Is it really a nop?

smp_wmb() is ever since we killed OOSTORE, and smp_rmb() is effectively
(I don't think anybody uses PPRO_FENCE).

The normal x86 TSO memory model doesn't need read or write barriers --
only weird and wonderful (broken) x86 bits do.

Re: [RFC][PATCH]: x86: clarify/fix no-op barriers for text_poke_bp()

[Jiri Kosina]

On Mon, 31 Jul 2017, Peter Zijlstra wrote:

> So I was looking at text_poke_bp() today and I couldn't make sense of
> the barriers there.
> 
> How's for something like so?

Hm, agreed, the barrier after unsetting in_progress is superfluous, and 
the comments are definitely more understandable this way; I probably 
didn't have one of my best days when writing those back then.

	Acked-by: Jiri Kosina <jkosina@suse.cz>

Thanks,

-- 
Jiri Kosina
SUSE Labs

[tip:x86/asm] x86: Clarify/fix no-op barriers for text_poke_bp()

[tip-bot for Peter Zijlstra]

Commit-ID:  01651324edad9db4fe49fb39b905c76861649b4c
Gitweb:     http://git.kernel.org/tip/01651324edad9db4fe49fb39b905c76861649b4c
Author:     Peter Zijlstra <peterz@infradead.org>
AuthorDate: Mon, 31 Jul 2017 12:21:54 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Thu, 10 Aug 2017 17:35:19 +0200

x86: Clarify/fix no-op barriers for text_poke_bp()

So I was looking at text_poke_bp() today and I couldn't make sense of
the barriers there.

How's for something like so?

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Acked-by: Jiri Kosina <jkosina@suse.cz>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: masami.hiramatsu.pt@hitachi.com
Link: http://lkml.kernel.org/r/20170731102154.f57cvkjtnbmtctk6@hirez.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/alternative.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 32e14d1..3344d33 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -742,7 +742,16 @@ static void *bp_int3_handler, *bp_int3_addr;
 
 int poke_int3_handler(struct pt_regs *regs)
 {
-	/* bp_patching_in_progress */
+	/*
+	 * Having observed our INT3 instruction, we now must observe
+	 * bp_patching_in_progress.
+	 *
+	 * 	in_progress = TRUE		INT3
+	 * 	WMB				RMB
+	 * 	write INT3			if (in_progress)
+	 *
+	 * Idem for bp_int3_handler.
+	 */
 	smp_rmb();
 
 	if (likely(!bp_patching_in_progress))
@@ -788,9 +797,8 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)
 	bp_int3_addr = (u8 *)addr + sizeof(int3);
 	bp_patching_in_progress = true;
 	/*
-	 * Corresponding read barrier in int3 notifier for
-	 * making sure the in_progress flags is correctly ordered wrt.
-	 * patching
+	 * Corresponding read barrier in int3 notifier for making sure the
+	 * in_progress and handler are correctly ordered wrt. patching.
 	 */
 	smp_wmb();
 
@@ -815,9 +823,11 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler)
 	text_poke(addr, opcode, sizeof(int3));
 
 	on_each_cpu(do_sync_core, NULL, 1);
-
+	/*
+	 * sync_core() implies an smp_mb() and orders this store against
+	 * the writing of the new instruction.
+	 */
 	bp_patching_in_progress = false;
-	smp_wmb();
 
 	return addr;
 }