[PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[Jiri Olsa]

From: Jiri Olsa <olsajiri@gmail.com>

There's error path that could lead to inactive uprobe:

  1) uprobe_register succeeds - updates instruction to int3 and
     changes ref_ctr from 0 to 1
  2) uprobe_unregister fails  - int3 stays in place, but ref_ctr
     is changed to 0 (it's not restored to 1 in the fail path)
     uprobe is leaked
  3) another uprobe_register comes and re-uses the leaked uprobe
     and succeds - but int3 is already in place, so ref_ctr update
     is skipped and it stays 0 - uprobe CAN NOT be triggered now
  4) uprobe_unregister fails because ref_ctr value is unexpected

Fixing this by reverting the updated ref_ctr value back to 1 in step 2),
which is the case when uprobe_unregister fails (int3 stays in place),
but we have already updated refctr.

The new scenario will go as follows:

  1) uprobe_register succeeds - updates instruction to int3 and
     changes ref_ctr from 0 to 1
  2) uprobe_unregister fails  - int3 stays in place and ref_ctr
     is reverted to 1..  uprobe is leaked
  3) another uprobe_register comes and re-uses the leaked uprobe
     and succeds - but int3 is already in place, so ref_ctr update
     is skipped and it stays 1 - uprobe CAN be triggered now
  4) uprobe_unregister succeeds

Suggested-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
Please note it's based on mm-stable branch, because it has the
latest uprobe_write_opcode rewrite changes.

 kernel/events/uprobes.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 4c965ba77f9f..84ee7b590861 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -581,8 +581,8 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
 
 out:
 	/* Revert back reference counter if instruction update failed. */
-	if (ret < 0 && is_register && ref_ctr_updated)
-		update_ref_ctr(uprobe, mm, -1);
+	if (ret < 0 && ref_ctr_updated)
+		update_ref_ctr(uprobe, mm, is_register ? -1 : 1);
 
 	/* try collapse pmd for compound page */
 	if (ret > 0)
-- 
2.49.0

Re: [PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[David Hildenbrand]

On 13.05.25 14:21, Jiri Olsa wrote:
> From: Jiri Olsa <olsajiri@gmail.com>
> 

Thanks for debugging.

> There's error path that could lead to inactive uprobe:
> 
>    1) uprobe_register succeeds - updates instruction to int3 and
>       changes ref_ctr from 0 to 1
>    2) uprobe_unregister fails  - int3 stays in place, but ref_ctr
>       is changed to 0 (it's not restored to 1 in the fail path)
>       uprobe is leaked
>    3) another uprobe_register comes and re-uses the leaked uprobe
>       and succeds - but int3 is already in place, so ref_ctr update
>       is skipped and it stays 0 - uprobe CAN NOT be triggered now
>    4) uprobe_unregister fails because ref_ctr value is unexpected
> 
> Fixing this by reverting the updated ref_ctr value back to 1 in step 2),
> which is the case when uprobe_unregister fails (int3 stays in place),
> but we have already updated refctr.
> 
> The new scenario will go as follows:
> 
>    1) uprobe_register succeeds - updates instruction to int3 and
>       changes ref_ctr from 0 to 1
>    2) uprobe_unregister fails  - int3 stays in place and ref_ctr
>       is reverted to 1..  uprobe is leaked
>    3) another uprobe_register comes and re-uses the leaked uprobe
>       and succeds - but int3 is already in place, so ref_ctr update
>       is skipped and it stays 1 - uprobe CAN be triggered now
>    4) uprobe_unregister succeeds
> 
> Suggested-by: Oleg Nesterov <oleg@redhat.com>

If it's in mm-stable, we should have

Fixes: ...

here

> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
> Please note it's based on mm-stable branch, because it has the
> latest uprobe_write_opcode rewrite changes.
> 
>   kernel/events/uprobes.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
> index 4c965ba77f9f..84ee7b590861 100644
> --- a/kernel/events/uprobes.c
> +++ b/kernel/events/uprobes.c
> @@ -581,8 +581,8 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
>   
>   out:
>   	/* Revert back reference counter if instruction update failed. */
> -	if (ret < 0 && is_register && ref_ctr_updated)
> -		update_ref_ctr(uprobe, mm, -1);
> +	if (ret < 0 && ref_ctr_updated)
> +		update_ref_ctr(uprobe, mm, is_register ? -1 : 1);


Hm, but my patch essentially did here

         /* Revert back reference counter if instruction update failed. */
-       if (ret && is_register && ref_ctr_updated)
+       if (ret < 0 && is_register && ref_ctr_updated)
                 update_ref_ctr(uprobe, mm, -1);

So how come this wasn't a problem before?

-- 
Cheers,

David / dhildenb

Re: [PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[David Hildenbrand]

On 13.05.25 15:16, David Hildenbrand wrote:
> On 13.05.25 14:21, Jiri Olsa wrote:
>> From: Jiri Olsa <olsajiri@gmail.com>
>>
> 
> Thanks for debugging.
> 
>> There's error path that could lead to inactive uprobe:
>>
>>     1) uprobe_register succeeds - updates instruction to int3 and
>>        changes ref_ctr from 0 to 1
>>     2) uprobe_unregister fails  - int3 stays in place, but ref_ctr
>>        is changed to 0 (it's not restored to 1 in the fail path)
>>        uprobe is leaked
>>     3) another uprobe_register comes and re-uses the leaked uprobe
>>        and succeds - but int3 is already in place, so ref_ctr update
>>        is skipped and it stays 0 - uprobe CAN NOT be triggered now
>>     4) uprobe_unregister fails because ref_ctr value is unexpected
>>
>> Fixing this by reverting the updated ref_ctr value back to 1 in step 2),
>> which is the case when uprobe_unregister fails (int3 stays in place),
>> but we have already updated refctr.
>>
>> The new scenario will go as follows:
>>
>>     1) uprobe_register succeeds - updates instruction to int3 and
>>        changes ref_ctr from 0 to 1
>>     2) uprobe_unregister fails  - int3 stays in place and ref_ctr
>>        is reverted to 1..  uprobe is leaked
>>     3) another uprobe_register comes and re-uses the leaked uprobe
>>        and succeds - but int3 is already in place, so ref_ctr update
>>        is skipped and it stays 1 - uprobe CAN be triggered now
>>     4) uprobe_unregister succeeds
>>
>> Suggested-by: Oleg Nesterov <oleg@redhat.com>
> 
> If it's in mm-stable, we should have
> 
> Fixes: ...
> 
> here
> 
>> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
>> ---
>> Please note it's based on mm-stable branch, because it has the
>> latest uprobe_write_opcode rewrite changes.
>>
>>    kernel/events/uprobes.c | 4 ++--
>>    1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
>> index 4c965ba77f9f..84ee7b590861 100644
>> --- a/kernel/events/uprobes.c
>> +++ b/kernel/events/uprobes.c
>> @@ -581,8 +581,8 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
>>    
>>    out:
>>    	/* Revert back reference counter if instruction update failed. */
>> -	if (ret < 0 && is_register && ref_ctr_updated)
>> -		update_ref_ctr(uprobe, mm, -1);
>> +	if (ret < 0 && ref_ctr_updated)
>> +		update_ref_ctr(uprobe, mm, is_register ? -1 : 1);
> 
> 
> Hm, but my patch essentially did here
> 
>           /* Revert back reference counter if instruction update failed. */
> -       if (ret && is_register && ref_ctr_updated)
> +       if (ret < 0 && is_register && ref_ctr_updated)
>                   update_ref_ctr(uprobe, mm, -1);
> 
> So how come this wasn't a problem before?

Oh, or was this a problem before? Then we should find the corresponding 
commit that needs fixing.

-- 
Cheers,

David / dhildenb

Re: [PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[Jiri Olsa]

On Tue, May 13, 2025 at 03:17:46PM +0200, David Hildenbrand wrote:
> On 13.05.25 15:16, David Hildenbrand wrote:
> > On 13.05.25 14:21, Jiri Olsa wrote:
> > > From: Jiri Olsa <olsajiri@gmail.com>
> > > 
> > 
> > Thanks for debugging.
> > 
> > > There's error path that could lead to inactive uprobe:
> > > 
> > >     1) uprobe_register succeeds - updates instruction to int3 and
> > >        changes ref_ctr from 0 to 1
> > >     2) uprobe_unregister fails  - int3 stays in place, but ref_ctr
> > >        is changed to 0 (it's not restored to 1 in the fail path)
> > >        uprobe is leaked
> > >     3) another uprobe_register comes and re-uses the leaked uprobe
> > >        and succeds - but int3 is already in place, so ref_ctr update
> > >        is skipped and it stays 0 - uprobe CAN NOT be triggered now
> > >     4) uprobe_unregister fails because ref_ctr value is unexpected
> > > 
> > > Fixing this by reverting the updated ref_ctr value back to 1 in step 2),
> > > which is the case when uprobe_unregister fails (int3 stays in place),
> > > but we have already updated refctr.
> > > 
> > > The new scenario will go as follows:
> > > 
> > >     1) uprobe_register succeeds - updates instruction to int3 and
> > >        changes ref_ctr from 0 to 1
> > >     2) uprobe_unregister fails  - int3 stays in place and ref_ctr
> > >        is reverted to 1..  uprobe is leaked
> > >     3) another uprobe_register comes and re-uses the leaked uprobe
> > >        and succeds - but int3 is already in place, so ref_ctr update
> > >        is skipped and it stays 1 - uprobe CAN be triggered now
> > >     4) uprobe_unregister succeeds
> > > 
> > > Suggested-by: Oleg Nesterov <oleg@redhat.com>
> > 
> > If it's in mm-stable, we should have
> > 
> > Fixes: ...

ok

> > 
> > here
> > 
> > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > ---
> > > Please note it's based on mm-stable branch, because it has the
> > > latest uprobe_write_opcode rewrite changes.
> > > 
> > >    kernel/events/uprobes.c | 4 ++--
> > >    1 file changed, 2 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
> > > index 4c965ba77f9f..84ee7b590861 100644
> > > --- a/kernel/events/uprobes.c
> > > +++ b/kernel/events/uprobes.c
> > > @@ -581,8 +581,8 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
> > >    out:
> > >    	/* Revert back reference counter if instruction update failed. */
> > > -	if (ret < 0 && is_register && ref_ctr_updated)
> > > -		update_ref_ctr(uprobe, mm, -1);
> > > +	if (ret < 0 && ref_ctr_updated)
> > > +		update_ref_ctr(uprobe, mm, is_register ? -1 : 1);
> > 
> > 
> > Hm, but my patch essentially did here
> > 
> >           /* Revert back reference counter if instruction update failed. */
> > -       if (ret && is_register && ref_ctr_updated)
> > +       if (ret < 0 && is_register && ref_ctr_updated)
> >                   update_ref_ctr(uprobe, mm, -1);
> > 
> > So how come this wasn't a problem before?
> 
> Oh, or was this a problem before? Then we should find the corresponding
> commit that needs fixing.

yes, I think it was a problem before, introduced early on by:
  1cc33161a83d uprobes: Support SDT markers having reference count (semaphore)

it seems the scenario described in changelog will hit the same issue even
without your patch, I'll re-run the test to be sure

thanks,
jirka

Re: [PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[Oleg Nesterov]

On 05/13, Jiri Olsa wrote:
>
> --- a/kernel/events/uprobes.c
> +++ b/kernel/events/uprobes.c
> @@ -581,8 +581,8 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
>  
>  out:
>  	/* Revert back reference counter if instruction update failed. */
> -	if (ret < 0 && is_register && ref_ctr_updated)
> -		update_ref_ctr(uprobe, mm, -1);
> +	if (ret < 0 && ref_ctr_updated)
> +		update_ref_ctr(uprobe, mm, is_register ? -1 : 1);

Acked-by: Oleg Nesterov <oleg@redhat.com>

And just in case, I agree this has nothing to do with the recent changes from David.

Oleg.

Re: [PATCH mm-stable] uprobes: Revert ref_ctr_offset in uprobe_unregister error path

[David Hildenbrand]

On 13.05.25 17:46, Oleg Nesterov wrote:
> On 05/13, Jiri Olsa wrote:
>>
>> --- a/kernel/events/uprobes.c
>> +++ b/kernel/events/uprobes.c
>> @@ -581,8 +581,8 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct *vma,
>>   
>>   out:
>>   	/* Revert back reference counter if instruction update failed. */
>> -	if (ret < 0 && is_register && ref_ctr_updated)
>> -		update_ref_ctr(uprobe, mm, -1);
>> +	if (ret < 0 && ref_ctr_updated)
>> +		update_ref_ctr(uprobe, mm, is_register ? -1 : 1);
> 
> Acked-by: Oleg Nesterov <oleg@redhat.com>
> 
> And just in case, I agree this has nothing to do with the recent changes from David.

BTW, I stumbled over this when doing the rework.

Back then, I was wondering if this is to handle the case where 
un-registering effectively fails because someone MADV_DONTNEED'ed the page.

But, we only perform the update_ref_ctr() after verify_opcode(), so that 
does not apply.

With proper Fixes:

Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb