[2.6.37-rc1] sys_ioprio_set and RCU locking...

[2.6.37-rc1] sys_ioprio_set and RCU locking...

[Daniel J Blueman]

With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
across access to the task credentials.

Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
across the __task_cred call, which is presumably sufficient to prevent
the task credentials becoming stale.

Thus, is there preference to take the RCU lock for read across the
credential access eg at [2], or annotate the call?

Thanks,
  Daniel

--- [1]

===================================================

[ INFO: suspicious rcu_dereference_check() usage. ]

---------------------------------------------------

kernel/pid.c:419 invoked rcu_dereference_check() without protection!



other info that might help us debug this:




rcu_scheduler_active = 1, debug_locks = 1

1 lock held by start-stop-daem/2246:

 #0:  (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
sys_ioprio_set+0x8a/0x400



stack backtrace:

Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2

Call Trace:

 [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0

 [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90

 [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20

 [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400

 [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f

 [<ffffffff81003482>] system_call_fastpath+0x16/0x1b


--- [2]

Take the RCU lock for read across acquiring the pointer to the task
credentials and dereferencing it.

Signed-off-by: Daniel J Blueman <daniel.blueman@gmail.com>

diff --git a/fs/ioprio.c b/fs/ioprio.c
index 748cfb9..00cc0e5 100644
--- a/fs/ioprio.c
+++ b/fs/ioprio.c
@@ -139,8 +139,10 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who,
int, ioprio)
 				break;

 			do_each_thread(g, p) {
+				rcu_read_lock();
 				if (__task_cred(p)->uid != who)
 					continue;
+				rcu_read_unlock();
 				ret = set_task_ioprio(p, ioprio);
 				if (ret)
 					goto free_uid;
@@ -232,8 +234,10 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who)
 				break;

 			do_each_thread(g, p) {
+				rcu_read_lock();
 				if (__task_cred(p)->uid != user->uid)
 					continue;
+				rcu_read_unlock();
 				tmpio = get_task_ioprio(p);
 				if (tmpio < 0)
 					continue;
-- 
Daniel J Blueman

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Paul E. McKenney]

On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
> across access to the task credentials.
> 
> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
> across the __task_cred call, which is presumably sufficient to prevent
> the task credentials becoming stale.
> 
> Thus, is there preference to take the RCU lock for read across the
> credential access eg at [2], or annotate the call?
> 
> Thanks,
>   Daniel
> 
> --- [1]
> 
> ===================================================
> 
> [ INFO: suspicious rcu_dereference_check() usage. ]
> 
> ---------------------------------------------------
> 
> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
> 
> 
> 
> other info that might help us debug this:
> 
> 
> 
> 
> rcu_scheduler_active = 1, debug_locks = 1
> 
> 1 lock held by start-stop-daem/2246:
> 
>  #0:  (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
> sys_ioprio_set+0x8a/0x400
> 
> 
> 
> stack backtrace:
> 
> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
> 
> Call Trace:
> 
>  [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
> 
>  [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
> 
>  [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
> 
>  [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
> 
>  [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
> 
>  [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
> 
> 
> --- [2]
> 
> Take the RCU lock for read across acquiring the pointer to the task
> credentials and dereferencing it.

Jens, does this look sane?

							Thanx, Paul

> Signed-off-by: Daniel J Blueman <daniel.blueman@gmail.com>
> 
> diff --git a/fs/ioprio.c b/fs/ioprio.c
> index 748cfb9..00cc0e5 100644
> --- a/fs/ioprio.c
> +++ b/fs/ioprio.c
> @@ -139,8 +139,10 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who,
> int, ioprio)
>  				break;
> 
>  			do_each_thread(g, p) {
> +				rcu_read_lock();
>  				if (__task_cred(p)->uid != who)
>  					continue;
> +				rcu_read_unlock();
>  				ret = set_task_ioprio(p, ioprio);
>  				if (ret)
>  					goto free_uid;
> @@ -232,8 +234,10 @@ SYSCALL_DEFINE2(ioprio_get, int, which, int, who)
>  				break;
> 
>  			do_each_thread(g, p) {
> +				rcu_read_lock();
>  				if (__task_cred(p)->uid != user->uid)
>  					continue;
> +				rcu_read_unlock();
>  				tmpio = get_task_ioprio(p);
>  				if (tmpio < 0)
>  					continue;
> -- 
> Daniel J Blueman
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Jens Axboe]

On 2010-11-07 19:54, Paul E. McKenney wrote:
> On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
>> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
>> across access to the task credentials.
>>
>> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
>> across the __task_cred call, which is presumably sufficient to prevent
>> the task credentials becoming stale.
>>
>> Thus, is there preference to take the RCU lock for read across the
>> credential access eg at [2], or annotate the call?
>>
>> Thanks,
>>   Daniel
>>
>> --- [1]
>>
>> ===================================================
>>
>> [ INFO: suspicious rcu_dereference_check() usage. ]
>>
>> ---------------------------------------------------
>>
>> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
>>
>>
>>
>> other info that might help us debug this:
>>
>>
>>
>>
>> rcu_scheduler_active = 1, debug_locks = 1
>>
>> 1 lock held by start-stop-daem/2246:
>>
>>  #0:  (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
>> sys_ioprio_set+0x8a/0x400
>>
>>
>>
>> stack backtrace:
>>
>> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
>>
>> Call Trace:
>>
>>  [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
>>
>>  [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
>>
>>  [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
>>
>>  [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
>>
>>  [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>>
>>  [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
>>
>>
>> --- [2]
>>
>> Take the RCU lock for read across acquiring the pointer to the task
>> credentials and dereferencing it.
> 
> Jens, does this look sane?

Yes, looks clean enough to me.

-- 
Jens Axboe

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Paul E. McKenney]

On Mon, Nov 08, 2010 at 02:28:29PM +0100, Jens Axboe wrote:
> On 2010-11-07 19:54, Paul E. McKenney wrote:
> > On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
> >> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
> >> across access to the task credentials.
> >>
> >> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
> >> across the __task_cred call, which is presumably sufficient to prevent
> >> the task credentials becoming stale.
> >>
> >> Thus, is there preference to take the RCU lock for read across the
> >> credential access eg at [2], or annotate the call?
> >>
> >> Thanks,
> >>   Daniel
> >>
> >> --- [1]
> >>
> >> ===================================================
> >>
> >> [ INFO: suspicious rcu_dereference_check() usage. ]
> >>
> >> ---------------------------------------------------
> >>
> >> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
> >>
> >>
> >>
> >> other info that might help us debug this:
> >>
> >>
> >>
> >>
> >> rcu_scheduler_active = 1, debug_locks = 1
> >>
> >> 1 lock held by start-stop-daem/2246:
> >>
> >>  #0:  (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
> >> sys_ioprio_set+0x8a/0x400
> >>
> >>
> >>
> >> stack backtrace:
> >>
> >> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
> >>
> >> Call Trace:
> >>
> >>  [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
> >>
> >>  [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
> >>
> >>  [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
> >>
> >>  [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
> >>
> >>  [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
> >>
> >>  [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
> >>
> >>
> >> --- [2]
> >>
> >> Take the RCU lock for read across acquiring the pointer to the task
> >> credentials and dereferencing it.
> > 
> > Jens, does this look sane?
> 
> Yes, looks clean enough to me.

Very good!  Are you willing to take the patch in your tree?

							Thanx, Paul

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Jens Axboe]

On 2010-11-08 14:52, Paul E. McKenney wrote:
> On Mon, Nov 08, 2010 at 02:28:29PM +0100, Jens Axboe wrote:
>> On 2010-11-07 19:54, Paul E. McKenney wrote:
>>> On Tue, Nov 02, 2010 at 12:15:30PM +0000, Daniel J Blueman wrote:
>>>> With 2.6.37-rc1, I observe sys_ioprio_set not taking the RCU lock [1]
>>>> across access to the task credentials.
>>>>
>>>> Inspecting the code in fs/ioprio.c, the tasklist_lock is held for read
>>>> across the __task_cred call, which is presumably sufficient to prevent
>>>> the task credentials becoming stale.
>>>>
>>>> Thus, is there preference to take the RCU lock for read across the
>>>> credential access eg at [2], or annotate the call?
>>>>
>>>> Thanks,
>>>>   Daniel
>>>>
>>>> --- [1]
>>>>
>>>> ===================================================
>>>>
>>>> [ INFO: suspicious rcu_dereference_check() usage. ]
>>>>
>>>> ---------------------------------------------------
>>>>
>>>> kernel/pid.c:419 invoked rcu_dereference_check() without protection!
>>>>
>>>>
>>>>
>>>> other info that might help us debug this:
>>>>
>>>>
>>>>
>>>>
>>>> rcu_scheduler_active = 1, debug_locks = 1
>>>>
>>>> 1 lock held by start-stop-daem/2246:
>>>>
>>>>  #0:  (tasklist_lock){.?.?..}, at: [<ffffffff811a2dfa>]
>>>> sys_ioprio_set+0x8a/0x400
>>>>
>>>>
>>>>
>>>> stack backtrace:
>>>>
>>>> Pid: 2246, comm: start-stop-daem Not tainted 2.6.37-rc1-330cd+ #2
>>>>
>>>> Call Trace:
>>>>
>>>>  [<ffffffff8109f5f4>] lockdep_rcu_dereference+0xa4/0xc0
>>>>
>>>>  [<ffffffff81085651>] find_task_by_pid_ns+0x81/0x90
>>>>
>>>>  [<ffffffff8108567d>] find_task_by_vpid+0x1d/0x20
>>>>
>>>>  [<ffffffff811a3160>] sys_ioprio_set+0x3f0/0x400
>>>>
>>>>  [<ffffffff816efa79>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>>>>
>>>>  [<ffffffff81003482>] system_call_fastpath+0x16/0x1b
>>>>
>>>>
>>>> --- [2]
>>>>
>>>> Take the RCU lock for read across acquiring the pointer to the task
>>>> credentials and dereferencing it.
>>>
>>> Jens, does this look sane?
>>
>> Yes, looks clean enough to me.
> 
> Very good!  Are you willing to take the patch in your tree?

Certainly, I'm in the middle of patch monkeying now anyway. Will queue
it up.


-- 
Jens Axboe

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Jens Axboe]

On 2010-11-08 14:55, Jens Axboe wrote:
>>>>> Take the RCU lock for read across acquiring the pointer to the task
>>>>> credentials and dereferencing it.
>>>>
>>>> Jens, does this look sane?
>>>
>>> Yes, looks clean enough to me.
>>
>> Very good!  Are you willing to take the patch in your tree?
> 
> Certainly, I'm in the middle of patch monkeying now anyway. Will queue
> it up.

The patch was buggy, I discovered that it does not do the proper
RCU unlock if we fail the uid match. Merged version here:

http://git.kernel.dk/?p=linux-2.6-block.git;a=commit;h=1a882abdbf9579ad0e5655f928e4ede30db301e6

-- 
Jens Axboe

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Daniel J Blueman]

On 9 November 2010 20:35, Jens Axboe <jaxboe@fusionio.com> wrote:
> On 2010-11-08 14:55, Jens Axboe wrote:
>>>>>> Take the RCU lock for read across acquiring the pointer to the task
>>>>>> credentials and dereferencing it.
>>>>>
>>>>> Jens, does this look sane?
>>>>
>>>> Yes, looks clean enough to me.
>>>
>>> Very good!  Are you willing to take the patch in your tree?
>>
>> Certainly, I'm in the middle of patch monkeying now anyway. Will queue
>> it up.
>
> The patch was buggy, I discovered that it does not do the proper
> RCU unlock if we fail the uid match. Merged version here:
>
> http://git.kernel.dk/?p=linux-2.6-block.git;a=commit;h=1a882abdbf9579ad0e5655f928e4ede30db301e6

Brown paper bag moment! Good catch, Jens.

Dan
-- 
Daniel J Blueman

Re: [2.6.37-rc1] sys_ioprio_set and RCU locking...

[Paul E. McKenney]

On Tue, Nov 09, 2010 at 09:35:10PM +0100, Jens Axboe wrote:
> On 2010-11-08 14:55, Jens Axboe wrote:
> >>>>> Take the RCU lock for read across acquiring the pointer to the task
> >>>>> credentials and dereferencing it.
> >>>>
> >>>> Jens, does this look sane?
> >>>
> >>> Yes, looks clean enough to me.
> >>
> >> Very good!  Are you willing to take the patch in your tree?
> > 
> > Certainly, I'm in the middle of patch monkeying now anyway. Will queue
> > it up.
> 
> The patch was buggy, I discovered that it does not do the proper
> RCU unlock if we fail the uid match. Merged version here:
> 
> http://git.kernel.dk/?p=linux-2.6-block.git;a=commit;h=1a882abdbf9579ad0e5655f928e4ede30db301e6

Ouch!!!  I missed that one entirely, good catch!!!

							Thanx, Paul