Interesting race condition...

Interesting race condition...

[Rob Landley]

I just saw a funky thing.  Here's the cut and past from the xterm...

[root@(none) root]# ps ax | grep hack
 9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
[root@(none) root]# ps ax | grep hack
 9966 pts/1    S      0:00 grep hack

Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
or the 2.6.7 kernel or what...

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Interesting race condition...

[Barry K. Nathan]

On Thu, Jul 22, 2004 at 10:04:46PM -0500, Rob Landley wrote:
> Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
> or the 2.6.7 kernel or what...

I've seen this too, on Fedora Core 2 and pretty recent (within the last
2 weeks) fc-devel. Hard enough to reproduce (and not enough of a bother
IMO) that I haven't filed it in Bugzilla though.

(Now that you mention Fedora Core, I can't remember seeing this with any
other distributions.)

-Barry K. Nathan <barryn@pobox.com>

Re: Interesting race condition...

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

On Fri, Jul 23, 2004 at 12:33:00AM -0700, Barry K. Nathan wrote:
> On Thu, Jul 22, 2004 at 10:04:46PM -0500, Rob Landley wrote:
> > Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
> > or the 2.6.7 kernel or what...
> 
> I've seen this too, on Fedora Core 2 and pretty recent (within the last
> 2 weeks) fc-devel. Hard enough to reproduce (and not enough of a bother
> IMO) that I haven't filed it in Bugzilla though.
> 
> (Now that you mention Fedora Core, I can't remember seeing this with any
> other distributions.)

   I've seen it on Debian unstable/sid, running a 32-bit userspace on
a 64-bit amd64 2.6.6 from kernel.org.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 1C335860 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- There is no dark side to the Moon, really. As a matter of ---    
                          fact,  it's all dark.                          

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Interesting race condition...

[P. Benie]

On Fri, 23 Jul 2004, Rob Landley wrote:

> I just saw a funky thing.  Here's the cut and past from the xterm...
>
> [root@(none) root]# ps ax | grep hack
>  9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
> [root@(none) root]# ps ax | grep hack
>  9966 pts/1    S      0:00 grep hack
>
> Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
> or the 2.6.7 kernel or what...

The race is in the shell's pipeline - the processes don't start at exactly
the same time, and sometimes ps has completed before the shell has
started grep. This is the expected behaviour.

Peter

Re: Interesting race condition...

[Rob Landley]

On Friday 23 July 2004 02:33, Barry K. Nathan wrote:
> On Thu, Jul 22, 2004 at 10:04:46PM -0500, Rob Landley wrote:
> > Seems like some kind of race condition, dunno if it's in Fedore Core 1's
> > ps or the 2.6.7 kernel or what...
>
> I've seen this too, on Fedora Core 2 and pretty recent (within the last
> 2 weeks) fc-devel. Hard enough to reproduce (and not enough of a bother
> IMO) that I haven't filed it in Bugzilla though.
>
> (Now that you mention Fedora Core, I can't remember seeing this with any
> other distributions.)
>
> -Barry K. Nathan <barryn@pobox.com>

Oh I can't reproduce it either.  (Maybe if I set some kind of loop and left it 
running for a few days...)

I just thought it was really odd.  There should be a null terminator before 
the environment variables, and it's getting stomped somehow.  Maybe it was 
right as the process was starting...?

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Interesting race condition...

[Rob Landley]

On Friday 23 July 2004 05:01, P. Benie wrote:
> On Fri, 23 Jul 2004, Rob Landley wrote:
> > I just saw a funky thing.  Here's the cut and past from the xterm...
> >
> > [root@(none) root]# ps ax | grep hack
> >  9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm
> > HISTSIZE=1000 USER=root
> > LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=
> >40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.
> >com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz
> >= [root@(none) root]# ps ax | grep hack
> >  9966 pts/1    S      0:00 grep hack
> >
> > Seems like some kind of race condition, dunno if it's in Fedore Core 1's
> > ps or the 2.6.7 kernel or what...
>
> The race is in the shell's pipeline - the processes don't start at exactly
> the same time, and sometimes ps has completed before the shell has
> started grep. This is the expected behaviour.

It's expected behavior for PS to show a process's environment variables as 
part of its command line?

> Peter

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Interesting race condition...

[P. Benie]

On Sat, 24 Jul 2004, Rob Landley wrote:
> It's expected behavior for PS to show a process's environment variables as
> part of its command line?

Duh! Need more coffee...

Peter

Re: Interesting race condition...

[Marc Ballarin]

Rob Landley <rob <at> landley.net> writes: 
 
> Oh I can't reproduce it either.  (Maybe if I set some kind of loop and left 
> it running for a few days...) 
 
I could reproduce it on an otherwise idle system (2 GHz Athlon, kernel 2.6.7). 
On a loaded system the bug did not occur, which certainly indicates a race 
condition. 
 
Using the following Bash script, the bug appeared 23 times in 122,221 
iterations: 
while [ 1 ];do 
        ps ax | grep hack >> TEST 
done 
 
The bug *seems* to be in bash, since an equivalent script in tcsh had no 
problems: 
while ( 1 ) 
        ps ax | grep hack >> TEST2 
end 
 
This issue has the potential to break a lot of shell scripts in an almost 
undebugable way. Should someone file a bug report via 'bashbug'? 
 
Marc 

Re: Interesting race condition...

[David Weinehall]

On Sat, Jul 24, 2004 at 01:40:59PM +0000, Marc Ballarin wrote:
> Rob Landley <rob <at> landley.net> writes: 
>  
> > Oh I can't reproduce it either.  (Maybe if I set some kind of loop and left 
> > it running for a few days...) 
>  
> I could reproduce it on an otherwise idle system (2 GHz Athlon, kernel 2.6.7). 
> On a loaded system the bug did not occur, which certainly indicates a race 
> condition. 
>  
> Using the following Bash script, the bug appeared 23 times in 122,221 
> iterations: 
> while [ 1 ];do 
>         ps ax | grep hack >> TEST 
> done 
>  
> The bug *seems* to be in bash, since an equivalent script in tcsh had no 
> problems: 
> while ( 1 ) 
>         ps ax | grep hack >> TEST2 
> end 
>  
> This issue has the potential to break a lot of shell scripts in an almost 
> undebugable way. Should someone file a bug report via 'bashbug'? 

I get this using posh as well, so it doesn't seem to be bash-specific.


Regards: David Weinehall
-- 
 /) David Weinehall <tao@acc.umu.se> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Re: Interesting race condition...

[Marc Ballarin]

Marc Ballarin <Ballarin.Marc <at> gmx.de> writes: 
 
Ok, I could repruduce this issue in bash and tcsh, using both, procps 
3.1.5/3.2.2 from procps.sourceforge.net and procps 2.0.16 from 
tech9.net/rml/procps. 
 
So, the bug is not in the shell (which is obvious, on second thought), but in 
the kernel, glibc or procps - or the combination thereof. 
 
I'm using kernel 2.6.7, gcc-3.3.3, and glibc-2.3.3 with NPTL on Gentoo. 
 
Regards 

Re: Interesting race condition...

[Bill Davidsen]

Rob Landley wrote:
> On Friday 23 July 2004 05:01, P. Benie wrote:
> 
>>On Fri, 23 Jul 2004, Rob Landley wrote:
>>
>>>I just saw a funky thing.  Here's the cut and past from the xterm...
>>>
>>>[root@(none) root]# ps ax | grep hack
>>> 9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm
>>>HISTSIZE=1000 USER=root
>>>LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=
>>>40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.
>>>com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz
>>>= [root@(none) root]# ps ax | grep hack
>>> 9966 pts/1    S      0:00 grep hack
>>>
>>>Seems like some kind of race condition, dunno if it's in Fedore Core 1's
>>>ps or the 2.6.7 kernel or what...
>>
>>The race is in the shell's pipeline - the processes don't start at exactly
>>the same time, and sometimes ps has completed before the shell has
>>started grep. This is the expected behaviour.
> 
> 
> It's expected behavior for PS to show a process's environment variables as 
> part of its command line?

When piped. For instance
   ps eaxf
does not, while
   ps eaxf | cat
does, at least on my systems. I tried on RHEL AS3.0 thru a four year old 
version of Slackware, all did the same thing.


-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: Interesting race condition...

[Paul Jackson]

Bill writes:
> When piped. For instance
>    ps eaxf
> does not [show environ], while
>    ps eaxf | cat
> does

Neither the behaviour I'm seeing on my SuSE 8.2 box, and the code I see
in some random procps-3.2.1 I happened to have laying around, agree with
your description.

Rather, both behaviour and code have the 'e' option as a BSD option
(applicable if no '-' option flag) requesting that the environment be
shown, and I get the environment so shown, regardless of whether the
output is a pipe or a tty.

... This subthread is of course off-topic to Rob Landley's
original post - he wasn't using the 'e' option.

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373

Re: Interesting race condition...

[Paul Jackson]

Rob wrote:
> I just saw a funky thing.  Here's the cut and past from the xterm...

Can you reproduce this by cat'ing /proc/<pid>/cmdline?  Can you get a
dump of the proc cmdline file to leak the environment sometimes?

It is this file that 'ps' is dumping for these options.  Adding the
'e' option would also dump the /proc/<pid>/environ file (if readable).

But you aren't adding 'e', so presumably the environment is "leaking"
into the the cmdline file.

I suspect a kernel bug here - the ps code seems rather obvious and
unimpeachable.

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373

Re: Interesting race condition...

[Marc Ballarin]

On Wed, 28 Jul 2004 01:05:46 -0700
Paul Jackson <pj@sgi.com> wrote:

> Rob wrote:
> > I just saw a funky thing.  Here's the cut and past from the xterm...
> 
> Can you reproduce this by cat'ing /proc/<pid>/cmdline?  Can you get a
> dump of the proc cmdline file to leak the environment sometimes?
> 
> It is this file that 'ps' is dumping for these options.  Adding the
> 'e' option would also dump the /proc/<pid>/environ file (if readable).
> 
> But you aren't adding 'e', so presumably the environment is "leaking"
> into the the cmdline file.
> 
> I suspect a kernel bug here - the ps code seems rather obvious and
> unimpeachable.
> 

I ran the following loop for a while (> 9 million times) and could not
reproduce the bug, but that might just be coincidence.
Conditions were the same as in my other, succesful test.

while [ 1 ];do
        cat /proc/self/cmdline >> TEST
done

Marc

Re: Interesting race condition...

[Rob Landley]

On Wednesday 28 July 2004 03:05, Paul Jackson wrote:
> Rob wrote:
> > I just saw a funky thing.  Here's the cut and past from the xterm...
>
> Can you reproduce this by cat'ing /proc/<pid>/cmdline?  Can you get a
> dump of the proc cmdline file to leak the environment sometimes?

I saw it exactly once, I'm afraid.  Somebody else said they also saw it...  
once.  It smelled to me like a race condition, with the process 
starting/exiting right as ps looked at it, but I haven't seen it again.  (I 
could run the command in a loop overnight...)

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Interesting race condition...

[Rob Landley]

On Wednesday 28 July 2004 06:54, Marc Ballarin wrote:
> On Wed, 28 Jul 2004 01:05:46 -0700
>
> Paul Jackson <pj@sgi.com> wrote:
> > Rob wrote:
> > > I just saw a funky thing.  Here's the cut and past from the xterm...
> >
> > Can you reproduce this by cat'ing /proc/<pid>/cmdline?  Can you get a
> > dump of the proc cmdline file to leak the environment sometimes?
> >
> > It is this file that 'ps' is dumping for these options.  Adding the
> > 'e' option would also dump the /proc/<pid>/environ file (if readable).
> >
> > But you aren't adding 'e', so presumably the environment is "leaking"
> > into the the cmdline file.
> >
> > I suspect a kernel bug here - the ps code seems rather obvious and
> > unimpeachable.
>
> I ran the following loop for a while (> 9 million times) and could not
> reproduce the bug, but that might just be coincidence.
> Conditions were the same as in my other, succesful test.
>
> while [ 1 ];do
>         cat /proc/self/cmdline >> TEST
> done
>
> Marc

I might have actually just killed the process I was grepping for (I was 
grepping to see if it had actually gone away yet).  Dunno if that helps...

I haven't been able to reproduce it either.  I think the pipe to grep (or 
something similar) is important because your test isn't doing what mine did: 
one process was looking at another process at the instant of process creation 
(or perhaps exit).  Maybe the appropriate null terminator is written out 
after the process goes live?

This is a UP system, but I have preemption on...

Rob
-- 
www.linucon.org: Linux Expo and Science Fiction Convention
October 8-10, 2004 in Austin Texas.  (I'm the con chair.)

Re: Interesting race condition...

[Tristan Wibberley]

On Wed, 28 Jul 2004 11:42:37 -0500, Rob Landley wrote:

> On Wednesday 28 July 2004 03:05, Paul Jackson wrote:
>> Rob wrote:
>> > I just saw a funky thing.  Here's the cut and past from the xterm...
>>
>> Can you reproduce this by cat'ing /proc/<pid>/cmdline?  Can you get a
>> dump of the proc cmdline file to leak the environment sometimes?
> 
> I saw it exactly once, I'm afraid.  Somebody else said they also saw it...  
> once.  It smelled to me like a race condition, with the process 
> starting/exiting right as ps looked at it, but I haven't seen it again.  (I 
> could run the command in a loop overnight...)

I think I have seen this exactly once. I ran helixplayer from a bash
command line and it quit, immediately spouting something about how it
couldn't find a file whose name looked remarkably similar to
some environment variables. I had put it down to screwed terminal settings
so I couldn't see I had pasted something in, and I immediately issued a
reset, so I have no more information I'm afraid.

-- 
I have a little sig...

Re: Interesting race condition...

[Roger Luethi]

On Thu, 22 Jul 2004 22:04:46 -0500, Rob Landley wrote:
> I just saw a funky thing.  Here's the cut and past from the xterm...
> 
> [root@(none) root]# ps ax | grep hack
>  9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
> [root@(none) root]# ps ax | grep hack
>  9966 pts/1    S      0:00 grep hack
> 
> Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
> or the 2.6.7 kernel or what...

If somebody posted a solution for this, I didn't see it. There's a race in
the kernel, and considering the permissions on /proc/PID/{cmdline,environ}
a security bug as well: If you win the race with a starting process, you
can read its environment.

This should plug the hole. Can you give it a spin?

Roger

--- linux-2.6.8-rc2-bk1/fs/proc/base.c.orig	2004-07-30 01:43:23.535967505 +0200
+++ linux-2.6.8-rc2-bk1/fs/proc/base.c	2004-07-30 01:43:27.428303752 +0200
@@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
 	struct mm_struct *mm = get_task_mm(task);
 	if (!mm)
 		goto out;
+	if (!mm->arg_end)
+		goto out;	/* Shh! No looking before we're done */
 
  	len = mm->arg_end - mm->arg_start;
  

Re: Interesting race condition...

[Jesper Juhl]

On Fri, 30 Jul 2004, Roger Luethi wrote:

> On Thu, 22 Jul 2004 22:04:46 -0500, Rob Landley wrote:
>> I just saw a funky thing.  Here's the cut and past from the xterm...
>>
>> [root@(none) root]# ps ax | grep hack
>>  9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
>> [root@(none) root]# ps ax | grep hack
>>  9966 pts/1    S      0:00 grep hack
>>
>> Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
>> or the 2.6.7 kernel or what...
>
> If somebody posted a solution for this, I didn't see it. There's a race in
> the kernel, and considering the permissions on /proc/PID/{cmdline,environ}
> a security bug as well: If you win the race with a starting process, you
> can read its environment.
>
> This should plug the hole. Can you give it a spin?
>
> Roger
>
> --- linux-2.6.8-rc2-bk1/fs/proc/base.c.orig	2004-07-30 01:43:23.535967505 +0200
> +++ linux-2.6.8-rc2-bk1/fs/proc/base.c	2004-07-30 01:43:27.428303752 +0200
> @@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
> 	struct mm_struct *mm = get_task_mm(task);
> 	if (!mm)
> 		goto out;
> +	if (!mm->arg_end)
> +		goto out;	/* Shh! No looking before we're done */
>
>  	len = mm->arg_end - mm->arg_start;
>

I might be reading the code wrong or not be fully aware of the 
circumstances here, so please bear with me, but I took a look at 
get_task_mm() and it can return NULL. If it does you'd end up 
dereferencing a NULL pointer with "if (!mm->arg_end)" - or is that 
garanteed not to happen here?

Wouldn't something like

  if (!mm || !mm->arg_end)
 	goto out;

be safer?

Just my two pennies..


- Jesper Juhl <juhl-lkml@dif.dk>

Re: Interesting race condition...

[Jesper Juhl]

On Fri, 30 Jul 2004, Jesper Juhl wrote:

> On Fri, 30 Jul 2004, Roger Luethi wrote:
>
>> On Thu, 22 Jul 2004 22:04:46 -0500, Rob Landley wrote:
>>> I just saw a funky thing.  Here's the cut and past from the xterm...
>>> 
>>> [root@(none) root]# ps ax | grep hack
>>>  9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash 
>>> TERM=xterm HISTSIZE=1000 USER=root 
>>> LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
>>> [root@(none) root]# ps ax | grep hack
>>>  9966 pts/1    S      0:00 grep hack
>>> 
>>> Seems like some kind of race condition, dunno if it's in Fedore Core 1's 
>>> ps
>>> or the 2.6.7 kernel or what...
>> 
>> If somebody posted a solution for this, I didn't see it. There's a race in
>> the kernel, and considering the permissions on /proc/PID/{cmdline,environ}
>> a security bug as well: If you win the race with a starting process, you
>> can read its environment.
>> 
>> This should plug the hole. Can you give it a spin?
>> 
>> Roger
>> 
>> --- linux-2.6.8-rc2-bk1/fs/proc/base.c.orig	2004-07-30 01:43:23.535967505 
>> +0200
>> +++ linux-2.6.8-rc2-bk1/fs/proc/base.c	2004-07-30 01:43:27.428303752 
>> +0200
>> @@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
>> 	struct mm_struct *mm = get_task_mm(task);
>> 	if (!mm)
>> 		goto out;
>> +	if (!mm->arg_end)
>> +		goto out;	/* Shh! No looking before we're done */
>> 
>>  	len = mm->arg_end - mm->arg_start;
>> 
>
> I might be reading the code wrong or not be fully aware of the circumstances 
> here, so please bear with me, but I took a look at get_task_mm() and it can 
> return NULL. If it does you'd end up dereferencing a NULL pointer with "if 
> (!mm->arg_end)" - or is that garanteed not to happen here?
>
> Wouldn't something like
>
> if (!mm || !mm->arg_end)
> 	goto out;
>
> be safer?
>
Arrgh, that's what happens when you try to read patches at 02:25
For some reason my eyes read the patch as

-    if (!mm)
-            goto out;
+    if (!mm->arg_end)
+            goto out;       /* Shh! No looking before we're done */

but that's not what it looks like :(  Sorry about that. There's no problem 
except for me being too sleepy.

/Jesper Juhl

Re: Interesting race condition...

[Marc Ballarin]

On Fri, 30 Jul 2004 01:56:54 +0200
Roger Luethi <rl@hellgate.ch> wrote:

> If somebody posted a solution for this, I didn't see it. There's a race
> in the kernel, and considering the permissions on
> /proc/PID/{cmdline,environ} a security bug as well: If you win the race
> with a starting process, you can read its environment.
> 
> This should plug the hole. Can you give it a spin?
> 
> Roger
> 
> --- linux-2.6.8-rc2-bk1/fs/proc/base.c.orig	2004-07-30 01:43:23.535967505 +0200
> +++ linux-2.6.8-rc2-bk1/fs/proc/base.c	2004-07-30 01:43:27.428303752 +0200
> @@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
>  	struct mm_struct *mm = get_task_mm(task);
>  	if (!mm)
>  		goto out;
> +	if (!mm->arg_end)
> +		goto out;	/* Shh! No looking before we're done */
>  
>   	len = mm->arg_end - mm->arg_start;
>   

Yes, this seems to fix it. First I replaced "goto out" with a printk, and
the printks matched the occurence of the bug.
However, I got multiple printks per bug (between 2 and 7). Is that
supposed to happen?

Anyway, I've added back the "goto out" (after printk), and the bug no
longer
occurs. The printk still happens, so this code path does get hit.


Regards, and thanks for the fix

Re: Interesting race condition...

[Roger Luethi]

On Fri, 30 Jul 2004 10:27:26 +0200, Marc Ballarin wrote:
> > --- linux-2.6.8-rc2-bk1/fs/proc/base.c.orig	2004-07-30 01:43:23.535967505 +0200
> > +++ linux-2.6.8-rc2-bk1/fs/proc/base.c	2004-07-30 01:43:27.428303752 +0200
> > @@ -329,6 +329,8 @@ static int proc_pid_cmdline(struct task_
> >  	struct mm_struct *mm = get_task_mm(task);
> >  	if (!mm)
> >  		goto out;
> > +	if (!mm->arg_end)
> > +		goto out;	/* Shh! No looking before we're done */
> >  
> >   	len = mm->arg_end - mm->arg_start;
> >   
> 
> Yes, this seems to fix it. First I replaced "goto out" with a printk, and
> the printks matched the occurence of the bug.
> However, I got multiple printks per bug (between 2 and 7). Is that
> supposed to happen?

Every time we proceed through proc_pid_cmdline with mm->arg_end == 0 is a
bug. AFAICT anyway. Whether you see an "occurence of the bug" depends on
how you measure it. Of course you are wecome to add more printks following
the code path to see how and when it propagates to where you are catching
it.

Roger

RE: Interesting race condition...

[Robert White]

Using procps version 2.0.7 the inclusion of "e" in the arguments is documented to
return environment of the process.

So the below issues with "ps eaxf" are only surprising in that you claim the pipe is
necessary to see the environment.  Ps has "always" done slightly different things
when used at the head of a pipe.  In particular it will trim it's right side when not
in a pipe to keep the display readable, while in a pipe the lines will be "as long as
necessary."

The question of why the original poster was getting the environment when only using
"ps ax" is interesting.  I'd look for PS_PERSONALITY (etc) in the ps-executing
environment.  [consult your manual.]

And so on...

Rob.

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-owner@vger.kernel.org]
On Behalf Of Bill Davidsen
Sent: Tuesday, July 27, 2004 1:40 PM
To: linux-kernel@vger.kernel.org
Subject: Re: Interesting race condition...

Rob Landley wrote:
> On Friday 23 July 2004 05:01, P. Benie wrote:
> 
>>On Fri, 23 Jul 2004, Rob Landley wrote:
>>
>>>I just saw a funky thing.  Here's the cut and past from the xterm...
>>>
>>>[root@(none) root]# ps ax | grep hack
>>> 9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm
>>>HISTSIZE=1000 USER=root
>>>LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=
>>>40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.
>>>com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz
>>>= [root@(none) root]# ps ax | grep hack
>>> 9966 pts/1    S      0:00 grep hack
>>>
>>>Seems like some kind of race condition, dunno if it's in Fedore Core 1's
>>>ps or the 2.6.7 kernel or what...
>>
>>The race is in the shell's pipeline - the processes don't start at exactly
>>the same time, and sometimes ps has completed before the shell has
>>started grep. This is the expected behaviour.
> 
> 
> It's expected behavior for PS to show a process's environment variables as 
> part of its command line?

When piped. For instance
   ps eaxf
does not, while
   ps eaxf | cat
does, at least on my systems. I tried on RHEL AS3.0 thru a four year old 
version of Slackware, all did the same thing.


-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Interesting race condition...

[Roger Luethi]

On Wed, 04 Aug 2004 13:03:50 -0700, Robert White wrote:
> Using procps version 2.0.7 the inclusion of "e" in the arguments is
> documented to return environment of the process.

Not the environment of somebody else's process, though. But that is
currently possible in mainline, the problem is not with ps but with
the kernel.

> The question of why the original poster was getting the environment when
> only using "ps ax" is interesting.  I'd look for PS_PERSONALITY (etc) in

Basically, if anyone reads /proc/pid/cmdline early enough, when
mm->arg_end is still 0, the kernel will blast out the process environment
through that interface. Thus, you get the data of /proc/pid/environ
without the access restrictions of that file. Not good if you happen
to pass sensitive information using environment variables.

Check out the patch I posted earlier in this thread.

Roger

Re: Interesting race condition...

[Lee Revell]

On Thu, 2004-07-29 at 19:56, Roger Luethi wrote:
> On Thu, 22 Jul 2004 22:04:46 -0500, Rob Landley wrote:
> > I just saw a funky thing.  Here's the cut and past from the xterm...
> > 
> > [root@(none) root]# ps ax | grep hack
> >  9964 pts/1    R      0:00 grep hack HOSTNAME= SHELL=/bin/bash TERM=xterm HISTSIZE=1000 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=
> > [root@(none) root]# ps ax | grep hack
> >  9966 pts/1    S      0:00 grep hack
> > 
> > Seems like some kind of race condition, dunno if it's in Fedore Core 1's ps
> > or the 2.6.7 kernel or what...
> 
> If somebody posted a solution for this, I didn't see it. There's a race in
> the kernel, and considering the permissions on /proc/PID/{cmdline,environ}
> a security bug as well: If you win the race with a starting process, you
> can read its environment.
> 
> This should plug the hole. Can you give it a spin?
> 

Was this ever merged?  I just hit this bug with 2.6.8.1:

rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10509  0.0  0.0     4    4 pts/1    R+   06:09   0:00 grep jack SSH_AGENT_PID=667 TERM=rxvt SHELL=/bin/bash CVSROOT=:pserver:anonymous@cvs.alsa-project.org:/cvsroot/alsa GTK_RC_FILES=/etc/gtk/gtkrc:/home/rlrevell/.gtkrc-1.2-gnome2 WINDOWID=33554437 USER=rlrevell LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.ogg=01;35:*.mp3=01;35:*.wav=01;35: GNOME_KEYRING_SOCKET=/tmp/keyring-oFgJ2h/socket SSH_AUTH_SOCK=/tmp/ssh-JvFuVGV623/agent.623 SESSION_MANAGER=local/mindpipe:/tmp/.ICE-unix/623 USERNAME=rlrevell DESKTOP_SESSION=default PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games PWD=/home/rlrevell GDMSESSION=default HISTCONTROL=ignoredups HOME=/home/rlrevell SHLVL=1 GNOME_DESKTOP_SESSION_ID=Default LOGNAME=rlrevell DISPLAY=:0.0 COLORTERM=rxvt XAUTHORITY=/home/rlrevell/.Xauthority _=/bin/grep /bin/grep    
rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10511  0.0  0.0  1576  464 pts/1    S+   06:09   0:00 grep jack
rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10513  0.0  0.0   240  124 pts/1    R+   06:09   0:00 grep jack
rlrevell@mindpipe:~$ ps auxww | grep jack
rlrevell 10498  0.1  5.9 28656 28624 pts/0   SLl+ 06:07   0:00 jackd -n 1000000 --realtime -d alsa --rate 48000 -D -P hw:0,0 -C hw:0,2 -p 32 -S
rlrevell 10502  0.1  2.3 11432 11432 pts/2   SLl+ 06:07   0:00 jack_simple_client foo
rlrevell 10515  0.0  0.0  1576  464 pts/1    S+   06:09   0:00 grep jack
rlrevell@mindpipe:~$ ps auxww | grep jack

Lee

Re: Interesting race condition...

[Marc Ballarin]

On Fri, 20 Aug 2004 06:15:09 -0400
Lee Revell <rlrevell@joe-job.com> wrote:

> 
> Was this ever merged?  I just hit this bug with 2.6.8.1:
>

It's only in -mm series so far.

Regards