From: Grzegorz Nosek <grzegorz.nosek@gmail.com>
To: Andrew Morton <akpm@osdl.org>
Cc: linux-kernel@vger.kernel.org
Subject: Re: [PATCH] race condition in procfs
Date: Tue, 29 Nov 2005 09:38:29 +0100 [thread overview]
Message-ID: <121a28810511290038h37067fecx@mail.gmail.com> (raw)
In-Reply-To: <20051129000916.6306da8b.akpm@osdl.org>
[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]
2005/11/29, Andrew Morton <akpm@osdl.org>:
> > I found a race condition in procfs on SMP systems. The result is an
> > oops in processes like pidof. Apparently ->proc_read() gets passed a
> > potentially NULL pointer.
>
> Do you know what the race is?
Apparently it's a race between deleting a process and accessing its
/proc/pid entries. It came out in pidof while it was accessing
/proc/pid/stat (fs/proc/array.c:do_task_stat crashed on first
instruction - it was an inline function accessing task->state,
get_task_state IIRC). oops (with vserver history data - I'm using a
patch mentioned below) is attached.
>
> How does one reproduce it?
I managed to reproduce it (although not reliably) during high CPU load
and I/O (parallel kernel compiles) on SMP systems with the vserver
patch (http://linux-vserver.org, the exact patch is
http://vserver.13thfloor.at/Experimental/patch-2.6.14.2-vs2.1.0-rc8.diff),
but the vserver maintainer pointed out that it probably is a mainline
issue. We're not using 2.6 systems too much except for the vserver
test beds so I cannot tell if it happens on vanilla kernels.
>
> > The following micro-patch seems to fix it.
>
> It might be right, or it might be a workaround..
>
I'm not a kernel guru so it's just my proposal. Can it break anything?
An alternative _might_ be somewhat coarser task_struct locking
(do_task_stat grabs a spinlock but then it's already too late).
However, if no "right" solution appears, I'll keep using my two-liner
because it seems to help, at least in my setup.
Best regards,
Grzegorz Nosek
[-- Attachment #2: oops.s35 --]
[-- Type: application/octet-stream, Size: 8426 bytes --]
Nov 27 00:15:26 s35 [43281574.240000] Unable to handle kernel NULL pointer dereference
Nov 27 00:15:26 s35 at virtual address 00000000
Nov 27 00:15:26 s35 [43281574.240000] printing eip:
Nov 27 00:15:26 s35 [43281574.240000] a01b50eb
Nov 27 00:15:26 s35 [43281574.240000] *pde = 00000000
Nov 27 00:15:26 s35 [43281574.240000] Oops: 0000 [#1]
Nov 27 00:15:26 s35 [43281574.240000] SMP
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Modules linked in:
Nov 27 00:15:26 s35 ipt_owner
Nov 27 00:15:26 s35 ipt_state
Nov 27 00:15:26 s35 iptable_filter
Nov 27 00:15:26 s35 netconsole
Nov 27 00:15:26 s35 uhci_hcd
Nov 27 00:15:26 s35 ohci_hcd
Nov 27 00:15:26 s35 ehci_hcd
Nov 27 00:15:26 s35 usbcore
Nov 27 00:15:26 s35 ip_conntrack_ftp
Nov 27 00:15:26 s35 ip_conntrack
Nov 27 00:15:26 s35 forcedeth
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] CPU: 1
Nov 27 00:15:26 s35 [43281574.240000] EIP: 0060:[<a01b50eb>] Not tainted VLI
Nov 27 00:15:26 s35 [43281574.240000] EFLAGS: 00010257 (2.6.14.2amd64smp.17)
Nov 27 00:15:26 s35 [43281574.240000] EIP is at do_task_stat+0x8b/0x890
Nov 27 00:15:26 s35 [43281574.240000] eax: 00000000 ebx: 00000000 ecx: a0601700 edx: c804ad48
Nov 27 00:15:26 s35 [43281574.240000] esi: b3fbe000 edi: f666aa70 ebp: d7e65f20 esp: d7e65da0
Nov 27 00:15:26 s35 [43281574.240000] ds: 007b es: 007b ss: 0068
Nov 27 00:15:26 s35 [43281574.240000] Process pidof (pid: 4723, threadinfo=d7e64000 task=e24e7550)
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Stack:
Nov 27 00:15:26 s35 a01b1e2e
Nov 27 00:15:26 s35 f666aa70
Nov 27 00:15:26 s35 d7e65f28
Nov 27 00:15:26 s35 a8cab11c
Nov 27 00:15:26 s35 d7e65e24
Nov 27 00:15:26 s35 d7e65de8
Nov 27 00:15:26 s35 a0184934
Nov 27 00:15:26 s35 d7e65e24
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000]
Nov 27 00:15:26 s35 a8cab544
Nov 27 00:15:26 s35 d7e65de8
Nov 27 00:15:26 s35 a019090d
Nov 27 00:15:26 s35 a8cab544
Nov 27 00:15:26 s35 a0720a00
Nov 27 00:15:26 s35 d7e65df8
Nov 27 00:15:26 s35 a2227140
Nov 27 00:15:26 s35 00000000
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000]
Nov 27 00:15:26 s35 00000000
Nov 27 00:15:26 s35 d7e65e2c
Nov 27 00:15:26 s35 d7e65e48
Nov 27 00:15:26 s35 a0185664
Nov 27 00:15:26 s35 a8cab544
Nov 27 00:15:26 s35 d7e65e2c
Nov 27 00:15:26 s35 d7e65e24
Nov 27 00:15:26 s35 c94ff00b
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Call Trace:
Nov 27 00:15:26 s35 [43281574.240000] [<a0103e9f>]
Nov 27 00:15:26 s35 show_stack+0x7f/0xa0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a010403d>]
Nov 27 00:15:26 s35 show_registers+0x15d/0x1d0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a0104252>]
Nov 27 00:15:26 s35 die+0x112/0x1c0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a055c2b9>]
Nov 27 00:15:26 s35 do_page_fault+0x3d9/0x650
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a0103b53>]
Nov 27 00:15:26 s35 error_code+0x4f/0x54
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a01b5940>]
Nov 27 00:15:26 s35 proc_tgid_stat+0x20/0x30
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a01b0f75>]
Nov 27 00:15:26 s35 proc_info_read+0x55/0xa0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a0174d68>]
Nov 27 00:15:26 s35 vfs_read+0x198/0x1a0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a017506b>]
Nov 27 00:15:26 s35 sys_read+0x4b/0x80
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a010302d>]
Nov 27 00:15:26 s35 syscall_call+0x7/0xb
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Code:
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 85
Nov 27 00:15:26 s35 6c
Nov 27 00:15:26 s35 ff
Nov 27 00:15:26 s35 ff
Nov 27 00:15:26 s35 ff
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 8b
Nov 27 00:15:26 s35 07
Nov 27 00:15:26 s35 8b
Nov 27 00:15:26 s35 9f
Nov 27 00:15:26 s35 84
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 25
Nov 27 00:15:26 s35 8f
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 83
Nov 27 00:15:26 s35 e3
Nov 27 00:15:26 s35 30
Nov 27 00:15:26 s35 09
Nov 27 00:15:26 s35 d8
Nov 27 00:15:26 s35 eb
Nov 27 00:15:26 s35 05
Nov 27 00:15:26 s35 83
Nov 27 00:15:26 s35 c1
Nov 27 00:15:26 s35 04
Nov 27 00:15:26 s35 d1
Nov 27 00:15:26 s35 e8
Nov 27 00:15:26 s35 75
Nov 27 00:15:26 s35 f9
Nov 27 00:15:26 s35 8b
Nov 27 00:15:26 s35 01
Nov 27 00:15:26 s35 unparseable log message: "<0f> "
Nov 27 00:15:26 s35 b6
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 45
Nov 27 00:15:26 s35 c8
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 45
Nov 27 00:15:26 s35 cc
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 45
Nov 27 00:15:26 s35 d0
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] History: SEQ: 3ddca14 NR_CPUS: 8
Nov 27 00:15:26 s35 [43281574.240000] (#c9ae,*0):a04d546e set_vx_info f6e48000[#830,190.71] @f4fcf4e8
Nov 27 00:15:26 s35 [43281574.240000] (#c964,*1):a013ac82 release_vx_info f6e48000[#830,190.74] @c5cdb030
Nov 27 00:15:26 s35 [43281574.240000] (#ca13,*0):a04d40b2 clr_vx_info f6e48000[#830,188.71] @db738068
Nov 27 00:15:26 s35 [43281574.240000] (#ca14,*1):a0104140 oops
Nov 27 00:15:26 s35 [43281574.240000] (#ca12,*0):a04d40b2 clr_vx_info f6e48000[#830,189.71] @db739b68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0f,*1):a011c57c clr_vx_info f6e1e000[#831,151.39] @f6f1bad0
Nov 27 00:15:26 s35 [43281574.240000] (#ca11,*0):a04d40b2 clr_vx_info f6e48000[#830,190.71] @ad3a16e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca0e,*1):a011c45c set_vx_info f6e1e000[#831,150.39] @f6f1a210
Nov 27 00:15:26 s35 [43281574.240000] (#ca10,*0):a04d40b2 clr_vx_info f6e48000[#830,191.71] @ed1a7b68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0d,*1):a04d40b2 clr_vx_info f6e48000[#830,192.71] @b6b98ae8
Nov 27 00:15:26 s35 [43281574.240000] (#ca03,*0):a04d4556 set_vx_info f6e48000[#830,190.71] @ba389268
Nov 27 00:15:26 s35 [43281574.240000] (#ca0c,*1):a011ddac claim_vx_info f6e1e000[#831,150.38] @e24e7550
Nov 27 00:15:26 s35 [43281574.240000] (#ca02,*0):a04d40b2 clr_vx_info f6e48000[#830,191.71] @b6b98d68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0b,*1):a011c45c set_vx_info f6e1e000[#831,149.38] @f6f1bad0
Nov 27 00:15:26 s35 [43281574.240000] (#ca01,*0):a04d546e set_vx_info f6e48000[#830,190.71] @b6b98d68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0a,*1):a011d38c init_vx_info f6e1e000[#831,148.38] @e24e79f8
Nov 27 00:15:26 s35 [43281574.240000] (#ca00,*0):a04d40b2 clr_vx_info f6e48000[#830,191.71] @f4fcf4e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca09,*1):a011bf10 clr_vx_info f6e1e000[#831,149.38] @e24e79f8
Nov 27 00:15:26 s35 [43281574.240000] (#c9ff,*0):a04d40b2 clr_vx_info f6e48000[#830,192.71] @f56bbde8
Nov 27 00:15:26 s35 [43281574.240000] (#ca08,*1):a04d546e set_vx_info f6e48000[#830,191.71] @b6b98ae8
Nov 27 00:15:26 s35 [43281574.240000] (#c9fe,*0):a04d546e set_vx_info f6e48000[#830,191.71] @f56bbde8
Nov 27 00:15:26 s35 [43281574.240000] (#ca07,*1):a04d40b2 clr_vx_info f6e48000[#830,192.71] @cbec5068
Nov 27 00:15:26 s35 [43281574.240000] (#c9fd,*0):a04d546e set_vx_info f6e48000[#830,190.71] @f4fcf4e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca06,*1):a04d40b2 clr_vx_info f6e48000[#830,193.71] @cbec5ba8
Nov 27 00:15:26 s35 [43281574.240000] (#c9fc,*0):a04d4556 set_vx_info f6e48000[#830,189.71] @ad3a16e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca05,*1):a04d546e set_vx_info f6e48000[#830,192.71] @cbec5ba8
Nov 27 00:15:26 s35 [43281574.240000] (#c9fb,*0):a04d40b2 clr_vx_info f6e48000[#830,190.71] @ad3a16e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca04,*1):a04d546e set_vx_info f6e48000[#830,191.71] @cbec5068
Nov 27 00:15:26 s35 [43281574.240000] (#c9f7,*0):a011c57c clr_vx_info f6e1e000[#831,148.37] @f6f1a790
Nov 27 00:15:26 s35 [43281574.240000] (#c9fa,*1):a011ddac claim_vx_info f6e1e000[#831,149.37] @e4856550
next prev parent reply other threads:[~2005-11-29 8:38 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-11-29 7:17 [PATCH] race condition in procfs Grzegorz Nosek
2005-11-29 8:09 ` Andrew Morton
2005-11-29 8:38 ` Grzegorz Nosek [this message]
2005-11-29 13:25 ` Grzegorz Nosek
2005-11-29 14:04 ` Grzegorz Nosek
2005-11-29 14:28 ` Steven Rostedt
2005-11-29 14:39 ` Grzegorz Nosek
2005-11-29 14:49 ` Steven Rostedt
2005-11-30 14:41 ` Grzegorz Nosek
2005-11-30 15:14 ` Steven Rostedt
2005-11-30 15:29 ` Grzegorz Nosek
2005-11-30 16:25 ` Steven Rostedt
2005-11-30 17:23 ` Grzegorz Nosek
2005-12-01 20:38 ` Grzegorz Nosek
2005-11-29 15:22 ` [PATCH] shrinks dentry struct Eric Dumazet
2005-11-30 2:06 ` Paul Jackson
2005-11-30 2:14 ` Andrew Morton
2005-11-30 2:43 ` Paul Jackson
2005-11-30 6:56 ` Hugh Dickins
2005-12-03 1:15 ` [PATCH] remove unused blkp field in percpu_data Eric Dumazet
2005-12-13 18:03 ` [PATCH] shrinks dentry struct Paul E. McKenney
2005-12-13 18:24 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=121a28810511290038h37067fecx@mail.gmail.com \
--to=grzegorz.nosek@gmail.com \
--cc=akpm@osdl.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox