* Re: [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes
2019-01-02 20:59 [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes Andrei Vagin
@ 2019-01-02 21:16 ` Andrei Vagin
2019-01-02 21:17 ` Andrei Vagin
2019-01-08 16:50 ` Oleg Nesterov
2 siblings, 0 replies; 7+ messages in thread
From: Andrei Vagin @ 2019-01-02 21:16 UTC (permalink / raw)
To: Andrew Morton, Oleg Nesterov; +Cc: linux-kernel, Eric W. Biederman
[-- Attachment #1: Type: text/plain, Size: 2980 bytes --]
Hello,
The reproducer for this problem is attached to this message.
Below you can find its effect without this fix:
We have one alive process which stucks in
zap_pid_ns_processes:
$ ps axf
...
11831 pts/0 S 0:00 [ptrace_pidns]
11833 pts/0 Zl 0:00 \_ [ptrace_pidns] <defunct>
$ cat /proc/11831/stack
[<0>] do_wait+0x1fa/0x2c0
[<0>] kernel_wait4+0x9e/0x150
[<0>] zap_pid_ns_processes+0x17d/0x270
[<0>] do_exit+0xa15/0xbd0
[<0>] do_group_exit+0x47/0xc0
[<0>] get_signal+0x28c/0x850
[<0>] do_signal+0x36/0x630
[<0>] exit_to_usermode_loop+0x62/0xc0
[<0>] prepare_exit_to_usermode+0xb4/0xe0
[<0>] retint_user+0x8/0x18
[<0>] 0xffffffffffffffff
The child process has two threads which were ptraced by parent:
$ ls /proc/11833/task/
11833 11834
The parent can't wait the child, becase a thread group isn't empty, but
a thread is in the dead list:
$ cat /proc/1183{1,3,4}/status | grep '\(NSpid\|TracerPid\|State\)'
State: S (sleeping)
TracerPid: 0
NSpid: 11831 1
State: Z (zombie)
TracerPid: 0
NSpid: 11833 2
State: X (dead)
TracerPid: 0
NSpid: 11834 3
On Wed, Jan 02, 2019 at 12:59:39PM -0800, Andrei Vagin wrote:
> Currently, exit_ptrace() adds all ptraced tasks in a dead list, than
> zap_pid_ns_processes() waits all tasks in a current pid ns, and only
> then tasks from the dead list are released.
>
> zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In
> this case, we will have one unkillable process with one or more dead
> children.
>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: "Eric W. Biederman" <ebiederm@xmission.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Signed-off-by: Andrei Vagin <avagin@gmail.com>
> ---
> kernel/exit.c | 15 +++++++++++----
> 1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/kernel/exit.c b/kernel/exit.c
> index 0e21e6d21f35..ccaa6f6549ba 100644
> --- a/kernel/exit.c
> +++ b/kernel/exit.c
> @@ -664,9 +664,6 @@ static void forget_original_parent(struct task_struct *father,
> {
> struct task_struct *p, *t, *reaper;
>
> - if (unlikely(!list_empty(&father->ptraced)))
> - exit_ptrace(father, dead);
> -
> /* Can drop and reacquire tasklist_lock */
> reaper = find_child_reaper(father);
> if (list_empty(&father->children))
> @@ -705,8 +702,18 @@ static void exit_notify(struct task_struct *tsk, int group_dead)
> LIST_HEAD(dead);
>
> write_lock_irq(&tasklist_lock);
> - forget_original_parent(tsk, &dead);
> + if (unlikely(!list_empty(&tsk->ptraced)))
> + exit_ptrace(tsk, &dead);
> + write_unlock_irq(&tasklist_lock);
> +
> + /* Ptraced tasks have to be released before zap_pid_ns_processes(). */
> + list_for_each_entry_safe(p, n, &dead, ptrace_entry) {
> + list_del_init(&p->ptrace_entry);
> + release_task(p);
> + }
>
> + write_lock_irq(&tasklist_lock);
> + forget_original_parent(tsk, &dead);
> if (group_dead)
> kill_orphaned_pgrp(tsk->group_leader, NULL);
>
> --
> 2.17.2
>
[-- Attachment #2: ptrace_pidns.c --]
[-- Type: text/plain, Size: 1096 bytes --]
#define _GNU_SOURCE /* See feature_test_macros(7) */
#include <unistd.h>
#include <sys/ptrace.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
#include <sys/wait.h>
#include <sys/syscall.h> /* For SYS_xxx definitions */
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
int pfd[2];
void *thread_fn()
{
pid_t tid = syscall(SYS_gettid);
write(pfd[1], &tid, sizeof(tid));
sleep(1000);
return NULL;
}
int main()
{
pid_t pid, tid, ppid = getpid();
pthread_t t;
if (pipe(pfd))
return 1;
pid = fork();
if (pid < 0)
return 1;
if (pid == 0) {
pthread_create(&t, NULL, thread_fn, (void *)(unsigned long)ppid);
sleep(1000);
return 0;
}
printf("fork: %d\n", pid);
if (read(pfd[0], &tid, sizeof(tid)) != sizeof(tid))
return 1;
printf("thread: %d\n", tid);
if (ptrace(PTRACE_ATTACH, tid, 0, 0))
return 1;
if (wait4(tid, NULL, __WALL, NULL) != tid)
return 1;
if (ptrace(PTRACE_ATTACH, pid, 0, 0))
return 1;
if (wait4(pid, NULL, __WALL, NULL) != pid)
return 1;
kill(pid, SIGKILL);
*((int *)(0)) = 0xdead;
return 0;
}
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes
2019-01-02 20:59 [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes Andrei Vagin
2019-01-02 21:16 ` Andrei Vagin
@ 2019-01-02 21:17 ` Andrei Vagin
2019-01-08 16:50 ` Oleg Nesterov
2 siblings, 0 replies; 7+ messages in thread
From: Andrei Vagin @ 2019-01-02 21:17 UTC (permalink / raw)
To: Alexander Viro, Andrew Morton, Oleg Nesterov
Cc: linux-kernel, Eric W. Biederman
Oops, Al is here by mistake. Sorry about that.
On Wed, Jan 02, 2019 at 12:59:39PM -0800, Andrei Vagin wrote:
> Currently, exit_ptrace() adds all ptraced tasks in a dead list, than
> zap_pid_ns_processes() waits all tasks in a current pid ns, and only
> then tasks from the dead list are released.
>
> zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In
> this case, we will have one unkillable process with one or more dead
> children.
>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: "Eric W. Biederman" <ebiederm@xmission.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Signed-off-by: Andrei Vagin <avagin@gmail.com>
> ---
> kernel/exit.c | 15 +++++++++++----
> 1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/kernel/exit.c b/kernel/exit.c
> index 0e21e6d21f35..ccaa6f6549ba 100644
> --- a/kernel/exit.c
> +++ b/kernel/exit.c
> @@ -664,9 +664,6 @@ static void forget_original_parent(struct task_struct *father,
> {
> struct task_struct *p, *t, *reaper;
>
> - if (unlikely(!list_empty(&father->ptraced)))
> - exit_ptrace(father, dead);
> -
> /* Can drop and reacquire tasklist_lock */
> reaper = find_child_reaper(father);
> if (list_empty(&father->children))
> @@ -705,8 +702,18 @@ static void exit_notify(struct task_struct *tsk, int group_dead)
> LIST_HEAD(dead);
>
> write_lock_irq(&tasklist_lock);
> - forget_original_parent(tsk, &dead);
> + if (unlikely(!list_empty(&tsk->ptraced)))
> + exit_ptrace(tsk, &dead);
> + write_unlock_irq(&tasklist_lock);
> +
> + /* Ptraced tasks have to be released before zap_pid_ns_processes(). */
> + list_for_each_entry_safe(p, n, &dead, ptrace_entry) {
> + list_del_init(&p->ptrace_entry);
> + release_task(p);
> + }
>
> + write_lock_irq(&tasklist_lock);
> + forget_original_parent(tsk, &dead);
> if (group_dead)
> kill_orphaned_pgrp(tsk->group_leader, NULL);
>
> --
> 2.17.2
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes
2019-01-02 20:59 [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes Andrei Vagin
2019-01-02 21:16 ` Andrei Vagin
2019-01-02 21:17 ` Andrei Vagin
@ 2019-01-08 16:50 ` Oleg Nesterov
2019-01-08 20:36 ` Andrei Vagin
2 siblings, 1 reply; 7+ messages in thread
From: Oleg Nesterov @ 2019-01-08 16:50 UTC (permalink / raw)
To: Andrei Vagin
Cc: Alexander Viro, Andrew Morton, linux-kernel, Eric W. Biederman
Sorry for delay, vacation,
On 01/02, Andrei Vagin wrote:
>
> zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In
> this case, we will have one unkillable process with one or more dead
> children.
Thanks!
> --- a/kernel/exit.c
> +++ b/kernel/exit.c
> @@ -664,9 +664,6 @@ static void forget_original_parent(struct task_struct *father,
> {
> struct task_struct *p, *t, *reaper;
>
> - if (unlikely(!list_empty(&father->ptraced)))
> - exit_ptrace(father, dead);
> -
> /* Can drop and reacquire tasklist_lock */
> reaper = find_child_reaper(father);
> if (list_empty(&father->children))
> @@ -705,8 +702,18 @@ static void exit_notify(struct task_struct *tsk, int group_dead)
> LIST_HEAD(dead);
>
> write_lock_irq(&tasklist_lock);
> - forget_original_parent(tsk, &dead);
> + if (unlikely(!list_empty(&tsk->ptraced)))
> + exit_ptrace(tsk, &dead);
> + write_unlock_irq(&tasklist_lock);
> +
> + /* Ptraced tasks have to be released before zap_pid_ns_processes(). */
> + list_for_each_entry_safe(p, n, &dead, ptrace_entry) {
> + list_del_init(&p->ptrace_entry);
> + release_task(p);
> + }
>
> + write_lock_irq(&tasklist_lock);
> + forget_original_parent(tsk, &dead);
> if (group_dead)
> kill_orphaned_pgrp(tsk->group_leader, NULL);
How about a different fix below? It avoids additional write_lock/unlock(tasklist),
and another list_for_each_entry_safe(dead) loop is called only if it is actually
needed.
Or I missed something?
Oleg.
--- x/kernel/exit.c
+++ x/kernel/exit.c
@@ -558,12 +558,14 @@ static struct task_struct *find_alive_th
return NULL;
}
-static struct task_struct *find_child_reaper(struct task_struct *father)
+static struct task_struct *find_child_reaper(struct task_struct *father,
+ struct list_head *dead)
__releases(&tasklist_lock)
__acquires(&tasklist_lock)
{
struct pid_namespace *pid_ns = task_active_pid_ns(father);
struct task_struct *reaper = pid_ns->child_reaper;
+ struct task_struct *p, *n;
if (likely(reaper != father))
return reaper;
@@ -579,6 +581,11 @@ static struct task_struct *find_child_re
panic("Attempted to kill init! exitcode=0x%08x\n",
father->signal->group_exit_code ?: father->exit_code);
}
+
+ list_for_each_entry_safe(p, n, &dead, ptrace_entry) {
+ list_del_init(&p->ptrace_entry);
+ release_task(p);
+ }
zap_pid_ns_processes(pid_ns);
write_lock_irq(&tasklist_lock);
@@ -668,7 +675,7 @@ static void forget_original_parent(struc
exit_ptrace(father, dead);
/* Can drop and reacquire tasklist_lock */
- reaper = find_child_reaper(father);
+ reaper = find_child_reaper(father, dead);
if (list_empty(&father->children))
return;
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes
2019-01-08 16:50 ` Oleg Nesterov
@ 2019-01-08 20:36 ` Andrei Vagin
2019-01-09 17:08 ` Oleg Nesterov
0 siblings, 1 reply; 7+ messages in thread
From: Andrei Vagin @ 2019-01-08 20:36 UTC (permalink / raw)
To: Oleg Nesterov; +Cc: Andrew Morton, linux-kernel, Eric W. Biederman
On Tue, Jan 08, 2019 at 05:50:43PM +0100, Oleg Nesterov wrote:
> Sorry for delay, vacation,
>
> On 01/02, Andrei Vagin wrote:
> >
> > zap_pid_ns_processes() can stuck on waiting tasks from the dead list. In
> > this case, we will have one unkillable process with one or more dead
> > children.
>
> Thanks!
>
> > --- a/kernel/exit.c
> > +++ b/kernel/exit.c
> > @@ -664,9 +664,6 @@ static void forget_original_parent(struct task_struct *father,
> > {
> > struct task_struct *p, *t, *reaper;
> >
> > - if (unlikely(!list_empty(&father->ptraced)))
> > - exit_ptrace(father, dead);
> > -
> > /* Can drop and reacquire tasklist_lock */
> > reaper = find_child_reaper(father);
> > if (list_empty(&father->children))
> > @@ -705,8 +702,18 @@ static void exit_notify(struct task_struct *tsk, int group_dead)
> > LIST_HEAD(dead);
> >
> > write_lock_irq(&tasklist_lock);
> > - forget_original_parent(tsk, &dead);
> > + if (unlikely(!list_empty(&tsk->ptraced)))
> > + exit_ptrace(tsk, &dead);
> > + write_unlock_irq(&tasklist_lock);
> > +
> > + /* Ptraced tasks have to be released before zap_pid_ns_processes(). */
> > + list_for_each_entry_safe(p, n, &dead, ptrace_entry) {
> > + list_del_init(&p->ptrace_entry);
> > + release_task(p);
> > + }
> >
> > + write_lock_irq(&tasklist_lock);
> > + forget_original_parent(tsk, &dead);
> > if (group_dead)
> > kill_orphaned_pgrp(tsk->group_leader, NULL);
>
> How about a different fix below? It avoids additional write_lock/unlock(tasklist),
> and another list_for_each_entry_safe(dead) loop is called only if it is actually
> needed.
>
> Or I missed something?
No, you don't. The patch looks really nice. Thanks!
BTW: We probably need to add the "Fixes:" tag, but I am not sure to which
commit, it looks like the issue is here for years.
Acked-by: Andrei Vagin <avagin@gmail.com>
>
> Oleg.
>
>
> --- x/kernel/exit.c
> +++ x/kernel/exit.c
> @@ -558,12 +558,14 @@ static struct task_struct *find_alive_th
> return NULL;
> }
>
> -static struct task_struct *find_child_reaper(struct task_struct *father)
> +static struct task_struct *find_child_reaper(struct task_struct *father,
> + struct list_head *dead)
> __releases(&tasklist_lock)
> __acquires(&tasklist_lock)
> {
> struct pid_namespace *pid_ns = task_active_pid_ns(father);
> struct task_struct *reaper = pid_ns->child_reaper;
> + struct task_struct *p, *n;
>
> if (likely(reaper != father))
> return reaper;
> @@ -579,6 +581,11 @@ static struct task_struct *find_child_re
> panic("Attempted to kill init! exitcode=0x%08x\n",
> father->signal->group_exit_code ?: father->exit_code);
> }
> +
> + list_for_each_entry_safe(p, n, &dead, ptrace_entry) {
> + list_del_init(&p->ptrace_entry);
> + release_task(p);
> + }
> zap_pid_ns_processes(pid_ns);
> write_lock_irq(&tasklist_lock);
>
> @@ -668,7 +675,7 @@ static void forget_original_parent(struc
> exit_ptrace(father, dead);
>
> /* Can drop and reacquire tasklist_lock */
> - reaper = find_child_reaper(father);
> + reaper = find_child_reaper(father, dead);
> if (list_empty(&father->children))
> return;
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes
2019-01-08 20:36 ` Andrei Vagin
@ 2019-01-09 17:08 ` Oleg Nesterov
2019-01-10 8:36 ` Andrei Vagin
0 siblings, 1 reply; 7+ messages in thread
From: Oleg Nesterov @ 2019-01-09 17:08 UTC (permalink / raw)
To: Andrei Vagin; +Cc: Andrew Morton, linux-kernel, Eric W. Biederman
On 01/08, Andrei Vagin wrote:
>
> No, you don't. The patch looks really nice. Thanks!
>
> BTW: We probably need to add the "Fixes:" tag, but I am not sure to which
> commit, it looks like the issue is here for years.
I hate you ;) Ok, OK, this was broken by me in 7c8bd2322c7fd973d089b27de55e29c92c667a06
"exit: ptrace: shift "reap dead" code from exit_ptrace() to forget_original_parent()"
> Acked-by: Andrei Vagin <avagin@gmail.com>
Thanks, but if you agree with this patch may I ask you to send v2?
You found the real problem, my version is only slightly better than yours,
I don't want to steal the "From" tag. Feel free to add mine signed-of-by.
Oleg.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] kernel: release ptraced tasks before zap_pid_ns_processes
2019-01-09 17:08 ` Oleg Nesterov
@ 2019-01-10 8:36 ` Andrei Vagin
0 siblings, 0 replies; 7+ messages in thread
From: Andrei Vagin @ 2019-01-10 8:36 UTC (permalink / raw)
To: Oleg Nesterov; +Cc: Andrew Morton, linux-kernel, Eric W. Biederman
On Wed, Jan 09, 2019 at 06:08:30PM +0100, Oleg Nesterov wrote:
> On 01/08, Andrei Vagin wrote:
> >
> > No, you don't. The patch looks really nice. Thanks!
> >
> > BTW: We probably need to add the "Fixes:" tag, but I am not sure to which
> > commit, it looks like the issue is here for years.
>
> I hate you ;)
I know :)
> Ok, OK, this was broken by me in 7c8bd2322c7fd973d089b27de55e29c92c667a06
> "exit: ptrace: shift "reap dead" code from exit_ptrace() to forget_original_parent()"
>
> > Acked-by: Andrei Vagin <avagin@gmail.com>
>
> Thanks, but if you agree with this patch may I ask you to send v2?
>
> You found the real problem, my version is only slightly better than yours,
> I don't want to steal the "From" tag. Feel free to add mine signed-of-by.
Thanks Oleg! I will send a second version of this patch.
>
> Oleg.
>
^ permalink raw reply [flat|nested] 7+ messages in thread