All of lore.kernel.org
 help / color / mirror / Atom feed
From: Oleg Nesterov <oleg@redhat.com>
To: Shayan Pooya <shayan@liveve.org>
Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>,
	Michal Hocko <mhocko@kernel.org>,
	koct9i@gmail.com, cgroups mailinglist <cgroups@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-mm@kvack.org
Subject: Re: bug in memcg oom-killer results in a hung syscall in another process in the same cgroup
Date: Thu, 14 Jul 2016 15:22:59 +0200	[thread overview]
Message-ID: <20160714132258.GA1333@redhat.com> (raw)
In-Reply-To: <CABAubTiVb8j8wEbcr16FAJnBxxS7QzghpPiJUcmV+=Ji=QgL=A@mail.gmail.com>

On 07/12, Shayan Pooya wrote:
>
> > Yep. Bug still not fixed in upstream. In our kernel I've plugged it with
> > this:
> >
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -2808,8 +2808,9 @@ asmlinkage __visible void schedule_tail(struct
> > task_struct *prev)
> >         balance_callback(rq);
> >         preempt_enable();
> >
> > -       if (current->set_child_tid)
> > -               put_user(task_pid_vnr(current), current->set_child_tid);
> > +       if (current->set_child_tid &&
> > +           put_user(task_pid_vnr(current), current->set_child_tid))
> > +               force_sig(SIGSEGV, current);
> >  }
>
> I just verified that with your patch there is no hung processes and I
> see processes getting SIGSEGV as expected.

Well, but we can't do this. And "as expected" is actually just wrong. I still
think that the whole FAULT_FLAG_USER logic is not right. This needs another email.

fork() should not fail because there is a memory hog in the same memcg. Worse,
pthread_create() can kill the caller by the same reason. And we have the same
or even worse problem with ->clear_child_tid, pthread_join() can hang forever.
Unlikely we want to kill the application in this case ;)

And in fact I think that the problem has nothing to do with set/claer_child_tid
in particular.

I am just curious... can you reproduce the problem reliably? If yes, can you try
the patch below ? Just in case, this is not the real fix in any case...

Oleg.

--- x/kernel/sched/core.c
+++ x/kernel/sched/core.c
@@ -2793,8 +2793,11 @@ asmlinkage __visible void schedule_tail(struct task_struct *prev)
 	balance_callback(rq);
 	preempt_enable();
 
-	if (current->set_child_tid)
+	if (current->set_child_tid) {
+		mem_cgroup_oom_enable();
 		put_user(task_pid_vnr(current), current->set_child_tid);
+		mem_cgroup_oom_disable();
+	}
 }
 
 /*


WARNING: multiple messages have this Message-ID (diff)
From: Oleg Nesterov <oleg@redhat.com>
To: Shayan Pooya <shayan@liveve.org>
Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>,
	Michal Hocko <mhocko@kernel.org>,
	koct9i@gmail.com, cgroups mailinglist <cgroups@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-mm@kvack.org
Subject: Re: bug in memcg oom-killer results in a hung syscall in another process in the same cgroup
Date: Thu, 14 Jul 2016 15:22:59 +0200	[thread overview]
Message-ID: <20160714132258.GA1333@redhat.com> (raw)
In-Reply-To: <CABAubTiVb8j8wEbcr16FAJnBxxS7QzghpPiJUcmV+=Ji=QgL=A@mail.gmail.com>

On 07/12, Shayan Pooya wrote:
>
> > Yep. Bug still not fixed in upstream. In our kernel I've plugged it with
> > this:
> >
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -2808,8 +2808,9 @@ asmlinkage __visible void schedule_tail(struct
> > task_struct *prev)
> >         balance_callback(rq);
> >         preempt_enable();
> >
> > -       if (current->set_child_tid)
> > -               put_user(task_pid_vnr(current), current->set_child_tid);
> > +       if (current->set_child_tid &&
> > +           put_user(task_pid_vnr(current), current->set_child_tid))
> > +               force_sig(SIGSEGV, current);
> >  }
>
> I just verified that with your patch there is no hung processes and I
> see processes getting SIGSEGV as expected.

Well, but we can't do this. And "as expected" is actually just wrong. I still
think that the whole FAULT_FLAG_USER logic is not right. This needs another email.

fork() should not fail because there is a memory hog in the same memcg. Worse,
pthread_create() can kill the caller by the same reason. And we have the same
or even worse problem with ->clear_child_tid, pthread_join() can hang forever.
Unlikely we want to kill the application in this case ;)

And in fact I think that the problem has nothing to do with set/claer_child_tid
in particular.

I am just curious... can you reproduce the problem reliably? If yes, can you try
the patch below ? Just in case, this is not the real fix in any case...

Oleg.

--- x/kernel/sched/core.c
+++ x/kernel/sched/core.c
@@ -2793,8 +2793,11 @@ asmlinkage __visible void schedule_tail(struct task_struct *prev)
 	balance_callback(rq);
 	preempt_enable();
 
-	if (current->set_child_tid)
+	if (current->set_child_tid) {
+		mem_cgroup_oom_enable();
 		put_user(task_pid_vnr(current), current->set_child_tid);
+		mem_cgroup_oom_disable();
+	}
 }
 
 /*

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2016-07-14 13:22 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-09 23:49 bug in memcg oom-killer results in a hung syscall in another process in the same cgroup Shayan Pooya
2016-07-09 23:49 ` Shayan Pooya
     [not found] ` <CABAubThf6gbi243BqYgoCjqRW36sXJuJ6e_8zAqzkYRiu0GVtQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-07-11  6:41   ` Michal Hocko
2016-07-11  6:41     ` Michal Hocko
2016-07-11  6:41     ` Michal Hocko
2016-07-11 17:40     ` Shayan Pooya
2016-07-11 17:40       ` Shayan Pooya
2016-07-11 18:33       ` Shayan Pooya
2016-07-11 18:33         ` Shayan Pooya
2016-07-12  7:19         ` Michal Hocko
2016-07-12  7:19           ` Michal Hocko
2016-07-12 15:35           ` Shayan Pooya
2016-07-12 15:35             ` Shayan Pooya
2016-07-12 15:52             ` Konstantin Khlebnikov
2016-07-12 15:52               ` Konstantin Khlebnikov
2016-07-12 16:52               ` Oleg Nesterov
2016-07-12 16:52                 ` Oleg Nesterov
2016-07-12 22:57               ` Shayan Pooya
2016-07-12 22:57                 ` Shayan Pooya
2016-07-14 13:22                 ` Oleg Nesterov [this message]
2016-07-14 13:22                   ` Oleg Nesterov
2016-07-14 15:35                   ` Shayan Pooya
2016-07-14 15:35                     ` Shayan Pooya
2016-07-15 16:58                     ` Shayan Pooya
2016-07-15 16:58                       ` Shayan Pooya
     [not found]                       ` <CABAubTjjD6nmAtMNze5O6-bE-ivMmb24Jd4u2mMpBZFBFR1CnA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-07-18 13:53                         ` Oleg Nesterov
2016-07-18 13:53                           ` Oleg Nesterov
2016-07-18 13:53                           ` Oleg Nesterov
2016-07-13  8:08             ` Michal Hocko
2016-07-13  8:08               ` Michal Hocko
     [not found]       ` <CABAubThHfngHTQW_AEuW71VCvLyD_9b5Z05tSud5bf8JKjuA9Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-07-12  7:17         ` Michal Hocko
2016-07-12  7:17           ` Michal Hocko
2016-07-12  7:17           ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160714132258.GA1333@redhat.com \
    --to=oleg@redhat.com \
    --cc=cgroups@vger.kernel.org \
    --cc=khlebnikov@yandex-team.ru \
    --cc=koct9i@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=shayan@liveve.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.