Openembedded Bitbake Development
 help / color / mirror / Atom feed
From: Jason Wessel <jason.wessel@windriver.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: bitbake-devel <bitbake-devel@lists.openembedded.org>
Subject: Re: [PATCH] process: Improve exit handling and hangs
Date: Tue, 27 Aug 2013 15:11:43 -0500	[thread overview]
Message-ID: <521D07FF.9000507@windriver.com> (raw)
In-Reply-To: <1377346040.6762.186.camel@ted>

On 08/24/2013 07:07 AM, Richard Purdie wrote:
> It turns out we have a number of different ways the process server termination can
> hang. If we call cancel_join_thread() on the event queue, it means that it can be left
> containing partial data. This means the reading of the event queue in the terminate()
> function can hang, the timeout and block parameters to Queue.get() don't make any
> difference.
>
> Equally, if we don't call cancel_join_thread(), the join_thread in terminate()
> will hang giving a different deadlock.
>
> The best solution I could find is to loop over the process is_alive() after requesting
> it stops,  trying to join the thread and if that fails, try and flush the event
> queue again.
>
> It wasn't clear what difference a force option should make in this case, we're
> gracefully trying to empty queues and shut down regardless of whether its a SIGTERM
> so I've simply removed the force option.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>
> Jason: Not sure if this or the other patch will help the hang you are
> seeing or not but they seem like good changes regardless and fix real
> world issues.


Certainly the behavior is a bit better with your additional patches, but it is not the root of the problem.  I have tested your patches in the heavy load situations where we have observed all the hangs.

I'll send a patch separately along with an explanation of the root cause of the hangs in the PR Server.

As for your patches, I have reviewed and tested them:  Acked-by: Jason Wessel <jason.wessel@windriver.com>

Cheers,
Jason.



      parent reply	other threads:[~2013-08-27 20:11 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-24 12:07 [PATCH] process: Improve exit handling and hangs Richard Purdie
2013-08-24 12:40 ` Richard Purdie
2013-08-27 20:11 ` Jason Wessel [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=521D07FF.9000507@windriver.com \
    --to=jason.wessel@windriver.com \
    --cc=bitbake-devel@lists.openembedded.org \
    --cc=richard.purdie@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox