* [PATCH] command/runqueue: Fix shutdown logic @ 2014-07-21 8:35 Richard Purdie 2014-07-22 14:46 ` Martin Jansa 0 siblings, 1 reply; 4+ messages in thread From: Richard Purdie @ 2014-07-21 8:35 UTC (permalink / raw) To: bitbake-devel If you hit Ctrl+C at the right point, the system processes the request but merrily continues building. It turns out finish_runqueue() is called but this doesn't stop the later generation and execution of the runqueue. This patch adjusts some of the conditionals to ensure the build really does stop. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> diff --git a/bitbake/lib/bb/command.py b/bitbake/lib/bb/command.py index 84fcdf9..d797fcf 100644 --- a/bitbake/lib/bb/command.py +++ b/bitbake/lib/bb/command.py @@ -86,7 +86,7 @@ class Command: def runAsyncCommand(self): try: - if self.cooker.state == bb.cooker.state.error: + if self.cooker.state in (bb.cooker.state.error, bb.cooker.state.shutdown, bb.cooker.state.forceshutdown): return False if self.currentAsyncCommand is not None: (command, options) = self.currentAsyncCommand diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py index 4ea4970..f68a11d 100644 --- a/bitbake/lib/bb/runqueue.py +++ b/bitbake/lib/bb/runqueue.py @@ -1065,7 +1065,7 @@ class RunQueue: if self.state is runQueueCleanUp: self.rqexe.finish() - if self.state is runQueueComplete or self.state is runQueueFailed: + if (self.state is runQueueComplete or self.state is runQueueFailed) and self.rqexe: self.teardown_workers() if self.rqexe.stats.failed: logger.info("Tasks Summary: Attempted %d tasks of which %d didn't need to be rerun and %d failed.", self.rqexe.stats.completed + self.rqexe.stats.failed, self.rqexe.stats.skipped, self.rqexe.stats.failed) @@ -1106,6 +1106,7 @@ class RunQueue: def finish_runqueue(self, now = False): if not self.rqexe: + self.state = runQueueComplete return if now: -- cgit v0.10.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] command/runqueue: Fix shutdown logic 2014-07-21 8:35 [PATCH] command/runqueue: Fix shutdown logic Richard Purdie @ 2014-07-22 14:46 ` Martin Jansa 2014-07-22 18:10 ` Richard Purdie 0 siblings, 1 reply; 4+ messages in thread From: Martin Jansa @ 2014-07-22 14:46 UTC (permalink / raw) To: Richard Purdie; +Cc: bitbake-devel [-- Attachment #1: Type: text/plain, Size: 4052 bytes --] On Mon, Jul 21, 2014 at 09:35:53AM +0100, Richard Purdie wrote: > If you hit Ctrl+C at the right point, the system processes the request > but merrily continues building. It turns out finish_runqueue() is called > but this doesn't stop the later generation and execution of the > runqueue. > > This patch adjusts some of the conditionals to ensure the build really > does stop. Great, I've included this change in my world builds to see if it fixes bitbake still running after jenkins job is aborted. I don't think it's caused by this change and I don't know how much we can do about it, but today I was testing snort build (which eats all memory in m4 call until OOMK kills it) and when I wanted to interrupt the build it failed with 2 tracebacks: NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Running task 569 of 610 (ID: 5, /OE/build/oe-core/meta-openembedded/meta-networking/recipes-connectivity/snort/snort_2.9.6.0.bb, do_configure) NOTE: recipe snort-2.9.6.0-r0: task do_configure: Started ^C^C^C^C^C^C^C^C^C^C^C^C^C^CTraceback (most recent call last): File "/OE/build/oe-core/bitbake/bin/bitbake", line 382, in <module> ret = main() File "/OE/build/oe-core/bitbake/bin/bitbake", line 372, in main bb.event.ui_queue = [] KeyboardInterrupt ^CException KeyboardInterrupt in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored ^CError in atexit._run_exitfuncs: ^CError in sys.exitfunc: Traceback (most recent call last): File "/usr/lib64/python2.7/atexit.py", line 30, in _run_exitfuncs traceback.print_exc() File "/usr/lib64/python2.7/traceback.py", line 233, in print_exc print_exception(etype, value, tb, limit, file) File "/usr/lib64/python2.7/traceback.py", line 110, in print_exception def print_exception(etype, value, tb, limit=None, file=None): KeyboardInterrupt There was also about 5 minute delay between first 2 Ctrl+C and actual exit, but that could be caused by huge load caused by that faulty m4. Regards, > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > diff --git a/bitbake/lib/bb/command.py b/bitbake/lib/bb/command.py > index 84fcdf9..d797fcf 100644 > --- a/bitbake/lib/bb/command.py > +++ b/bitbake/lib/bb/command.py > @@ -86,7 +86,7 @@ class Command: > > def runAsyncCommand(self): > try: > - if self.cooker.state == bb.cooker.state.error: > + if self.cooker.state in (bb.cooker.state.error, bb.cooker.state.shutdown, bb.cooker.state.forceshutdown): > return False > if self.currentAsyncCommand is not None: > (command, options) = self.currentAsyncCommand > diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py > index 4ea4970..f68a11d 100644 > --- a/bitbake/lib/bb/runqueue.py > +++ b/bitbake/lib/bb/runqueue.py > @@ -1065,7 +1065,7 @@ class RunQueue: > if self.state is runQueueCleanUp: > self.rqexe.finish() > > - if self.state is runQueueComplete or self.state is runQueueFailed: > + if (self.state is runQueueComplete or self.state is runQueueFailed) and self.rqexe: > self.teardown_workers() > if self.rqexe.stats.failed: > logger.info("Tasks Summary: Attempted %d tasks of which %d didn't need to be rerun and %d failed.", self.rqexe.stats.completed + self.rqexe.stats.failed, self.rqexe.stats.skipped, self.rqexe.stats.failed) > @@ -1106,6 +1106,7 @@ class RunQueue: > > def finish_runqueue(self, now = False): > if not self.rqexe: > + self.state = runQueueComplete > return > > if now: > -- > cgit v0.10.1 > > > > -- > _______________________________________________ > bitbake-devel mailing list > bitbake-devel@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/bitbake-devel -- Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 188 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] command/runqueue: Fix shutdown logic 2014-07-22 14:46 ` Martin Jansa @ 2014-07-22 18:10 ` Richard Purdie 2014-07-22 18:51 ` Martin Jansa 0 siblings, 1 reply; 4+ messages in thread From: Richard Purdie @ 2014-07-22 18:10 UTC (permalink / raw) To: Martin Jansa; +Cc: bitbake-devel On Tue, 2014-07-22 at 16:46 +0200, Martin Jansa wrote: > On Mon, Jul 21, 2014 at 09:35:53AM +0100, Richard Purdie wrote: > > If you hit Ctrl+C at the right point, the system processes the request > > but merrily continues building. It turns out finish_runqueue() is called > > but this doesn't stop the later generation and execution of the > > runqueue. > > > > This patch adjusts some of the conditionals to ensure the build really > > does stop. > > Great, I've included this change in my world builds to see if it fixes > bitbake still running after jenkins job is aborted. I've seen that too and I don't think this fix will address that unfortunately. Its on my list of things to look into. > I don't think it's caused by this change and I don't know how much we > can do about it, but today I was testing snort build (which eats all > memory in m4 call until OOMK kills it) I ended up excluding snort from my builds for that reason. Builds go a lot faster when its not destroying the machine! > and when I wanted to interrupt > the build it failed with 2 tracebacks: > > NOTE: Preparing runqueue > NOTE: Executing SetScene Tasks > NOTE: Executing RunQueue Tasks > NOTE: Running task 569 of 610 (ID: 5, /OE/build/oe-core/meta-openembedded/meta-networking/recipes-connectivity/snort/snort_2.9.6.0.bb, do_configure) > NOTE: recipe snort-2.9.6.0-r0: task do_configure: Started > ^C^C^C^C^C^C^C^C^C^C^C^C^C^CTraceback (most recent call last): > File "/OE/build/oe-core/bitbake/bin/bitbake", line 382, in <module> > ret = main() > File "/OE/build/oe-core/bitbake/bin/bitbake", line 372, in main > bb.event.ui_queue = [] > KeyboardInterrupt > ^CException KeyboardInterrupt in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored > > ^CError in atexit._run_exitfuncs: > ^CError in sys.exitfunc: > Traceback (most recent call last): > File "/usr/lib64/python2.7/atexit.py", line 30, in _run_exitfuncs > traceback.print_exc() > File "/usr/lib64/python2.7/traceback.py", line 233, in print_exc > print_exception(etype, value, tb, limit, file) > File "/usr/lib64/python2.7/traceback.py", line 110, in print_exception > def print_exception(etype, value, tb, limit=None, file=None): > KeyboardInterrupt > > There was also about 5 minute delay between first 2 Ctrl+C and actual > exit, but that could be caused by huge load caused by that faulty m4. Thanks, I'll have a look at those and see if they're significant and if we can do anything about them. It may be they are "one offs" and unlikely to reproduce but we'll see. Cheers, Richard ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] command/runqueue: Fix shutdown logic 2014-07-22 18:10 ` Richard Purdie @ 2014-07-22 18:51 ` Martin Jansa 0 siblings, 0 replies; 4+ messages in thread From: Martin Jansa @ 2014-07-22 18:51 UTC (permalink / raw) To: Richard Purdie; +Cc: bitbake-devel [-- Attachment #1: Type: text/plain, Size: 3169 bytes --] On Tue, Jul 22, 2014 at 07:10:31PM +0100, Richard Purdie wrote: > On Tue, 2014-07-22 at 16:46 +0200, Martin Jansa wrote: > > On Mon, Jul 21, 2014 at 09:35:53AM +0100, Richard Purdie wrote: > > > If you hit Ctrl+C at the right point, the system processes the request > > > but merrily continues building. It turns out finish_runqueue() is called > > > but this doesn't stop the later generation and execution of the > > > runqueue. > > > > > > This patch adjusts some of the conditionals to ensure the build really > > > does stop. > > > > Great, I've included this change in my world builds to see if it fixes > > bitbake still running after jenkins job is aborted. > > I've seen that too and I don't think this fix will address that > unfortunately. Its on my list of things to look into. > > > I don't think it's caused by this change and I don't know how much we > > can do about it, but today I was testing snort build (which eats all > > memory in m4 call until OOMK kills it) > > I ended up excluding snort from my builds for that reason. Builds go a > lot faster when its not destroying the machine! adding pkgconfig inherit fixed that, but indeed strange side effect and it seems to trigger this memory eating only in combination with more restrictive m4 dependencies or foreign flag (but I got OOM even after patching snort's configure.in to pass foreign). > > and when I wanted to interrupt > > the build it failed with 2 tracebacks: > > > > NOTE: Preparing runqueue > > NOTE: Executing SetScene Tasks > > NOTE: Executing RunQueue Tasks > > NOTE: Running task 569 of 610 (ID: 5, /OE/build/oe-core/meta-openembedded/meta-networking/recipes-connectivity/snort/snort_2.9.6.0.bb, do_configure) > > NOTE: recipe snort-2.9.6.0-r0: task do_configure: Started > > ^C^C^C^C^C^C^C^C^C^C^C^C^C^CTraceback (most recent call last): > > File "/OE/build/oe-core/bitbake/bin/bitbake", line 382, in <module> > > ret = main() > > File "/OE/build/oe-core/bitbake/bin/bitbake", line 372, in main > > bb.event.ui_queue = [] > > KeyboardInterrupt > > ^CException KeyboardInterrupt in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored > > > > ^CError in atexit._run_exitfuncs: > > ^CError in sys.exitfunc: > > Traceback (most recent call last): > > File "/usr/lib64/python2.7/atexit.py", line 30, in _run_exitfuncs > > traceback.print_exc() > > File "/usr/lib64/python2.7/traceback.py", line 233, in print_exc > > print_exception(etype, value, tb, limit, file) > > File "/usr/lib64/python2.7/traceback.py", line 110, in print_exception > > def print_exception(etype, value, tb, limit=None, file=None): > > KeyboardInterrupt > > > > There was also about 5 minute delay between first 2 Ctrl+C and actual > > exit, but that could be caused by huge load caused by that faulty m4. > > Thanks, I'll have a look at those and see if they're significant and if > we can do anything about them. It may be they are "one offs" and > unlikely to reproduce but we'll see. > > Cheers, > > Richard > -- Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 188 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-07-22 18:51 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-07-21 8:35 [PATCH] command/runqueue: Fix shutdown logic Richard Purdie 2014-07-22 14:46 ` Martin Jansa 2014-07-22 18:10 ` Richard Purdie 2014-07-22 18:51 ` Martin Jansa
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.