* Couple of nasty bitbake issues
@ 2011-06-16 13:54 Richard Purdie
2011-06-16 14:45 ` Jeremy Puhlman
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Richard Purdie @ 2011-06-16 13:54 UTC (permalink / raw)
To: bitbake-devel
We have two nasty bitbake issues I've found:
a) Our event handlers don't behave quite as you'd expect due to the use
of global namespace.
Basically, if you *ever* register an eventhandler, even in an optional
class like native or nativesdk, your will always subsequently get called
for any event.
This means a handler in native.bbclass will see parsing events for a
recipe which doesn't use it. The problem comes from ast.py:
all_handlers = {}
for var in bb.data.getVar('__BBHANDLERS', d) or []:
# try to add the handler
handler = bb.data.getVar(var, d)
bb.event.register(var, handler)
Since we register the handlers but never un-register them or only call
the ones which the current recipe uses.
I can work around this for now but we really should fix this.
b) The none server is totally broken for logging during parsing. Trying
to cram two combine two pieces of code into one main loop turns out not
to work so well.
The first problem is that any events coming from the sub processes
forked off during parallel parsing don't get sent to the server/UI. I
know this has been commented on by various people, this is the root
cause. I was able to fix that by borrowing queue code from the process
server, so far so good.
The problem then is the logger, since the logger in the UI uses the same
namespace and setup as the one in the server. The one in the server is
supposed to fire() events, the one in the UI is then meant to write them
to the console. Trying to do both things in the same process goes
horribly wrong. I have a hacky fix where each event is just seen twice
but it doesn't look like an easy problem to solve. I'm unable to find an
easy way to setup two loggers in parallel copying the config from one to
the other which is ideally what is needed.
I like the none server as its simple to debug. At present its looking
like it might be too ugly to fix this though. My hacks are:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/ml&id=a3a130b3c6ca760066ca830eb46fa472ecb84c88
Cheers,
Richard
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: Couple of nasty bitbake issues
2011-06-16 13:54 Couple of nasty bitbake issues Richard Purdie
@ 2011-06-16 14:45 ` Jeremy Puhlman
2011-06-16 23:35 ` Khem Raj
2011-06-17 1:59 ` Khem Raj
2 siblings, 0 replies; 4+ messages in thread
From: Jeremy Puhlman @ 2011-06-16 14:45 UTC (permalink / raw)
To: Richard Purdie; +Cc: bitbake-devel
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/ml&id=a3a130b3c6ca760066ca830eb46fa472ecb84c88
I did a quick build with this and it seems to cover the issue I
mentioned this morning. Ill kick off a more robust build, is there
anything I should be watching out for?
--
Jeremy Puhlman
Montavista Sofware, LLC.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Couple of nasty bitbake issues
2011-06-16 13:54 Couple of nasty bitbake issues Richard Purdie
2011-06-16 14:45 ` Jeremy Puhlman
@ 2011-06-16 23:35 ` Khem Raj
2011-06-17 1:59 ` Khem Raj
2 siblings, 0 replies; 4+ messages in thread
From: Khem Raj @ 2011-06-16 23:35 UTC (permalink / raw)
To: bitbake-devel
On 06/16/2011 06:54 AM, Richard Purdie wrote:
> We have two nasty bitbake issues I've found:
>
> a) Our event handlers don't behave quite as you'd expect due to the use
> of global namespace.
>
> Basically, if you *ever* register an eventhandler, even in an optional
> class like native or nativesdk, your will always subsequently get called
> for any event.
>
> This means a handler in native.bbclass will see parsing events for a
> recipe which doesn't use it. The problem comes from ast.py:
>
> all_handlers = {}
> for var in bb.data.getVar('__BBHANDLERS', d) or []:
> # try to add the handler
> handler = bb.data.getVar(var, d)
> bb.event.register(var, handler)
>
> Since we register the handlers but never un-register them or only call
> the ones which the current recipe uses.
>
> I can work around this for now but we really should fix this.
>
> b) The none server is totally broken for logging during parsing. Trying
> to cram two combine two pieces of code into one main loop turns out not
> to work so well.
>
> The first problem is that any events coming from the sub processes
> forked off during parallel parsing don't get sent to the server/UI. I
> know this has been commented on by various people, this is the root
> cause. I was able to fix that by borrowing queue code from the process
> server, so far so good.
>
> The problem then is the logger, since the logger in the UI uses the same
> namespace and setup as the one in the server. The one in the server is
> supposed to fire() events, the one in the UI is then meant to write them
> to the console. Trying to do both things in the same process goes
> horribly wrong. I have a hacky fix where each event is just seen twice
> but it doesn't look like an easy problem to solve. I'm unable to find an
> easy way to setup two loggers in parallel copying the config from one to
> the other which is ideally what is needed.
Does something like this can help
http://www.valuedlessons.com/2008/04/events-in-python.html
>
> I like the none server as its simple to debug. At present its looking
> like it might be too ugly to fix this though. My hacks are:
>
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/ml&id=a3a130b3c6ca760066ca830eb46fa472ecb84c88
>
> Cheers,
>
> Richard
>
>
>
>
>
>
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/bitbake-devel
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: Couple of nasty bitbake issues
2011-06-16 13:54 Couple of nasty bitbake issues Richard Purdie
2011-06-16 14:45 ` Jeremy Puhlman
2011-06-16 23:35 ` Khem Raj
@ 2011-06-17 1:59 ` Khem Raj
2 siblings, 0 replies; 4+ messages in thread
From: Khem Raj @ 2011-06-17 1:59 UTC (permalink / raw)
To: Richard Purdie; +Cc: bitbake-devel
On Thu, Jun 16, 2011 at 6:54 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> I like the none server as its simple to debug. At present its looking
> like it might be too ugly to fix this though. My hacks are:
>
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/ml&id=a3a130b3c6ca760066ca830eb46fa472ecb84c88
>
FWIW I was seeing a weird behaviour on one of my machines that Iwas
discussing on IRC today where bitbake was freezing my system and that
issue seems
to be gone with this patch too now I can bitbake can happily run two
jobs simultaneously
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-06-17 2:03 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-16 13:54 Couple of nasty bitbake issues Richard Purdie
2011-06-16 14:45 ` Jeremy Puhlman
2011-06-16 23:35 ` Khem Raj
2011-06-17 1:59 ` Khem Raj
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.