[RFC] rm_work

[RFC] rm_work

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I want to propose making rm_work inherited by default with the following changes:

* a KEEP_WORK = "1" var in bitbake.conf so it is opt-in
* a KEEP_WORK_pn-${PN} option to mark packages in local.conf
* a KEEP_WORK option in recipes to mark them, e.g. for llvm

Summary: it will default to 'off' (keep the workdir), but can be switched globally and per
package.

In the last few weeks that I've used rm_work, I haven't found any big problems with it,
but as Graeme points out, there is no easy way to disable yet.

What do you think?

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFGF25RMkyGM64RGpERAtDZAJsH34WyVDZaQCnrcpgEF8ezNySLfgCgu30W
299vq8pM/5y1KQMaplmmlhY=
=U4ym
-----END PGP SIGNATURE-----

Re: [RFC] rm_work

[Holger Freyther]

Am 07.04.2007 um 12:11 schrieb Koen Kooi:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> In the last few weeks that I've used rm_work, I haven't found any  
> big problems with it,
> but as Graeme points out, there is no easy way to disable yet.
>
> What do you think?

Hi,

how many recipes still break with rm_work, is there a documentation  
which tasks to add and link to unbreak rm_work (if it breaks). I'm  
still uncertain if the cleaning should be done by bitbake itself.


z.

Re: [RFC] rm_work

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Holger Freyther schreef:
> Am 07.04.2007 um 12:11 schrieb Koen Kooi:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>>
>> In the last few weeks that I've used rm_work, I haven't found any  
>> big problems with it,
>> but as Graeme points out, there is no easy way to disable yet.
>>
>> What do you think?
> 
> Hi,
> 
> how many recipes still break with rm_work

angstrom-x11-image, angstrom-console-image and openmoko-devel-image work for various machines.

>, is there a documentation  
> which tasks to add and link to unbreak rm_work (if it breaks).

No idea, Marcin?

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFGF3U+MkyGM64RGpERArPDAKCCtK9w8msutoFKGCJh1oQwf2n9qACfcilU
s7fZgbhu7A2KKs55G/qobeY=
=X890
-----END PGP SIGNATURE-----

Re: [RFC] rm_work

[Rod Whitby]

Koen Kooi wrote:
> Holger Freyther schreef:
>>> how many recipes still break with rm_work
> 
> angstrom-x11-image, angstrom-console-image and openmoko-devel-image work for various machines.

All recipes used by nslu2-linux are compatible with rm_work - we use it
on our autobuilders.

-- Rod

Re: [RFC] rm_work

[Holger Freyther]

Am 07.04.2007 um 12:41 schrieb Koen Kooi:

>> , is there a documentation
>> which tasks to add and link to unbreak rm_work (if it breaks).
>
> No idea, Marcin?


NOTE: package sjf2410-linux-native-20060807-r1_0: task do_rm_work:  
started
NOTE: package sjf2410-linux-native-20060807-r1_0: task do_rm_work:  
completed
NOTE: package sjf2410-linux-native-20060807-r1_0: task do_deploy:  
started
ERROR: function do_deploy failed
ERROR: log data follows (/home/friedrij/src/moko/mokomakefile/trunk/ 
build/tmp/work/i686-linux/sjf2410-linux-native-20060807-r1_0/temp/ 
log.do_deploy.27169)
| install: cannot stat `sjf2410': No such file or directory
NOTE: Task failed: /home/friedrij/src/moko/mokomakefile/trunk/build/ 
tmp/work/i686-linux/sjf2410-linux-native-20060807-r1_0/temp/ 
log.do_deploy.27169
NOTE: package sjf2410-linux-native-20060807-r1_0: task do_deploy: failed
ERROR: TaskFailed event exception, aborting
NOTE: package sjf2410-linux-native-20060807: failed
ERROR: Build of openmoko-devel-image faile


Wow, don't ask whom I paid for this... This is one of the reasons I  
believe that this functionality belongs into bitbake itself. It is  
the only way to reliable call a task after do_build!

RP do you have comments on that one?

z.

Re: [RFC] rm_work

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Holger Freyther schreef:
> Am 07.04.2007 um 12:41 schrieb Koen Kooi:
> 
>>> , is there a documentation
>>> which tasks to add and link to unbreak rm_work (if it breaks).
>> No idea, Marcin?
> 
> 
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_rm_work:  
> started
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_rm_work:  
> completed
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_deploy:  
> started
> ERROR: function do_deploy failed
> ERROR: log data follows (/home/friedrij/src/moko/mokomakefile/trunk/ 
> build/tmp/work/i686-linux/sjf2410-linux-native-20060807-r1_0/temp/ 
> log.do_deploy.27169)
> | install: cannot stat `sjf2410': No such file or directory
> NOTE: Task failed: /home/friedrij/src/moko/mokomakefile/trunk/build/ 
> tmp/work/i686-linux/sjf2410-linux-native-20060807-r1_0/temp/ 
> log.do_deploy.27169
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_deploy: failed
> ERROR: TaskFailed event exception, aborting
> NOTE: package sjf2410-linux-native-20060807: failed
> ERROR: Build of openmoko-devel-image faile

AFAIK, the sjf2410 recipe in OE, not in svn works[*]. I don't want to support buggy
external repos without getting paid.

regards,

Koen

[*] It built fine yesterday
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFGGNq/MkyGM64RGpERAnu+AJ9tdnLxfQCgypKuOBQDyPjTFPrNgwCaAzQo
RbdyAhSTRNJ0co2QB26lg3g=
=bQWA
-----END PGP SIGNATURE-----

Re: [RFC] rm_work

[Koen Kooi]

Quoting Koen Kooi <k.kooi@student.utwente.nl>:

> AFAIK, the sjf2410 recipe in OE, not in svn works

I just checked:
http://www.openembedded.org/bonsai/query/description/do_deploy/?scm_id=3

While I understand your point about documenting things like this for ISVs, your
example just shows an ISV too lazy/busy/whatever to merge back fixes.

regards,

Koen

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.

Re: [RFC] rm_work

[Michael 'Mickey' Lauer]

Koen Kooi wrote:
> Quoting Koen Kooi <k.kooi@student.utwente.nl>:

>> AFAIK, the sjf2410 recipe in OE, not in svn works

> I just checked:
> http://www.openembedded.org/bonsai/query/description/do_deploy/?scm_id=3

> While I understand your point about documenting things like this for ISVs, your
> example just shows an ISV too lazy/busy/whatever to merge back fixes.

Ouch ;)

Regards,

:M:
-- 
Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de

Re: [RFC] rm_work

[Richard Purdie]

On Sun, 2007-04-08 at 00:03 +0200, Holger Freyther wrote:
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_rm_work:  
> started
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_rm_work:  
> completed
> NOTE: package sjf2410-linux-native-20060807-r1_0: task do_deploy:  
> started
> ERROR: function do_deploy failed
[...]
> Wow, don't ask whom I paid for this... This is one of the reasons I  
> believe that this functionality belongs into bitbake itself. It is  
> the only way to reliable call a task after do_build!
> 
> RP do you have comments on that one?

Even if you do this in bitbake itself, what happens if two tasks want to
run last before do_build? Which do you choose?

My thoughts are that we need to improve the way we specify tasks to be
able to specify this kind of relationship more accurately. Even the
form:

addtask rm_work before do_build after do_package_write do_deploy

where the tasks might not always exist might be enough to get things
working?

Cheers,

Richard

Re: [RFC] rm_work

[Marcin Juszkiewicz]

Dnia niedziela, 8 kwietnia 2007, Richard Purdie napisał:

> Even if you do this in bitbake itself, what happens if two tasks want
> to run last before do_build? Which do you choose?

> My thoughts are that we need to improve the way we specify tasks to be
> able to specify this kind of relationship more accurately. Even the
> form:
>
> addtask rm_work before do_build after do_package_write do_deploy
>
> where the tasks might not always exist might be enough to get things
> working?

I wrote rm_work.bbclass few years ago and it was always known as 'you can 
hit your own foot' solution due to lack of possibility to define WHEN 
exactly task will be started:

addtask rm_work right_before do_build after all_other_tasks

Maybe rm_work.bbclass should do:

BB_DEFAULT_TASK = "rm_work"
addtask rm_work after do_build

But I did not tested it yet (got that idea during Easter).

-- 
JID: hrw-jabber.org
OpenEmbedded developer/consultant

                            It is your destiny.
                            		-- Darth Vader

Re: [RFC] rm_work

[Marcin Juszkiewicz]

Dnia środa, 11 kwietnia 2007, Marcin Juszkiewicz napisał:

> Maybe rm_work.bbclass should do:
>
> BB_DEFAULT_TASK = "rm_work"
> addtask rm_work after do_build
>
> But I did not tested it yet (got that idea during Easter).

Looks like it is a way - atleast it works in Poky:

08:52 hrw@home:build$ bitbake base-files
NOTE: package base-files-3.0.14: started
NOTE: package base-files-3.0.14-r55: task do_fetch: started
NOTE: package base-files-3.0.14-r55: task do_fetch: completed
NOTE: package base-files-3.0.14-r55: task do_unpack: started
NOTE: package base-files-3.0.14-r55: task do_unpack: completed
NOTE: package base-files-3.0.14-r55: task do_patch: started
NOTE: package base-files-3.0.14-r55: task do_patch: completed
NOTE: package base-files-3.0.14-r55: task do_configure: started
NOTE: package base-files-3.0.14-r55: task do_configure: completed
NOTE: package base-files-3.0.14-r55: task do_compile: started
NOTE: package base-files-3.0.14-r55: task do_compile: completed
NOTE: package base-files-3.0.14-r55: task do_install: started
NOTE: package base-files-3.0.14-r55: task do_install: completed
NOTE: package base-files-3.0.14-r55: task do_package: started
NOTE: package base-files-3.0.14-r55: task do_package: completed
NOTE: package base-files-3.0.14-r55: task do_package_write: started
NOTE: package base-files-3.0.14-r55: task do_package_write: completed
NOTE: package base-files-3.0.14-r55: task do_populate_staging: started
NOTE: package base-files-3.0.14-r55: task do_populate_staging: completed
NOTE: package base-files-3.0.14-r55: task do_build: started
NOTE: package base-files-3.0.14-r55: task do_build: completed
NOTE: package base-files-3.0.14-r55: task do_rm_work: started
NOTE: package base-files-3.0.14-r55: task do_rm_work: completed
NOTE: package base-files-3.0.14: completed
NOTE: Tasks Summary: Attempted 204 tasks of which 193 didn't need to be rerun and 0 failed.
NOTE: build 200704110852: completed
08:53 hrw@home:build$                                                       

-- 
JID: hrw-jabber.org
OpenEmbedded developer/consultant

                        If it works, don't fix it.
                        		-- Sam Rayburn

Re: [RFC] rm_work

[Richard Purdie]

On Wed, 2007-04-11 at 08:55 +0200, Marcin Juszkiewicz wrote:
> Dnia środa, 11 kwietnia 2007, Marcin Juszkiewicz napisał:
> 
> > Maybe rm_work.bbclass should do:
> >
> > BB_DEFAULT_TASK = "rm_work"
> > addtask rm_work after do_build
> >
> > But I did not tested it yet (got that idea during Easter).
> 
> Looks like it is a way - atleast it works in Poky:

Its a nice way to cheat but that might not work for bitbake 1.6? If it
doesn't work for 1.6, it could probably get backported easily enough. I
would like to see support for specifying multiple tasks to the addtask
syntax though to solve this problem properly.

Cheers,

Richard

Re: [RFC] rm_work

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marcin Juszkiewicz schreef:

> Maybe rm_work.bbclass should do:
> 
> BB_DEFAULT_TASK = "rm_work"
> addtask rm_work after do_build

This patch works for me:

koen@bitbake:~/OE/monotone/org.openembedded.dev$ mtn diff classes/rm_work.bbclass
#
# old_revision [59b29b63107f2448a7c5cb337360ab2aff6624e3]
#
# patch "classes/rm_work.bbclass"
#  from [00ead2158340f64f924a89c99a7b91b3a0a18c85]
#    to [6c2338de624467c193c7c1e9790b4ac7ed172ef4]
#
============================================================
- --- classes/rm_work.bbclass     00ead2158340f64f924a89c99a7b91b3a0a18c85
+++ classes/rm_work.bbclass     6c2338de624467c193c7c1e9790b4ac7ed172ef4
@@ -6,6 +6,9 @@
 # INHERIT += "rm_work"
 #

+BB_DEFAULT_TASK = "rm_work"
+addtask rm_work after do_build
+
 do_rm_work () {
     cd ${WORKDIR}
     for dir in *
@@ -18,5 +21,3 @@ do_rm_work () {
     done
 }

- -addtask rm_work before do_build
- -addtask rm_work after do_populate_staging
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFGH9mvMkyGM64RGpERAnzVAJ4yyRXj1IExTxpuojspwrKoAxrmKgCcD0Ni
gPZAcEuVRyTDpJk13KShCis=
=tw4Q
-----END PGP SIGNATURE-----

Re: [RFC] rm_work

[Richard Purdie]

On Fri, 2007-04-13 at 21:27 +0200, Koen Kooi wrote:
> Marcin Juszkiewicz schreef:
> 
> > Maybe rm_work.bbclass should do:
> > 
> > BB_DEFAULT_TASK = "rm_work"
> > addtask rm_work after do_build
> 
> This patch works for me:

This makes me very nervous since thats not how that variable was
intended to be used and I don't think its a good idea in the long term.
Can we try and fix bitbake to allow the correct syntax needed and then
write the metadata in a less invasive way?

That trick works precisely once if you want an illustration of why its
bad.

Richard

Re: [RFC] rm_work

[Florian Boor]

Hi,

Holger Freyther schrieb:
> how many recipes still break with rm_work, is there a documentation  
> which tasks to add and link to unbreak rm_work (if it breaks). I'm  
> still uncertain if the cleaning should be done by bitbake itself.

I'm using it for some time now - the only trouble I ran into were some breaking
kernel packages. But if there are still some broken remaining these would be
easy to fix.

Greetings

Florian

-- 
The dream of yesterday                  Florian Boor
is the hope of today                    Tel: +49 271-771091-15
and the reality of tomorrow.            Fax: +49 271-771091-19
[Robert Hutchings Goddard, 1904]        florian.boor@kernelconcepts.de

1D78 2D4D 6C53 1CA4 5588  D07B A8E7 940C 25B7 9A76

Re: [RFC] rm_work

[Richard Purdie]

Hi,

On Sat, 2007-04-07 at 12:11 +0200, Koen Kooi wrote:
> I want to propose making rm_work inherited by default with the following changes:
> 
> * a KEEP_WORK = "1" var in bitbake.conf so it is opt-in
> * a KEEP_WORK_pn-${PN} option to mark packages in local.conf
> * a KEEP_WORK option in recipes to mark them, e.g. for llvm
> 
> Summary: it will default to 'off' (keep the workdir), but can be switched globally and per
> package.
> 
> In the last few weeks that I've used rm_work, I haven't found any big problems with it,
> but as Graeme points out, there is no easy way to disable yet.
> 
> What do you think?

As I see it, rm_work is a policy which can be applied by an individual
or a distro. I don't think it should be mandatory but if certain distros
wish to use it be default, that is up to them and I'm fine with that.

Cheers,

Richard

Re: [RFC] rm_work

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Richard Purdie schreef:
> Hi,
> 
> On Sat, 2007-04-07 at 12:11 +0200, Koen Kooi wrote:
>> I want to propose making rm_work inherited by default with the following changes:
>>
>> * a KEEP_WORK = "1" var in bitbake.conf so it is opt-in
>> * a KEEP_WORK_pn-${PN} option to mark packages in local.conf
>> * a KEEP_WORK option in recipes to mark them, e.g. for llvm
>>
>> Summary: it will default to 'off' (keep the workdir), but can be switched globally and per
>> package.
>>
>> In the last few weeks that I've used rm_work, I haven't found any big problems with it,
>> but as Graeme points out, there is no easy way to disable yet.
>>
>> What do you think?
> 
> As I see it, rm_work is a policy which can be applied by an individual
> or a distro. I don't think it should be mandatory but if certain distros
> wish to use it be default, that is up to them and I'm fine with that.

Right, my proposal does that, but skips the need to add it to INHERITS manually. It's a
small nuance, but not big enough to discuss further :)

regards,

Koen

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFGGR9NMkyGM64RGpERAs8HAJ9TRAdrubJyaCt4ZALHVoLsIX8yowCglcVy
XWzFX4zfjMoX8wuSA/sBxqw=
=SvAE
-----END PGP SIGNATURE-----

Re: [RFC] rm_work

[Richard Purdie]

On Sun, 2007-04-08 at 19:01 +0200, Koen Kooi wrote:
> Richard Purdie schreef:
> > As I see it, rm_work is a policy which can be applied by an individual
> > or a distro. I don't think it should be mandatory but if certain distros
> > wish to use it be default, that is up to them and I'm fine with that.
> 
> Right, my proposal does that, but skips the need to add it to INHERITS manually. It's a
> small nuance, but not big enough to discuss further :)

Well, your proposed solution requires a magic new variable, using
INHERITS requires no new magic variable and use a standardised method
already used for other such features...

Lets not over complicate things...

Cheers,

Richard