forcing a task to run while building an image

forcing a task to run while building an image

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

Hello all,
	How do I force a particular task to run every time while building an
image? At present in my custom image recipe I've added the following
lines
        
        SOME_VARIABLE = "Blah"
        do_bootimg[depends] += "base-files:do_install"
        
But the contents of ${SOME_VARIABLE} is not available in the
'base-files' recipe. Even

        export SOME_VARIABLE = "Blah"
        
doesn't solve the problem. So, I would like to know whether the approach
used here to force a task is correct. Moreover how can I make the
variable defined here to be available in the forced task.

Regards
Joshua

-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: forcing a task to run while building an image

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

Hello,

On Tue, 2012-01-31 at 12:40 +0530, Joshua Immanuel wrote:
> At present in my custom image recipe I've added the following
> lines
>         
>         SOME_VARIABLE = "Blah"
>         do_bootimg[depends] += "base-files:do_install"
>         
> But the contents of ${SOME_VARIABLE} is not available in the
> 'base-files' recipe. Even
> 
>         export SOME_VARIABLE = "Blah"
>         
> doesn't solve the problem. 

        $ bitbake custom-image -c devshell

In the above devshell I can find the contents of ${SOME_VARIABLE}
available but the content of it is empty in 'base-files' recipe. Can
someone guide me?

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

creating global variables in a recipes

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 1188 bytes --]

Hello all,

On Wed, 2012-02-01 at 12:07 +0530, Joshua Immanuel wrote:
> > At present in my custom image recipe I've added the following
> > lines
> >         
> >         SOME_VARIABLE = "Blah"
> >         do_bootimg[depends] += "base-files:do_install"
> >         
> > But the contents of ${SOME_VARIABLE} is not available in the
> > 'base-files' recipe. Even
> > 
> >         export SOME_VARIABLE = "Blah"
> >         
> > doesn't solve the problem. 
> 
>         $ bitbake custom-image -c devshell
> 
> In the above devshell I can find the contents of ${SOME_VARIABLE}
> available but the content of it is empty in 'base-files' recipe. 

In other words, I need to define global variables in a package which can
be used across multiple packages. At present the only option I have is
to export the variable in build/conf/local.conf. IMHO, this doesn't feel
like a proper solution (as the variable is my layer specific). Moreover,
I can't change the value of it in one package to be used in other
package(s).

Is there any other way to do this? Please guide me.

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: creating global variables in a recipes

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1790 bytes --]

Nope. Recipes cannot modify the metadata of other recipes or the configuration metadata, ever. In particular cases, we emit content to tmpdir and read it back in elsewhere, but in general you should just not do what you're trying to do, and find a better solution.

-- 
Christopher Larson


On Wednesday, February 1, 2012 at 8:49 AM, Joshua Immanuel wrote:

> Hello all,
> 
> On Wed, 2012-02-01 at 12:07 +0530, Joshua Immanuel wrote:
> > > At present in my custom image recipe I've added the following
> > > lines
> > > 
> > > SOME_VARIABLE = "Blah"
> > > do_bootimg[depends] += "base-files:do_install"
> > > 
> > > But the contents of ${SOME_VARIABLE} is not available in the
> > > 'base-files' recipe. Even
> > > 
> > > export SOME_VARIABLE = "Blah"
> > > 
> > > doesn't solve the problem. 
> > > 
> > 
> > 
> > $ bitbake custom-image -c devshell
> > 
> > In the above devshell I can find the contents of ${SOME_VARIABLE}
> > available but the content of it is empty in 'base-files' recipe. 
> > 
> 
> 
> In other words, I need to define global variables in a package which can
> be used across multiple packages. At present the only option I have is
> to export the variable in build/conf/local.conf. IMHO, this doesn't feel
> like a proper solution (as the variable is my layer specific). Moreover,
> I can't change the value of it in one package to be used in other
> package(s).
> 
> Is there any other way to do this? Please guide me.
> 
> Regards
> Joshua
> -- 
> Joshua Immanuel
> HiPro IT Solutions Private Limited
> http://hipro.co.in
> 
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org (mailto:yocto@yoctoproject.org)
> https://lists.yoctoproject.org/listinfo/yocto
> 
> 



[-- Attachment #2: Type: text/html, Size: 2875 bytes --]

Re: creating global variables in a recipes

[William Mills]

On 02/01/2012 10:49 AM, Joshua Immanuel wrote:
> Hello all,
>
> On Wed, 2012-02-01 at 12:07 +0530, Joshua Immanuel wrote:
>>> At present in my custom image recipe I've added the following
>>> lines
>>>
>>>          SOME_VARIABLE = "Blah"
>>>          do_bootimg[depends] += "base-files:do_install"
>>>
>>> But the contents of ${SOME_VARIABLE} is not available in the
>>> 'base-files' recipe. Even
>>>
>>>          export SOME_VARIABLE = "Blah"
>>>
>>> doesn't solve the problem.
>>
>>          $ bitbake custom-image -c devshell
>>
>> In the above devshell I can find the contents of ${SOME_VARIABLE}
>> available but the content of it is empty in 'base-files' recipe.
>
> In other words, I need to define global variables in a package which can
> be used across multiple packages. At present the only option I have is
> to export the variable in build/conf/local.conf. IMHO, this doesn't feel
> like a proper solution (as the variable is my layer specific). Moreover,
> I can't change the value of it in one package to be used in other
> package(s).
>
> Is there any other way to do this? Please guide me.
>

I am not sure I can give you a better option that local.conf but I can 
explain why what your doing does not work.

It may feel from your usage that the image recipe is the ancestor of all 
the recipes but this is really not true.  The settings in a image recipe 
effect the assembly of that image and not the packages that they depend 
on.  An image could be assembled from packages that were built long ago 
or inherited via shared state.

settings in local,conf on the other hand affect all recipes.  If you add 
something there it will be seen by all recipes.  Unfortunately this 
means that all recipes are dependent on the settings and everything will 
need to be rebuilt in case the new setting effects them.  I believe this 
also means you will not be able to used share state with someone with a 
different setting of (or unset) SOME_VARIABLE.  (Well you can but you 
will both be rebuild everything.)

Alternatives to the whole approach are to make alternative packages for 
the various cases (busybox-minimal vs busybox-full) or use an image task 
to create an asset in the file system that then makes a difference at 
run time.

> Regards
> Joshua
>
>
>
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: creating global variables in a recipes

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 1952 bytes --]

Hello William, Christopher,

Thanks for the detailed explanation.

On Wed, 2012-02-01 at 11:50 -0500, William Mills wrote:
> I am not sure I can give you a better option that local.conf but I can
> explain why what your doing does not work.
> 
> It may feel from your usage that the image recipe is the ancestor of
> all the recipes but this is really not true.  The settings in a image
> recipe effect the assembly of that image and not the packages that
> they depend on.  An image could be assembled from packages that were
> built long ago or inherited via shared state.
> 
> settings in local,conf on the other hand affect all recipes.  If you
> add something there it will be seen by all recipes.  Unfortunately
> this means that all recipes are dependent on the settings and
> everything will need to be rebuilt in case the new setting effects
> them.  I believe this also means you will not be able to used share
> state with someone with a different setting of (or unset)
> SOME_VARIABLE.  (Well you can but you will both be rebuild
> everything.)
> 
> Alternatives to the whole approach are to make alternative packages
> for the various cases (busybox-minimal vs busybox-full) or use an
> image task to create an asset in the file system that then makes a
> difference at run time. 

My use-case is that I am creating an /etc/fstab file via
base-files.bbappend recipe. In that fstab, I use uuid (generated via
uuidgen) to denote different partition filesystems. So, I have to use
this uuid in the extended image_types.bbclass file to set them to the
generated partition images appropriately (via tune2fs).

One more approach I can think of is to echo the uuid to some file in the
deploy images directory (while creating the fstab file) and read them
back in image_types class. But this process looks messy.

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: creating global variables in a recipes

[Chris Larson]

On Wed, Feb 1, 2012 at 10:17 PM, Joshua Immanuel <josh@hipro.co.in> wrote:
> On Wed, 2012-02-01 at 11:50 -0500, William Mills wrote:
>> I am not sure I can give you a better option that local.conf but I can
>> explain why what your doing does not work.
>>
>> It may feel from your usage that the image recipe is the ancestor of
>> all the recipes but this is really not true.  The settings in a image
>> recipe effect the assembly of that image and not the packages that
>> they depend on.  An image could be assembled from packages that were
>> built long ago or inherited via shared state.
>>
>> settings in local,conf on the other hand affect all recipes.  If you
>> add something there it will be seen by all recipes.  Unfortunately
>> this means that all recipes are dependent on the settings and
>> everything will need to be rebuilt in case the new setting effects
>> them.  I believe this also means you will not be able to used share
>> state with someone with a different setting of (or unset)
>> SOME_VARIABLE.  (Well you can but you will both be rebuild
>> everything.)
>>
>> Alternatives to the whole approach are to make alternative packages
>> for the various cases (busybox-minimal vs busybox-full) or use an
>> image task to create an asset in the file system that then makes a
>> difference at run time.
>
> My use-case is that I am creating an /etc/fstab file via
> base-files.bbappend recipe. In that fstab, I use uuid (generated via
> uuidgen) to denote different partition filesystems. So, I have to use
> this uuid in the extended image_types.bbclass file to set them to the
> generated partition images appropriately (via tune2fs).
>
> One more approach I can think of is to echo the uuid to some file in the
> deploy images directory (while creating the fstab file) and read them
> back in image_types class. But this process looks messy.

I'd say generate them in the configuration metadata via inline python
or an anonymous python function, then use those variables both in the
base-files recipe and images.
-- 
Christopher Larson

Re: creating global variables in a recipes

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]

Hello Chris,

Thanks for the reply.

On Thu, 2012-02-02 at 08:17 -0700, Chris Larson wrote:
> > My use-case is that I am creating an /etc/fstab file via
> > base-files.bbappend recipe. In that fstab, I use uuid (generated via
> > uuidgen) to denote different partition filesystems. So, I have to
> > use this uuid in the extended image_types.bbclass file to set them
> > to the generated partition images appropriately (via tune2fs).
> >
> > One more approach I can think of is to echo the uuid to some file in
> > the deploy images directory (while creating the fstab file) and read
> > them back in image_types class. But this process looks messy.
> 
> I'd say generate them in the configuration metadata via inline python
> or an anonymous python function, then use those variables both in the
> base-files recipe and images. 

Please bear with my ignorance. What do you mean by "generating them in
configuration metadata"? In which file should I do that?

At present as an ugly hack I exported the uuids in the
build/conf/local.conf. 

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: creating global variables in a recipes

[Richard Purdie]

On Wed, 2012-02-01 at 11:50 -0500, William Mills wrote:
> I am not sure I can give you a better option that local.conf but I can 
> explain why what your doing does not work.
> 
> It may feel from your usage that the image recipe is the ancestor of all 
> the recipes but this is really not true.  The settings in a image recipe 
> effect the assembly of that image and not the packages that they depend 
> on.  An image could be assembled from packages that were built long ago 
> or inherited via shared state.
> 
> settings in local,conf on the other hand affect all recipes.  If you add 
> something there it will be seen by all recipes.  Unfortunately this 
> means that all recipes are dependent on the settings and everything will 
> need to be rebuilt in case the new setting effects them.  I believe this 
> also means you will not be able to used share state with someone with a 
> different setting of (or unset) SOME_VARIABLE.  (Well you can but you 
> will both be rebuild everything.)

Just a small clarification:

With sstate, it actually depends whether SOME_VARIABLE is used by the
given sstate task or its dependencies. We have variable dependency
tracking code and can tell where variables are used and with what value.

The end result is you can have different local.conf files and share
sstate output in many cases, but real differences to the individual task
inputs will get accounted for.

Cheers

Richard

Re: creating global variables in a recipes

[William Mills]

On 02/02/2012 05:27 PM, Richard Purdie wrote:
> Just a small clarification:

Thanks Richard.  I wondered about that but figured you would jump in if 
I over simplified.

Re: forcing a task to run while building an image

[Sathishkumar Duraisamy]

On Tue, Jan 31, 2012 at 12:40 PM, Joshua Immanuel <josh@hipro.co.in> wrote:
> Hello all,
>        How do I force a particular task to run every time while building an
> image?

You may have look at this
http://docs.openembedded.org/usermanual/usermanual.html#usage_workwithsinglerecipe.

I think, image are build from the packages that are build from the
recipe. And there is no way for the bitbake to detect that source is
changed like the makefile.

But there is always another way to do things we need. Here is my tip,
you can do to this.( need comments on this).

add a task in your recipe, like,

do_mytask() {
    #add commands to clean and do your stuff.
    #you can also pull from version control.
    #ordinary shell commands , need not to master anythings.

}

addtask do_mytask before do_build

add this to one of your recipe to build the image, now use bitbake -b
<your recipe> -c <do_mytask()>



-- 
Regards,
Sathishkumar D
http://flowersopenlab.weebly.com/

Re: forcing a task to run while building an image

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

Hello Sathishkumar,

On Wed, 2012-02-01 at 16:34 +0530, Sathishkumar Duraisamy wrote:
> I think, image are build from the packages that are build from the
> recipe. And there is no way for the bitbake to detect that source is
> changed like the makefile.

My use-case is such that I need to run a particular task irrespective
of the changes in that recipe. By adding the task to do_bootimg[depends]
I was able to accomplish that. (I am not sure whether it is the
preferred way of doing it)

However, the variables defined in the custom image recipe were not
getting exposed to the task added to the do_bootimg[depends]. Thats the
part that I've to figure out (need help).

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: forcing a task to run while building an image

[Sathishkumar Duraisamy]

On Wed, Feb 1, 2012 at 5:12 PM, Joshua Immanuel <josh@hipro.co.in> wrote:
> Hello Sathishkumar,
hello
> On Wed, 2012-02-01 at 16:34 +0530, Sathishkumar Duraisamy wrote:
>> I think, image are build from the packages that are build from the
>> recipe. And there is no way for the bitbake to detect that source is
>> changed like the makefile.
>
> My use-case is such that I need to run a particular task irrespective
> of the changes in that recipe. By adding the task to do_bootimg[depends]
> I was able to accomplish that. (I am not sure whether it is the
> preferred way of doing it)
>
> However, the variables defined in the custom image recipe were not
> getting exposed to the task added to the do_bootimg[depends]. Thats the
> part that I've to figure out (need help).

I understood what you trying to mean. Can you please clarify what
image means, whether final image of rfs like jffs2 or package from a
recipe.

-- 
Regards,
Sathishkumar D
http://flowersopenlab.weebly.com/

Re: forcing a task to run while building an image

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 400 bytes --]

Hello Sathishkumar,

On Wed, 2012-02-01 at 17:52 +0530, Sathishkumar Duraisamy wrote:
> I understood what you trying to mean. Can you please clarify what
> image means, whether final image of rfs like jffs2 or package from a
> recipe.

I meant the final rootfs image as in 'core-image-minimal'.

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: forcing a task to run while building an image

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

Hello,

On Wed, 2012-02-01 at 17:12 +0530, Joshua Immanuel wrote:
> My use-case is such that I need to run a particular task irrespective
> of the changes in that recipe. By adding the task to
> do_bootimg[depends] I was able to accomplish that. (I am not sure
> whether it is the preferred way of doing it) 

After removing the INHERIT += "rm_work" from my build/conf/local.conf,
the dependency added to following

        do_bootimg[depends] += "base-files:do_install"
        
failed to force rebuild. So, again I am stuck with local.conf file
manipulation :(

I guess there should be a cleaner approach to this than doing

        $ bitbake -c clean base-files
        $ bitbake custom-image
        
the above, every time I build my custom image.

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: forcing a task to run while building an image

[Sathishkumar Duraisamy]

The following is the scenario, you are in. If I wrong, please correct me.
" You have your development program/application in some place. You
want to build and include it in your rootfs. And you don't want to
edit the recipe/other things for each build of rootfs."

The scenario is very common. What we actually do is, for development
we usually test by downloading the specific application binary to rfs
by compiling with the sdk (created from oe). And usually our
application will be in some version control say hg or git. Once the
apps is ready, modify the recipe to include the particular version of
apps.


-- 
Regards,
Sathishkumar D
http://flowersopenlab.weebly.com/

Re: forcing a task to run while building an image

[Joshua Immanuel]

[-- Attachment #1: Type: text/plain, Size: 2019 bytes --]

Hello Sathishkumar,

On Fri, 2012-02-03 at 14:21 +0530, Sathishkumar Duraisamy wrote:
> The following is the scenario, you are in. If I wrong, please correct
> me.
> " You have your development program/application in some place. You
> want to build and include it in your rootfs. And you don't want to
> edit the recipe/other things for each build of rootfs." 

Let me explain (though I've mentioned in many places in bits and
pieces). At present situation, when we build an image (say
'core-image-minimal') everything goes under one partition. But, for my
use-case I am trying to generate two sets of images denoting different
partitions as I already mentioned in
https://lists.yoctoproject.org/pipermail/yocto/2012-January/006611.html
So, in order to do that, I've to identify the different partitions in my
etc/fstab file. This is done by denoting filesystems by their uuids. 

So, every time I build my custom-image I have to construct
the /etc/fstab file with the generated uuids (by using uuidgen) and set
the same uuids in the generated images using tune2fs. Setting the uuid
to generated images is done by overriding image_types.bbclass file. 

Similarly constructing the fstab file is done by adding
base-files_3.0.14.bbappend in my custom layer.

So, I have to force the base-files:do_install task to run whenever we
build the custom-image. Otherwise the uuids in the /etc/fstab will be
the older ones and the uuids set to the generated images will be the
newer ones. They will not match thereby defeating the whole purpose of
building partition based images.

In this context, I was trying to create a global variables to store the
uuids in order to be shared across multiple packages (i.e base-files
and image_types) But that too doesn't look like a piece of cake.

Am I trying something too weird?

Sorry for bombarding the list over and over again with the same issue.

Regards
Joshua
-- 
Joshua Immanuel
HiPro IT Solutions Private Limited
http://hipro.co.in

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]