Source Archiver Class

Source Archiver Class

[Saul Wold]

This is clearly coming to a head and we need to a better definition of 
what we are doing and direction we are going.  We recently had the 2 
different implementation suggested on the list to accomplish WORKDIR 
archiving along with the existing sourcepkg, distribute_src, 
copyleft_compliance and other possibly others.

My understanding of what we are trying to tackle include a variety of 
legal and policy requirements.

This is a progression list of what the source archives should include:

1 - Original Upstream Archive & Patches
     - 2 archives (tarballs)
2 - Original Source code & Patches
     - could include additional code
     - post unpack
     - 1 archive
3 - Original Source code & Patches & temp (scripts & logs)
     - Could possibly include the .bb and .inc files
     - 2 archives (from #2 & temp tarball)
4 - Patched source code
     - original source code could be removed
     - post do_patch
5 - Patched source code & temp
     - 2 archives (from #4 & temp tarball)
6 - Configured Source
     - post do_configure
7 - SRPM format of Original Source & Patches
   - rpm will apply the patches
   - internally contains #2 above
8 - SRPM various of #3 above
...
100 - Buildable SRPM
   - can actually build, this is way future!

(Patches = patch files & series list)

Each of these build on the previous in some way, the key being that we 
generate a tarball for the source from the existing state of the 
WORKDIR, a challenge maybe to create the source snapshot after the build 
has already occurred.

The SPDX License info could also be included in any of these

After talking with Richard, we think we have a novel approach to make 
this work. It would entail using the postfuncs feature similar to the 
way that Shared State does it's work. The existing copyleft_compliance 
class functionality can be folded into this as a filter. Additional 
flags could be passed from individual classes to a core set of methods 
in an archiver class defining the type of data (source, patches, temp, 
env, recipe info, ...) and format (tar, sprm, ...)

I noted that recipe type code might be better suited as generic code, as 
I believe there are other places that could benefit from that code.

The sourcepkg class seems to be a basic archiver and differ that 
includes the metadata/environment (as dumpdata), this could be replaced 
by the new approach.  While the src_distribute class copies the 
downloaded archive and then creates a link, into LICENSE directories 
along with the patches. The src_distribute by default seems to move 
files and create links (incorrectly btw!). This work can be done by the 
archiver class.

The Nugget: Create a new core "archiver" class that implements a general 
functions that can archive the original tarball or workdir at various 
states along task list with additional metadata (recipe info, temp dir, 
environment). This class would be inherited by a set of classes that use 
the postfunc (similar to sstate) that setup what level of archive is 
needed (based on the list above).

Thoughts, Comments?

Thanks



-- 
     Sau!

Saul Wold
Yocto Component Wrangler @ Intel
Yocto Project / Poky Build System

Re: [oe] Source Archiver Class

[Xiaofeng Yan]

Hi Saul,

I have some issues when I writing design document. The following 
description is my  understanding. I take package "zlib" for example .
>
>
> This is a progression list of what the source archives should include:
>
> 1 - Original Upstream Archive & Patches
>     - 2 archives (tarballs)
$ls zlib-orgsource
zlib-1.2.5.tar.bz2(after do_fetch)
zlib-patches.tar.bz2( the patches come from 
meta/recipes-core/zlib/zlib-1.2.5)
> 2 - Original Source code & Patches
>     - could include additional code
          Please give me more detailed information
>     - post unpack
>     - 1 archive
$ls zlib-orgsource
zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this 
package.)

No patch in this directory.
>
> 3 - Original Source code & Patches & temp (scripts & logs)
>     - Could possibly include the .bb and .inc files
>     - 2 archives (from #2 & temp tarball)
$ls zlib-orgsource
  zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this 
package.)
  zlib-patches.tar.bz2( the patches come from 
meta/recipes-core/zlib/zlib-1.2.5)
  zlib-scripts-logs.tar.bz2(include zlib_1.2.5.bb,zlib.inc and tmp/log_*)

So there are 3 tarballs in this directory.
> 4 - Patched source code
>     - original source code could be removed
>     - post do_patch
$ls zlib-source
zlib-1.2.5.tar.bz2(after do_patch)
> 5 - Patched source code & temp
>     - 2 archives (from #4 & temp tarball)
$ls zlib-source
zlib-1.2.5.tar.bz2(after do_patch)
zlib-logs.tar.bz2(include tmp/log_*)
> 6 - Configured Source
>     - post do_configure
$ls zlib-source
zlib-1.2.5.tar.bz2(after do_configure)
> 7 - SRPM format of Original Source & Patches
>   - rpm will apply the patches
>   - internally contains #2 above
$ls zlib-srpm
zlib-1.2.5.src.rpm

zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches 
are not included in this package.) and zlib-patches.tar.bz2
> 8 - SRPM various of #3 above
$ls zlib-srpm
zlib-1.2.5.src.rpm

zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches 
are not included in this package.) ,zlib-patches.tar.bz2 and 
zlib-scripts-logs.tar.bz2

Do you have any suggestion about the above description?

Thanks
Yan


> ...
> 100 - Buildable SRPM
>   - can actually build, this is way future!
>
> (Patches = patch files & series list)
>
> Each of these build on the previous in some way, the key being that we
> generate a tarball for the source from the existing state of the
> WORKDIR, a challenge maybe to create the source snapshot after the build
> has already occurred.
>
> The SPDX License info could also be included in any of these
>
> After talking with Richard, we think we have a novel approach to make
> this work. It would entail using the postfuncs feature similar to the
> way that Shared State does it's work. The existing copyleft_compliance
> class functionality can be folded into this as a filter. Additional
> flags could be passed from individual classes to a core set of methods
> in an archiver class defining the type of data (source, patches, temp,
> env, recipe info, ...) and format (tar, sprm, ...)
>
> I noted that recipe type code might be better suited as generic code, as
> I believe there are other places that could benefit from that code.
>
> The sourcepkg class seems to be a basic archiver and differ that
> includes the metadata/environment (as dumpdata), this could be replaced
> by the new approach.  While the src_distribute class copies the
> downloaded archive and then creates a link, into LICENSE directories
> along with the patches. The src_distribute by default seems to move
> files and create links (incorrectly btw!). This work can be done by the
> archiver class.
>
> The Nugget: Create a new core "archiver" class that implements a general
> functions that can archive the original tarball or workdir at various
> states along task list with additional metadata (recipe info, temp dir,
> environment). This class would be inherited by a set of classes that use
> the postfunc (similar to sstate) that setup what level of archive is
> needed (based on the list above).
>
> Thoughts, Comments?
>
> Thanks
>
>
>

Re: [oe] Source Archiver Class

[Saul Wold]

On 02/13/2012 02:41 AM, Xiaofeng Yan wrote:
> Hi Saul,
>
> I have some issues when I writing design document. The following
> description is my understanding. I take package "zlib" for example .
>>
>>
>> This is a progression list of what the source archives should include:
>>
>> 1 - Original Upstream Archive & Patches
>> - 2 archives (tarballs)
> $ls zlib-orgsource
> zlib-1.2.5.tar.bz2(after do_fetch)
> zlib-patches.tar.bz2( the patches come from
> meta/recipes-core/zlib/zlib-1.2.5)

Almost correct, I believe that the work Bruce and Chris have done 
recently will allow you to get the patches.

I am not sure we want to just tar up the 
meta/recipes-core/zlib/lib-1.2.5 directory since it might contain other 
file or patches that we don't actaully use.  We need to get the patches 
that are actually applied.

>> 2 - Original Source code & Patches
>> - could include additional code
> Please give me more detailed information
One tarball with the patches extracted in the patches directory with the 
series file (possibly a script to apply the patches).

>> - post unpack
>> - 1 archive
> $ls zlib-orgsource
> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
> package.)
>
> No patch in this directory.
Need to add the patches via patch mechanism to the source tarball, but 
without actually applying the patches.

So, this is just one tarball.
zlib-1.2.5-prepatch.tar.bz2
>>
>> 3 - Original Source code & Patches & temp (scripts & logs)
>> - Could possibly include the .bb and .inc files
>> - 2 archives (from #2 & temp tarball)
> $ls zlib-orgsource
> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
> package.)
> zlib-patches.tar.bz2( the patches come from
> meta/recipes-core/zlib/zlib-1.2.5)

Again see above, we need to get only the patches that will actually be 
applied.

> zlib-scripts-logs.tar.bz2(include zlib_1.2.5.bb,zlib.inc and tmp/log_*)
>
> So there are 3 tarballs in this directory.
Yes

>> 4 - Patched source code
>> - original source code could be removed
>> - post do_patch
> $ls zlib-source
> zlib-1.2.5.tar.bz2(after do_patch)
Correct
Maybe better zlib-1.2.5-patched.tar.bz2

>> 5 - Patched source code & temp
>> - 2 archives (from #4 & temp tarball)
> $ls zlib-source
> zlib-1.2.5.tar.bz2(after do_patch)
> zlib-logs.tar.bz2(include tmp/log_*)
Correct
Same as #4 change the tarball  name

>> 6 - Configured Source
>> - post do_configure
> $ls zlib-source
> zlib-1.2.5.tar.bz2(after do_configure)
zlib-1.2.5-configured.tar.bz2

>> 7 - SRPM format of Original Source & Patches
>> - rpm will apply the patches
>> - internally contains #2 above
> $ls zlib-srpm
> zlib-1.2.5.src.rpm
>
> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
> are not included in this package.) and zlib-patches.tar.bz2
>> 8 - SRPM various of #3 above
> $ls zlib-srpm
> zlib-1.2.5.src.rpm
>
> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
> are not included in this package.) ,zlib-patches.tar.bz2 and
> zlib-scripts-logs.tar.bz2
>
> Do you have any suggestion about the above description?
>
Just the naming and where the patches come from we need to provide the 
mechanism (series file) for how to apply the patches also.

Sau!

> Thanks
> Yan
>
>
>> ...
>> 100 - Buildable SRPM
>> - can actually build, this is way future!
>>
>> (Patches = patch files & series list)
>>
>> Each of these build on the previous in some way, the key being that we
>> generate a tarball for the source from the existing state of the
>> WORKDIR, a challenge maybe to create the source snapshot after the build
>> has already occurred.
>>
>> The SPDX License info could also be included in any of these
>>
>> After talking with Richard, we think we have a novel approach to make
>> this work. It would entail using the postfuncs feature similar to the
>> way that Shared State does it's work. The existing copyleft_compliance
>> class functionality can be folded into this as a filter. Additional
>> flags could be passed from individual classes to a core set of methods
>> in an archiver class defining the type of data (source, patches, temp,
>> env, recipe info, ...) and format (tar, sprm, ...)
>>
>> I noted that recipe type code might be better suited as generic code, as
>> I believe there are other places that could benefit from that code.
>>
>> The sourcepkg class seems to be a basic archiver and differ that
>> includes the metadata/environment (as dumpdata), this could be replaced
>> by the new approach. While the src_distribute class copies the
>> downloaded archive and then creates a link, into LICENSE directories
>> along with the patches. The src_distribute by default seems to move
>> files and create links (incorrectly btw!). This work can be done by the
>> archiver class.
>>
>> The Nugget: Create a new core "archiver" class that implements a general
>> functions that can archive the original tarball or workdir at various
>> states along task list with additional metadata (recipe info, temp dir,
>> environment). This class would be inherited by a set of classes that use
>> the postfunc (similar to sstate) that setup what level of archive is
>> needed (based on the list above).
>>
>> Thoughts, Comments?
>>
>> Thanks
>>
>>
>>
>
>

Re: [oe] Source Archiver Class

[Saul Wold]

Xiaofeng,

I replied last night to your IRC, but was not sure if you saw it, I am 
adding it to this email.

(11:05:36 PM) sgw: xiaofeng, you around?
(11:08:22 PM) sgw: zlib is a bad example, since there are no patches!
(11:08:38 PM) sgw: Let's use "at" since it is a simple recipe and has 
patches
(11:09:27 PM) sgw: if you start with bitbake -c cleanall at, so we have 
a fresh start.
(11:12:26 PM) sgw: bitbake -c unpack at, will give you the upacked 
source in at-3.1.12-r7/at-3.1.12/..., this will give you a base to build 
from
(11:15:12 PM) sgw: "bitbake -c patch at" will then patch the above 
sources and create the patches directory with the series file.  So given 
that info, you be able to add a do_patch[prefunc] to first tar the 
unpatched directory and then a do_patch[postfunc] to tar the patches 
directory into the tar file from the first tar.
(11:16:32 PM) sgw: Does this make sense?
(11:17:49 PM) sgw: Something else we will want to consider is creating 
one tarball of all the gnu source and patches in one large tarball.


On 02/14/2012 08:51 AM, Saul Wold wrote:
> On 02/13/2012 02:41 AM, Xiaofeng Yan wrote:
>> Hi Saul,
>>
>> I have some issues when I writing design document. The following
>> description is my understanding. I take package "zlib" for example .
>>>
>>>
>>> This is a progression list of what the source archives should include:
>>>
>>> 1 - Original Upstream Archive & Patches
>>> - 2 archives (tarballs)
>> $ls zlib-orgsource
>> zlib-1.2.5.tar.bz2(after do_fetch)
>> zlib-patches.tar.bz2( the patches come from
>> meta/recipes-core/zlib/zlib-1.2.5)
>
> Almost correct, I believe that the work Bruce and Chris have done
> recently will allow you to get the patches.
>
> I am not sure we want to just tar up the
> meta/recipes-core/zlib/lib-1.2.5 directory since it might contain other
> file or patches that we don't actaully use. We need to get the patches
> that are actually applied.
>
>>> 2 - Original Source code & Patches
>>> - could include additional code
>> Please give me more detailed information
> One tarball with the patches extracted in the patches directory with the
> series file (possibly a script to apply the patches).
>
>>> - post unpack
>>> - 1 archive
>> $ls zlib-orgsource
>> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
>> package.)
>>
>> No patch in this directory.
> Need to add the patches via patch mechanism to the source tarball, but
> without actually applying the patches.
>
> So, this is just one tarball.
> zlib-1.2.5-prepatch.tar.bz2
>>>
>>> 3 - Original Source code & Patches & temp (scripts & logs)
>>> - Could possibly include the .bb and .inc files
>>> - 2 archives (from #2 & temp tarball)
>> $ls zlib-orgsource
>> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
>> package.)
>> zlib-patches.tar.bz2( the patches come from
>> meta/recipes-core/zlib/zlib-1.2.5)
>
> Again see above, we need to get only the patches that will actually be
> applied.
>
>> zlib-scripts-logs.tar.bz2(include zlib_1.2.5.bb,zlib.inc and tmp/log_*)
>>
>> So there are 3 tarballs in this directory.
> Yes
>
>>> 4 - Patched source code
>>> - original source code could be removed
>>> - post do_patch
>> $ls zlib-source
>> zlib-1.2.5.tar.bz2(after do_patch)
> Correct
> Maybe better zlib-1.2.5-patched.tar.bz2
>
>>> 5 - Patched source code & temp
>>> - 2 archives (from #4 & temp tarball)
>> $ls zlib-source
>> zlib-1.2.5.tar.bz2(after do_patch)
>> zlib-logs.tar.bz2(include tmp/log_*)
> Correct
> Same as #4 change the tarball name
>
>>> 6 - Configured Source
>>> - post do_configure
>> $ls zlib-source
>> zlib-1.2.5.tar.bz2(after do_configure)
> zlib-1.2.5-configured.tar.bz2
>
>>> 7 - SRPM format of Original Source & Patches
>>> - rpm will apply the patches
>>> - internally contains #2 above
>> $ls zlib-srpm
>> zlib-1.2.5.src.rpm
>>
>> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
>> are not included in this package.) and zlib-patches.tar.bz2
>>> 8 - SRPM various of #3 above
>> $ls zlib-srpm
>> zlib-1.2.5.src.rpm
>>
>> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
>> are not included in this package.) ,zlib-patches.tar.bz2 and
>> zlib-scripts-logs.tar.bz2
>>
>> Do you have any suggestion about the above description?
>>
> Just the naming and where the patches come from we need to provide the
> mechanism (series file) for how to apply the patches also.
>
> Sau!
>
>> Thanks
>> Yan
>>
>>
>>> ...
>>> 100 - Buildable SRPM
>>> - can actually build, this is way future!
>>>
>>> (Patches = patch files & series list)
>>>
>>> Each of these build on the previous in some way, the key being that we
>>> generate a tarball for the source from the existing state of the
>>> WORKDIR, a challenge maybe to create the source snapshot after the build
>>> has already occurred.
>>>
>>> The SPDX License info could also be included in any of these
>>>
>>> After talking with Richard, we think we have a novel approach to make
>>> this work. It would entail using the postfuncs feature similar to the
>>> way that Shared State does it's work. The existing copyleft_compliance
>>> class functionality can be folded into this as a filter. Additional
>>> flags could be passed from individual classes to a core set of methods
>>> in an archiver class defining the type of data (source, patches, temp,
>>> env, recipe info, ...) and format (tar, sprm, ...)
>>>
>>> I noted that recipe type code might be better suited as generic code, as
>>> I believe there are other places that could benefit from that code.
>>>
>>> The sourcepkg class seems to be a basic archiver and differ that
>>> includes the metadata/environment (as dumpdata), this could be replaced
>>> by the new approach. While the src_distribute class copies the
>>> downloaded archive and then creates a link, into LICENSE directories
>>> along with the patches. The src_distribute by default seems to move
>>> files and create links (incorrectly btw!). This work can be done by the
>>> archiver class.
>>>
>>> The Nugget: Create a new core "archiver" class that implements a general
>>> functions that can archive the original tarball or workdir at various
>>> states along task list with additional metadata (recipe info, temp dir,
>>> environment). This class would be inherited by a set of classes that use
>>> the postfunc (similar to sstate) that setup what level of archive is
>>> needed (based on the list above).
>>>
>>> Thoughts, Comments?
>>>
>>> Thanks
>>>
>>>
>>>
>>
>>
>

Re: [oe] Source Archiver Class

[Xiaofeng Yan]

On 2012年02月15日 23:23, Saul Wold wrote:
>
> Xiaofeng,
>
> I replied last night to your IRC, but was not sure if you saw it, I am 
> adding it to this email.
>
> (11:05:36 PM) sgw: xiaofeng, you around?
> (11:08:22 PM) sgw: zlib is a bad example, since there are no patches!
> (11:08:38 PM) sgw: Let's use "at" since it is a simple recipe and has 
> patches
> (11:09:27 PM) sgw: if you start with bitbake -c cleanall at, so we 
> have a fresh start.
> (11:12:26 PM) sgw: bitbake -c unpack at, will give you the upacked 
> source in at-3.1.12-r7/at-3.1.12/..., this will give you a base to 
> build from
> (11:15:12 PM) sgw: "bitbake -c patch at" will then patch the above 
> sources and create the patches directory with the series file. So 
> given that info, you be able to add a do_patch[prefunc] to first tar 
> the unpatched directory and then a do_patch[postfunc] to tar the 
> patches directory into the tar file from the first tar.
> (11:16:32 PM) sgw: Does this make sense?
> (11:17:49 PM) sgw: Something else we will want to consider is creating 
> one tarball of all the gnu source and patches in one large tarball.
>
>
Hi Saul,

I saw the reply in IRC yesterday.
Thanks very much. I will also realize a function to include all of gnu 
sources and patches in one tarball in archiver.bbclass.
> On 02/14/2012 08:51 AM, Saul Wold wrote:
>> On 02/13/2012 02:41 AM, Xiaofeng Yan wrote:
>>> Hi Saul,
>>>
>>> I have some issues when I writing design document. The following
>>> description is my understanding. I take package "zlib" for example .
>>>>
>>>>
>>>> This is a progression list of what the source archives should include:
>>>>
>>>> 1 - Original Upstream Archive & Patches
>>>> - 2 archives (tarballs)
>>> $ls zlib-orgsource
>>> zlib-1.2.5.tar.bz2(after do_fetch)
>>> zlib-patches.tar.bz2( the patches come from
>>> meta/recipes-core/zlib/zlib-1.2.5)
>>
>> Almost correct, I believe that the work Bruce and Chris have done
>> recently will allow you to get the patches.
>>
>> I am not sure we want to just tar up the
>> meta/recipes-core/zlib/lib-1.2.5 directory since it might contain other
>> file or patches that we don't actaully use. We need to get the patches
>> that are actually applied.
>>
>>>> 2 - Original Source code & Patches
>>>> - could include additional code
>>> Please give me more detailed information
>> One tarball with the patches extracted in the patches directory with the
>> series file (possibly a script to apply the patches).
>>
>>>> - post unpack
>>>> - 1 archive
>>> $ls zlib-orgsource
>>> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
>>> package.)
>>>
>>> No patch in this directory.
>> Need to add the patches via patch mechanism to the source tarball, but
>> without actually applying the patches.
>>
>> So, this is just one tarball.
>> zlib-1.2.5-prepatch.tar.bz2
>>>>
>>>> 3 - Original Source code & Patches & temp (scripts & logs)
>>>> - Could possibly include the .bb and .inc files
>>>> - 2 archives (from #2 & temp tarball)
>>> $ls zlib-orgsource
>>> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
>>> package.)
>>> zlib-patches.tar.bz2( the patches come from
>>> meta/recipes-core/zlib/zlib-1.2.5)
>>
>> Again see above, we need to get only the patches that will actually be
>> applied.
>>
>>> zlib-scripts-logs.tar.bz2(include zlib_1.2.5.bb,zlib.inc and tmp/log_*)
>>>
>>> So there are 3 tarballs in this directory.
>> Yes
>>
>>>> 4 - Patched source code
>>>> - original source code could be removed
>>>> - post do_patch
>>> $ls zlib-source
>>> zlib-1.2.5.tar.bz2(after do_patch)
>> Correct
>> Maybe better zlib-1.2.5-patched.tar.bz2
>>
>>>> 5 - Patched source code & temp
>>>> - 2 archives (from #4 & temp tarball)
>>> $ls zlib-source
>>> zlib-1.2.5.tar.bz2(after do_patch)
>>> zlib-logs.tar.bz2(include tmp/log_*)
>> Correct
>> Same as #4 change the tarball name
>>
>>>> 6 - Configured Source
>>>> - post do_configure
>>> $ls zlib-source
>>> zlib-1.2.5.tar.bz2(after do_configure)
>> zlib-1.2.5-configured.tar.bz2
>>
>>>> 7 - SRPM format of Original Source & Patches
>>>> - rpm will apply the patches
>>>> - internally contains #2 above
>>> $ls zlib-srpm
>>> zlib-1.2.5.src.rpm
>>>
>>> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
>>> are not included in this package.) and zlib-patches.tar.bz2
>>>> 8 - SRPM various of #3 above
>>> $ls zlib-srpm
>>> zlib-1.2.5.src.rpm
>>>
>>> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
>>> are not included in this package.) ,zlib-patches.tar.bz2 and
>>> zlib-scripts-logs.tar.bz2
>>>
>>> Do you have any suggestion about the above description?
>>>
>> Just the naming and where the patches come from we need to provide the
>> mechanism (series file) for how to apply the patches also.
>>
>> Sau!
>>
>>> Thanks
>>> Yan
>>>
>>>
>>>> ...
>>>> 100 - Buildable SRPM
>>>> - can actually build, this is way future!
>>>>
>>>> (Patches = patch files & series list)
>>>>
>>>> Each of these build on the previous in some way, the key being that we
>>>> generate a tarball for the source from the existing state of the
>>>> WORKDIR, a challenge maybe to create the source snapshot after the 
>>>> build
>>>> has already occurred.
>>>>
>>>> The SPDX License info could also be included in any of these
>>>>
>>>> After talking with Richard, we think we have a novel approach to make
>>>> this work. It would entail using the postfuncs feature similar to the
>>>> way that Shared State does it's work. The existing copyleft_compliance
>>>> class functionality can be folded into this as a filter. Additional
>>>> flags could be passed from individual classes to a core set of methods
>>>> in an archiver class defining the type of data (source, patches, temp,
>>>> env, recipe info, ...) and format (tar, sprm, ...)
>>>>
>>>> I noted that recipe type code might be better suited as generic 
>>>> code, as
>>>> I believe there are other places that could benefit from that code.
>>>>
>>>> The sourcepkg class seems to be a basic archiver and differ that
>>>> includes the metadata/environment (as dumpdata), this could be 
>>>> replaced
>>>> by the new approach. While the src_distribute class copies the
>>>> downloaded archive and then creates a link, into LICENSE directories
>>>> along with the patches. The src_distribute by default seems to move
>>>> files and create links (incorrectly btw!). This work can be done by 
>>>> the
>>>> archiver class.
>>>>
>>>> The Nugget: Create a new core "archiver" class that implements a 
>>>> general
>>>> functions that can archive the original tarball or workdir at various
>>>> states along task list with additional metadata (recipe info, temp 
>>>> dir,
>>>> environment). This class would be inherited by a set of classes 
>>>> that use
>>>> the postfunc (similar to sstate) that setup what level of archive is
>>>> needed (based on the list above).
>>>>
>>>> Thoughts, Comments?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>
>>>
>>
>

Re: [oe] Source Archiver Class

[Saul Wold]

Xiaofeng & Community:

We had a chat with Chris Larson and Mark Hatle here at ELC.

We focused this issue down to a fewer number of options that meet the 
needs of the licenses.  Please review this and let us know if there are 
any issues or concerns.


Best Practices:
  - Archive during build, we do not support nor recommend post archiving
  - Original tarball and patches w/ordering file (maybe comment the
    series), grab non-patch files also.
  - Unless requesting configured sources, which will just be the
    configured source tarball
  - Grab temp as postfunc of do_package - latest logs via links & pid

For sstate Builds (LATER):
  - need to verify that fetch/patch/configure will get re-run for archiving
  - Add temp dir to sstate capture (without links)

3 Implementations classes
  - Filter in oe/lib/license.py
  - source_archive.bb
    - archives tarballs to ${BP}/...
    - reuse copyleft_compliance for patch handling
  - srpm
    - take output from above and create SRPM - (LATER)

2 Configuration classes
  - uses prefunc/postfunc at correct place
  - Original Tarball / Patches
  - Post Configuration

SOURCE_ARCHIVE_PACKAGE_TYPE = {tar, srpm}
SOURCE_ARCHIVE_LOG = {True, False}

Please let us know if you have any questions.

Sau!

On 02/15/2012 05:19 PM, Xiaofeng Yan wrote:
> On 2012年02月15日 23:23, Saul Wold wrote:
>>
>> Xiaofeng,
>>
>> I replied last night to your IRC, but was not sure if you saw it, I am
>> adding it to this email.
>>
>> (11:05:36 PM) sgw: xiaofeng, you around?
>> (11:08:22 PM) sgw: zlib is a bad example, since there are no patches!
>> (11:08:38 PM) sgw: Let's use "at" since it is a simple recipe and has
>> patches
>> (11:09:27 PM) sgw: if you start with bitbake -c cleanall at, so we
>> have a fresh start.
>> (11:12:26 PM) sgw: bitbake -c unpack at, will give you the upacked
>> source in at-3.1.12-r7/at-3.1.12/..., this will give you a base to
>> build from
>> (11:15:12 PM) sgw: "bitbake -c patch at" will then patch the above
>> sources and create the patches directory with the series file. So
>> given that info, you be able to add a do_patch[prefunc] to first tar
>> the unpatched directory and then a do_patch[postfunc] to tar the
>> patches directory into the tar file from the first tar.
>> (11:16:32 PM) sgw: Does this make sense?
>> (11:17:49 PM) sgw: Something else we will want to consider is creating
>> one tarball of all the gnu source and patches in one large tarball.
>>
>>
> Hi Saul,
>
> I saw the reply in IRC yesterday.
> Thanks very much. I will also realize a function to include all of gnu
> sources and patches in one tarball in archiver.bbclass.


>> On 02/14/2012 08:51 AM, Saul Wold wrote:
>>> On 02/13/2012 02:41 AM, Xiaofeng Yan wrote:
>>>> Hi Saul,
>>>>
>>>> I have some issues when I writing design document. The following
>>>> description is my understanding. I take package "zlib" for example .
>>>>>
>>>>>
>>>>> This is a progression list of what the source archives should include:
>>>>>
>>>>> 1 - Original Upstream Archive & Patches
>>>>> - 2 archives (tarballs)
>>>> $ls zlib-orgsource
>>>> zlib-1.2.5.tar.bz2(after do_fetch)
>>>> zlib-patches.tar.bz2( the patches come from
>>>> meta/recipes-core/zlib/zlib-1.2.5)
>>>
>>> Almost correct, I believe that the work Bruce and Chris have done
>>> recently will allow you to get the patches.
>>>
>>> I am not sure we want to just tar up the
>>> meta/recipes-core/zlib/lib-1.2.5 directory since it might contain other
>>> file or patches that we don't actaully use. We need to get the patches
>>> that are actually applied.
>>>
>>>>> 2 - Original Source code & Patches
>>>>> - could include additional code
>>>> Please give me more detailed information
>>> One tarball with the patches extracted in the patches directory with the
>>> series file (possibly a script to apply the patches).
>>>
>>>>> - post unpack
>>>>> - 1 archive
>>>> $ls zlib-orgsource
>>>> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
>>>> package.)
>>>>
>>>> No patch in this directory.
>>> Need to add the patches via patch mechanism to the source tarball, but
>>> without actually applying the patches.
>>>
>>> So, this is just one tarball.
>>> zlib-1.2.5-prepatch.tar.bz2
>>>>>
>>>>> 3 - Original Source code & Patches & temp (scripts & logs)
>>>>> - Could possibly include the .bb and .inc files
>>>>> - 2 archives (from #2 & temp tarball)
>>>> $ls zlib-orgsource
>>>> zlib-1.2.5.tar.bz2(after do_unpack, patches are not included in this
>>>> package.)
>>>> zlib-patches.tar.bz2( the patches come from
>>>> meta/recipes-core/zlib/zlib-1.2.5)
>>>
>>> Again see above, we need to get only the patches that will actually be
>>> applied.
>>>
>>>> zlib-scripts-logs.tar.bz2(include zlib_1.2.5.bb,zlib.inc and tmp/log_*)
>>>>
>>>> So there are 3 tarballs in this directory.
>>> Yes
>>>
>>>>> 4 - Patched source code
>>>>> - original source code could be removed
>>>>> - post do_patch
>>>> $ls zlib-source
>>>> zlib-1.2.5.tar.bz2(after do_patch)
>>> Correct
>>> Maybe better zlib-1.2.5-patched.tar.bz2
>>>
>>>>> 5 - Patched source code & temp
>>>>> - 2 archives (from #4 & temp tarball)
>>>> $ls zlib-source
>>>> zlib-1.2.5.tar.bz2(after do_patch)
>>>> zlib-logs.tar.bz2(include tmp/log_*)
>>> Correct
>>> Same as #4 change the tarball name
>>>
>>>>> 6 - Configured Source
>>>>> - post do_configure
>>>> $ls zlib-source
>>>> zlib-1.2.5.tar.bz2(after do_configure)
>>> zlib-1.2.5-configured.tar.bz2
>>>
>>>>> 7 - SRPM format of Original Source & Patches
>>>>> - rpm will apply the patches
>>>>> - internally contains #2 above
>>>> $ls zlib-srpm
>>>> zlib-1.2.5.src.rpm
>>>>
>>>> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
>>>> are not included in this package.) and zlib-patches.tar.bz2
>>>>> 8 - SRPM various of #3 above
>>>> $ls zlib-srpm
>>>> zlib-1.2.5.src.rpm
>>>>
>>>> zlib-1.2.5.src.rpm includes zlib-1.2.5.tar.bz2(after do_unpack, patches
>>>> are not included in this package.) ,zlib-patches.tar.bz2 and
>>>> zlib-scripts-logs.tar.bz2
>>>>
>>>> Do you have any suggestion about the above description?
>>>>
>>> Just the naming and where the patches come from we need to provide the
>>> mechanism (series file) for how to apply the patches also.
>>>
>>> Sau!
>>>
>>>> Thanks
>>>> Yan
>>>>
>>>>
>>>>> ...
>>>>> 100 - Buildable SRPM
>>>>> - can actually build, this is way future!
>>>>>
>>>>> (Patches = patch files & series list)
>>>>>
>>>>> Each of these build on the previous in some way, the key being that we
>>>>> generate a tarball for the source from the existing state of the
>>>>> WORKDIR, a challenge maybe to create the source snapshot after the
>>>>> build
>>>>> has already occurred.
>>>>>
>>>>> The SPDX License info could also be included in any of these
>>>>>
>>>>> After talking with Richard, we think we have a novel approach to make
>>>>> this work. It would entail using the postfuncs feature similar to the
>>>>> way that Shared State does it's work. The existing copyleft_compliance
>>>>> class functionality can be folded into this as a filter. Additional
>>>>> flags could be passed from individual classes to a core set of methods
>>>>> in an archiver class defining the type of data (source, patches, temp,
>>>>> env, recipe info, ...) and format (tar, sprm, ...)
>>>>>
>>>>> I noted that recipe type code might be better suited as generic
>>>>> code, as
>>>>> I believe there are other places that could benefit from that code.
>>>>>
>>>>> The sourcepkg class seems to be a basic archiver and differ that
>>>>> includes the metadata/environment (as dumpdata), this could be
>>>>> replaced
>>>>> by the new approach. While the src_distribute class copies the
>>>>> downloaded archive and then creates a link, into LICENSE directories
>>>>> along with the patches. The src_distribute by default seems to move
>>>>> files and create links (incorrectly btw!). This work can be done by
>>>>> the
>>>>> archiver class.
>>>>>
>>>>> The Nugget: Create a new core "archiver" class that implements a
>>>>> general
>>>>> functions that can archive the original tarball or workdir at various
>>>>> states along task list with additional metadata (recipe info, temp
>>>>> dir,
>>>>> environment). This class would be inherited by a set of classes
>>>>> that use
>>>>> the postfunc (similar to sstate) that setup what level of archive is
>>>>> needed (based on the list above).
>>>>>
>>>>> Thoughts, Comments?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
>

Re: [oe] Source Archiver Class

[Xiaofeng Yan]

On 2012年02月18日 07:52, Saul Wold wrote:
>
> Xiaofeng & Community:
>
> We had a chat with Chris Larson and Mark Hatle here at ELC.
>
> We focused this issue down to a fewer number of options that meet the 
> needs of the licenses.  Please review this and let us know if there 
> are any issues or concerns.
>
>
Hi Saul,

I comment my understanding as follow:
> Best Practices:
>  - Archive during build, we do not support nor recommend post archiving
>  - Original tarball and patches w/ordering file (maybe comment the
>    series), grab non-patch files also.
         for example:
         file://a.patch
         file://b.config.in
         we also archive b.config.in.
>  - Unless requesting configured sources, which will just be the
>    configured source tarball
>  - Grab temp as postfunc of do_package - latest logs via links & pid
>
        That means logs package include all of logs in temp. for example.
        You has described a following function we will realize at the 
previous email.
       3 - Original Source code & Patches & temp (scripts & logs)

        source codes and patches should be in the stage 
do_patch[postfunc] = " do_get_source"
        logs should be in the stage do_package[postfunc] = " do_get_logs"
        So logs file shouldn't be archived in do_patch[postfunc] = " 
do_get_logs"
        The logs in the stage of do_packge are more than do_patch.

> For sstate Builds (LATER):
>  - need to verify that fetch/patch/configure will get re-run for 
> archiving
>  - Add temp dir to sstate capture (without links)
>
> 3 Implementations classes
>  - Filter in oe/lib/license.py
>  - source_archive.bb
>    - archives tarballs to ${BP}/...
>    - reuse copyleft_compliance for patch handling
>  - srpm
>    - take output from above and create SRPM - (LATER)
>
We will do the archive  work according the license .  I will implement 
two functions, one is for left, the other for right.
> 2 Configuration classes
>  - uses prefunc/postfunc at correct place
>  - Original Tarball / Patches
>  - Post Configuration
>
We can define 8 classes to complete  8 kinds of archiving methods.  
every class will inherit archiver.bbclass.
and then realize do_stage[prefunc/postfunc] in this class.
stage include unpack, patch, configure, build package and so on.
for example. we want only to get original tarball, then we can define a 
bbclass named "source.bbclass"
source.bbclass:
   inherit archiver.bbclass
   do_unpack[postfunc] = " do_get_source"

if we want to get both original and patches tarball, then we can define 
bbclass names "source_patches.bbclass"
source_patches.bbclass
    inherit archiver.bbclass
    do_unpack[postfunc] = " do_get_source_patches"

> SOURCE_ARCHIVE_PACKAGE_TYPE = {tar, srpm}
> SOURCE_ARCHIVE_LOG = {True, False}
>
> Please let us know if you have any questions.
>
> Sau!
>
If my understanding don't meet your ideas, Please correct me.

Thanks for your help very much.

Thanks
Yan
> On 02/15/2012 05:19 PM, Xiaofeng Yan wrote:
>

Re: [oe] Source Archiver Class

[Saul Wold]

On 02/19/2012 10:40 PM, Xiaofeng Yan wrote:
> On 2012年02月18日 07:52, Saul Wold wrote:
>>
>> Xiaofeng & Community:
>>
>> We had a chat with Chris Larson and Mark Hatle here at ELC.
>>
>> We focused this issue down to a fewer number of options that meet the
>> needs of the licenses. Please review this and let us know if there are
>> any issues or concerns.
>>
>>
> Hi Saul,
>
> I comment my understanding as follow:
>> Best Practices:
>> - Archive during build, we do not support nor recommend post archiving
>> - Original tarball and patches w/ordering file (maybe comment the
>> series), grab non-patch files also.
> for example:
> file://a.patch
> file://b.config.in
> we also archive b.config.in.

Yes, correct.

>> - Unless requesting configured sources, which will just be the
>> configured source tarball
>> - Grab temp as postfunc of do_package - latest logs via links & pid
>>
> That means logs package include all of logs in temp. for example.
> You has described a following function we will realize at the previous
> email.
> 3 - Original Source code & Patches & temp (scripts & logs)
>
> source codes and patches should be in the stage do_patch[postfunc] = "
> do_get_source"
> logs should be in the stage do_package[postfunc] = " do_get_logs"
> So logs file shouldn't be archived in do_patch[postfunc] = " do_get_logs"
> The logs in the stage of do_packge are more than do_patch.
>
Correct, you want to get the logs and scripts from temp when they are 
most complete (after do_package not after do_patch).

As mentioned below, you would use the variable SOURCE_ARCHIVE_LOG in the 
do_package[postfunc] to determine if you need to archive the logs or not.


>> For sstate Builds (LATER):
>> - need to verify that fetch/patch/configure will get re-run for archiving
>> - Add temp dir to sstate capture (without links)
>>
>> 3 Implementations classes
>> - Filter in oe/lib/license.py
>> - source_archive.bb
>> - archives tarballs to ${BP}/...
>> - reuse copyleft_compliance for patch handling
>> - srpm
>> - take output from above and create SRPM - (LATER)
>>
> We will do the archive work according the license . I will implement two
> functions, one is for left, the other for right.
I am not sure what you mean by one is for left and the other is for right?

Can you explain your thoughts here.


>> 2 Configuration classes
>> - uses prefunc/postfunc at correct place
>> - Original Tarball / Patches
>> - Post Configuration
>>
> We can define 8 classes to complete 8 kinds of archiving methods. every
> class will inherit archiver.bbclass.

I am not sure we need 8 classes anymore, that was the point of this 
email to simplify the needs.

2 Classes:

source_patches.bbclass
  - Original Tarball & Patches & Other files
  - do_patch[postfunc] = <archive ${S}/patches with Original tarball>
  - Included temp dir if enabled via SOURCE_ARCHIVE_LOG

configured_source.bbclass
  - Get the ${S} dir (or $S & any build dir) after do_configure
  - do_configure[postfunc] = <archive configured source>
  - Include temp dir if enabled via SOURCE_ARCHIVE_LOG


> and then realize do_stage[prefunc/postfunc] in this class.
> stage include unpack, patch, configure, build package and so on.
> for example. we want only to get original tarball, then we can define a
> bbclass named "source.bbclass"
> source.bbclass:
> inherit archiver.bbclass
> do_unpack[postfunc] = " do_get_source"
>
> if we want to get both original and patches tarball, then we can define
> bbclass names "source_patches.bbclass"
> source_patches.bbclass
> inherit archiver.bbclass
> do_unpack[postfunc] = " do_get_source_patches"
>
>> SOURCE_ARCHIVE_PACKAGE_TYPE = {tar, srpm}
>> SOURCE_ARCHIVE_LOG = {True, False}
>>
>> Please let us know if you have any questions.
>>
>> Sau!
>>
> If my understanding don't meet your ideas, Please correct me.
>
I should be available via IRC in your afternoon, I will be away during 
your morning.

Sau!

> Thanks for your help very much.
>
> Thanks
> Yan
>> On 02/15/2012 05:19 PM, Xiaofeng Yan wrote:
>>
>

Re: [oe] Source Archiver Class

[Xiaofeng Yan]

On 2012年02月22日 00:57, Saul Wold wrote:
> On 02/19/2012 10:40 PM, Xiaofeng Yan wrote:
>> On 2012年02月18日 07:52, Saul Wold wrote:
>>>

Hi Saul,

I have implemented the most functions but srpm functions. I will 
complete the left work(merge functions from sourcepkg, src_distribute  
to archiver.bbclass, implement srpm functions. )

>>> Xiaofeng & Community:
>>>
>>> We had a chat with Chris Larson and Mark Hatle here at ELC.
>>>
>>> We focused this issue down to a fewer number of options that meet the
>>> needs of the licenses. Please review this and let us know if there are
>>> any issues or concerns.
>>>
>>>
>> Hi Saul,
>>
>> I comment my understanding as follow:
>>> Best Practices:
>>> - Archive during build, we do not support nor recommend post archiving
>>> - Original tarball and patches w/ordering file (maybe comment the
>>> series), grab non-patch files also.
>> for example:
>> file://a.patch
>> file://b.config.in
>> we also archive b.config.in.
>
> Yes, correct.
>
>>> - Unless requesting configured sources, which will just be the
>>> configured source tarball
>>> - Grab temp as postfunc of do_package - latest logs via links & pid
>>>
>> That means logs package include all of logs in temp. for example.
>> You has described a following function we will realize at the previous
>> email.
>> 3 - Original Source code & Patches & temp (scripts & logs)
>>
>> source codes and patches should be in the stage do_patch[postfunc] = "
>> do_get_source"
>> logs should be in the stage do_package[postfunc] = " do_get_logs"
>> So logs file shouldn't be archived in do_patch[postfunc] = " 
>> do_get_logs"
>> The logs in the stage of do_packge are more than do_patch.
>>
> Correct, you want to get the logs and scripts from temp when they are 
> most complete (after do_package not after do_patch).
>
> As mentioned below, you would use the variable SOURCE_ARCHIVE_LOG in 
> the do_package[postfunc] to determine if you need to archive the logs 
> or not.
>
>
>>> For sstate Builds (LATER):
>>> - need to verify that fetch/patch/configure will get re-run for 
>>> archiving
>>> - Add temp dir to sstate capture (without links)
>>>
>>> 3 Implementations classes
>>> - Filter in oe/lib/license.py
>>> - source_archive.bb
>>> - archives tarballs to ${BP}/...
>>> - reuse copyleft_compliance for patch handling
>>> - srpm
>>> - take output from above and create SRPM - (LATER)
>>>
>> We will do the archive work according the license . I will implement two
>> functions, one is for left, the other for right.
> I am not sure what you mean by one is for left and the other is for 
> right?
>
> Can you explain your thoughts here.
I mean I need implement a function to select to archive the packages 
from copyright(no free)or copyleft(free).
>
>
>>> 2 Configuration classes
>>> - uses prefunc/postfunc at correct place
>>> - Original Tarball / Patches
>>> - Post Configuration
>>>
>> We can define 8 classes to complete 8 kinds of archiving methods. every
>> class will inherit archiver.bbclass.
>
> I am not sure we need 8 classes anymore, that was the point of this 
> email to simplify the needs.
>
> 2 Classes:
>
> source_patches.bbclass
>  - Original Tarball & Patches & Other files
>  - do_patch[postfunc] = <archive ${S}/patches with Original tarball>
>  - Included temp dir if enabled via SOURCE_ARCHIVE_LOG
>
> configured_source.bbclass
>  - Get the ${S} dir (or $S & any build dir) after do_configure
>  - do_configure[postfunc] = <archive configured source>
>  - Include temp dir if enabled via SOURCE_ARCHIVE_LOG
>
>
got it, thanks
>> and then realize do_stage[prefunc/postfunc] in this class.
>> stage include unpack, patch, configure, build package and so on.
>> for example. we want only to get original tarball, then we can define a
>> bbclass named "source.bbclass"
>> source.bbclass:
>> inherit archiver.bbclass
>> do_unpack[postfunc] = " do_get_source"
>>
>> if we want to get both original and patches tarball, then we can define
>> bbclass names "source_patches.bbclass"
>> source_patches.bbclass
>> inherit archiver.bbclass
>> do_unpack[postfunc] = " do_get_source_patches"
>>
>>> SOURCE_ARCHIVE_PACKAGE_TYPE = {tar, srpm}
>>> SOURCE_ARCHIVE_LOG = {True, False}
>>>
>>> Please let us know if you have any questions.
>>>
>>> Sau!
>>>
>> If my understanding don't meet your ideas, Please correct me.
>>
> I should be available via IRC in your afternoon, I will be away during 
> your morning.
>
> Sau!
>
>> Thanks for your help very much.
>>
>> Thanks
>> Yan
>>> On 02/15/2012 05:19 PM, Xiaofeng Yan wrote:
>>>
>>
>

Re: [oe] Source Archiver Class

[Chris Larson]

On Wed, Feb 22, 2012 at 7:24 PM, Xiaofeng Yan
<xiaofeng.yan@windriver.com> wrote:
> On 2012年02月22日 00:57, Saul Wold wrote:
>>
>> On 02/19/2012 10:40 PM, Xiaofeng Yan wrote:
>>>
>>> On 2012年02月18日 07:52, Saul Wold wrote:
>>>>
>>>>
>
> Hi Saul,
>
> I have implemented the most functions but srpm functions. I will complete
> the left work(merge functions from sourcepkg, src_distribute  to
> archiver.bbclass, implement srpm functions. )
>
>
>>>> Xiaofeng & Community:
>>>>
>>>> We had a chat with Chris Larson and Mark Hatle here at ELC.
>>>>
>>>> We focused this issue down to a fewer number of options that meet the
>>>> needs of the licenses. Please review this and let us know if there are
>>>> any issues or concerns.
>>>>
>>>>
>>> Hi Saul,
>>>
>>> I comment my understanding as follow:
>>>>
>>>> Best Practices:
>>>> - Archive during build, we do not support nor recommend post archiving
>>>> - Original tarball and patches w/ordering file (maybe comment the
>>>> series), grab non-patch files also.
>>>
>>> for example:
>>> file://a.patch
>>> file://b.config.in
>>> we also archive b.config.in.
>>
>>
>> Yes, correct.
>>
>>>> - Unless requesting configured sources, which will just be the
>>>> configured source tarball
>>>> - Grab temp as postfunc of do_package - latest logs via links & pid
>>>>
>>> That means logs package include all of logs in temp. for example.
>>> You has described a following function we will realize at the previous
>>> email.
>>> 3 - Original Source code & Patches & temp (scripts & logs)
>>>
>>> source codes and patches should be in the stage do_patch[postfunc] = "
>>> do_get_source"
>>> logs should be in the stage do_package[postfunc] = " do_get_logs"
>>> So logs file shouldn't be archived in do_patch[postfunc] = " do_get_logs"
>>> The logs in the stage of do_packge are more than do_patch.
>>>
>> Correct, you want to get the logs and scripts from temp when they are most
>> complete (after do_package not after do_patch).
>>
>> As mentioned below, you would use the variable SOURCE_ARCHIVE_LOG in the
>> do_package[postfunc] to determine if you need to archive the logs or not.
>>
>>
>>>> For sstate Builds (LATER):
>>>> - need to verify that fetch/patch/configure will get re-run for
>>>> archiving
>>>> - Add temp dir to sstate capture (without links)
>>>>
>>>> 3 Implementations classes
>>>> - Filter in oe/lib/license.py
>>>> - source_archive.bb
>>>> - archives tarballs to ${BP}/...
>>>> - reuse copyleft_compliance for patch handling
>>>> - srpm
>>>> - take output from above and create SRPM - (LATER)
>>>>
>>> We will do the archive work according the license . I will implement two
>>> functions, one is for left, the other for right.
>>
>> I am not sure what you mean by one is for left and the other is for right?
>>
>> Can you explain your thoughts here.
>
> I mean I need implement a function to select to archive the packages from
> copyright(no free)or copyleft(free).

This is the filtering to put in license.py, and which is implemented
in copyleft_compliance.bbclass. It should largely be a copy/paste from
there..
-- 
Christopher Larson