[PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[Shakeel, Muhammad]

From: Muhammad Shakeel <muhammad_shakeel@mentor.com>

* Add archiver tasks correctly for any 'IMAGE_PKGTYPE'.

[YOCTO #3449]

Signed-off-by: Muhammad Shakeel <muhammad_shakeel@mentor.com>
Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
 meta/classes/archive-configured-source.bbclass |   15 +++++++++++++--
 meta/classes/archive-original-source.bbclass   |   15 +++++++++++++--
 meta/classes/archive-patched-source.bbclass    |   15 +++++++++++++--
 3 files changed, 39 insertions(+), 6 deletions(-)

diff --git a/meta/classes/archive-configured-source.bbclass b/meta/classes/archive-configured-source.bbclass
index 64799e3..50838e6 100644
--- a/meta/classes/archive-configured-source.bbclass
+++ b/meta/classes/archive-configured-source.bbclass
@@ -11,12 +11,19 @@ inherit archiver
 addtask do_archive_configured_sources after do_configure
 
 # Get archiving package with temp(logs) and scripts(.bb and inc files)
-addtask do_archive_scripts_logs after do_package_write_rpm
+addtask do_archive_scripts_logs
 
 # Get dump date and create diff file 
-addtask do_dumpdata_create_diff_gz after do_package_write_rpm before do_build
+addtask do_dumpdata_create_diff_gz before do_build
 
 python () {
+    packaging = d.getVar('IMAGE_PKGTYPE', True)
+    write_package_task = 'do_package_write_' + packaging
+
+    deps = d.getVarFlag('do_dumpdata_create_diff_gz', 'deps') or []
+    deps.append(write_package_task)
+    d.setVarFlag('do_dumpdata_create_diff_gz', 'deps', deps)
+
     if d.getVar('SOURCE_ARCHIVE_PACKAGE_TYPE', True) != 'srpm':
         """
         If package type is not 'srpm' then add tasks to move archive packages of
@@ -30,6 +37,10 @@ python () {
         recrdeptask += ' do_archive_configured_sources '
         d.setVarFlag('do_build', 'recrdeptask', recrdeptask)
 
+        deps = d.getVarFlag('do_archive_scripts_logs', 'deps') or []
+        deps.append(write_package_task)
+        d.setVarFlag('do_archive_scripts_logs', 'deps', deps)
+
         deps = d.getVarFlag('do_build', 'deps') or []
         deps.append('do_archive_scripts_logs')
         d.setVarFlag('do_build', 'deps', deps)
diff --git a/meta/classes/archive-original-source.bbclass b/meta/classes/archive-original-source.bbclass
index 44dffb4..77f9650 100644
--- a/meta/classes/archive-original-source.bbclass
+++ b/meta/classes/archive-original-source.bbclass
@@ -11,12 +11,19 @@ inherit archiver
 addtask do_archive_original_sources_patches after do_unpack
 
 # Get archiving package with temp(logs) and scripts(.bb and inc files)
-addtask do_archive_scripts_logs after do_package_write_rpm
+addtask do_archive_scripts_logs
 
 # Get dump date and create diff file 
-addtask do_dumpdata_create_diff_gz after do_package_write_rpm before do_build
+addtask do_dumpdata_create_diff_gz before do_build
 
 python () {
+    packaging = d.getVar('IMAGE_PKGTYPE', True)
+    write_package_task = 'do_package_write_' + packaging
+
+    deps = d.getVarFlag('do_dumpdata_create_diff_gz', 'deps') or []
+    deps.append(write_package_task)
+    d.setVarFlag('do_dumpdata_create_diff_gz', 'deps', deps)
+
     if d.getVar('SOURCE_ARCHIVE_PACKAGE_TYPE', True) != 'srpm':
         """
         If package type is not 'srpm' then add tasks to move archive packages of
@@ -30,6 +37,10 @@ python () {
         recrdeptask += ' do_archive_original_sources_patches '
         d.setVarFlag('do_build', 'recrdeptask', recrdeptask)
 
+        deps = d.getVarFlag('do_archive_scripts_logs', 'deps') or []
+        deps.append(write_package_task)
+        d.setVarFlag('do_archive_scripts_logs', 'deps', deps)
+
         deps = d.getVarFlag('do_build', 'deps') or []
         deps.append('do_archive_scripts_logs')
         d.setVarFlag('do_build', 'deps', deps)
diff --git a/meta/classes/archive-patched-source.bbclass b/meta/classes/archive-patched-source.bbclass
index 35bae54..11dd547 100644
--- a/meta/classes/archive-patched-source.bbclass
+++ b/meta/classes/archive-patched-source.bbclass
@@ -11,12 +11,19 @@ inherit archiver
 addtask do_archive_patched_sources after do_patch
 
 # Get archiving package with logs(temp) and scripts(.bb and .inc files)
-addtask do_archive_scripts_logs after do_package_write_rpm
+addtask do_archive_scripts_logs
 
 # Get dump date and create diff file 
-addtask do_dumpdata_create_diff_gz after do_package_write_rpm before do_build
+addtask do_dumpdata_create_diff_gz before do_build
 
 python () {
+    packaging = d.getVar('IMAGE_PKGTYPE', True)
+    write_package_task = 'do_package_write_' + packaging
+
+    deps = d.getVarFlag('do_dumpdata_create_diff_gz', 'deps') or []
+    deps.append(write_package_task)
+    d.setVarFlag('do_dumpdata_create_diff_gz', 'deps', deps)
+
     if d.getVar('SOURCE_ARCHIVE_PACKAGE_TYPE', True) != 'srpm':
         """
         If package type is not 'srpm' then add tasks to move archive packages of
@@ -30,6 +37,10 @@ python () {
         recrdeptask += ' do_archive_patched_sources '
         d.setVarFlag('do_build', 'recrdeptask', recrdeptask)
 
+        deps = d.getVarFlag('do_archive_scripts_logs', 'deps') or []
+        deps.append(write_package_task)
+        d.setVarFlag('do_archive_scripts_logs', 'deps', deps)
+
         deps = d.getVarFlag('do_build', 'deps') or []
         deps.append('do_archive_scripts_logs')
         d.setVarFlag('do_build', 'deps', deps)
-- 
1.7.9.5

Re: [PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[Otavio Salvador]

On Mon, Jan 14, 2013 at 11:48 AM, Shakeel, Muhammad
<muhammad_shakeel@mentor.com> wrote:
> From: Muhammad Shakeel <muhammad_shakeel@mentor.com>
>
> * Add archiver tasks correctly for any 'IMAGE_PKGTYPE'.
>
> [YOCTO #3449]
>
> Signed-off-by: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> Signed-off-by: Christopher Larson <chris_larson@mentor.com>

I'd prefer if this patch would be squashed in the first one. It does
looks like the same change for me as you're adding those hooks here as
well.

Richard, can you comment here?

--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[Richard Purdie]

On Mon, 2013-01-14 at 18:48 +0500, Shakeel, Muhammad wrote:
> From: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> 
> * Add archiver tasks correctly for any 'IMAGE_PKGTYPE'.
> 
> [YOCTO #3449]
> 
> Signed-off-by: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> Signed-off-by: Christopher Larson <chris_larson@mentor.com>
> ---
>  meta/classes/archive-configured-source.bbclass |   15 +++++++++++++--
>  meta/classes/archive-original-source.bbclass   |   15 +++++++++++++--
>  meta/classes/archive-patched-source.bbclass    |   15 +++++++++++++--
>  3 files changed, 39 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/classes/archive-configured-source.bbclass b/meta/classes/archive-configured-source.bbclass
> index 64799e3..50838e6 100644
> --- a/meta/classes/archive-configured-source.bbclass
> +++ b/meta/classes/archive-configured-source.bbclass
> @@ -11,12 +11,19 @@ inherit archiver
>  addtask do_archive_configured_sources after do_configure
>  
>  # Get archiving package with temp(logs) and scripts(.bb and inc files)
> -addtask do_archive_scripts_logs after do_package_write_rpm
> +addtask do_archive_scripts_logs
>  
>  # Get dump date and create diff file 
> -addtask do_dumpdata_create_diff_gz after do_package_write_rpm before do_build
> +addtask do_dumpdata_create_diff_gz before do_build
>  
>  python () {
> +    packaging = d.getVar('IMAGE_PKGTYPE', True)
> +    write_package_task = 'do_package_write_' + packaging
> +
> +    deps = d.getVarFlag('do_dumpdata_create_diff_gz', 'deps') or []
> +    deps.append(write_package_task)
> +    d.setVarFlag('do_dumpdata_create_diff_gz', 'deps', deps)

I'd not previously noticed bitbake was exposing this "deps" flag as a
list. I'd previously assumed we interacted with strings through the
deptask flag so we'd use something like:

d.appendVarFlag('do_dumpdata_create_diff_gz', 'deptask', ' do_package_write_' + packaging)

We should probably make a decision whether deps is meant to be exposed
or whether we should be using deptask. I appreciate this patch isn't at
fault, its the archiver code in general doing this. I agree the API is
less optimal using strings but it is also more consistent with every
other set of flags...

Cheers,

Richard

Re: [PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[Richard Purdie]

On Mon, 2013-01-14 at 15:33 -0200, Otavio Salvador wrote:
> On Mon, Jan 14, 2013 at 11:48 AM, Shakeel, Muhammad
> <muhammad_shakeel@mentor.com> wrote:
> > From: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> >
> > * Add archiver tasks correctly for any 'IMAGE_PKGTYPE'.
> >
> > [YOCTO #3449]
> >
> > Signed-off-by: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> > Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> > Signed-off-by: Christopher Larson <chris_larson@mentor.com>
> 
> I'd prefer if this patch would be squashed in the first one. It does
> looks like the same change for me as you're adding those hooks here as
> well.
> 
> Richard, can you comment here?

They do like different changes to me...

Cheers,

Richard

Re: [PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[Richard Purdie]

On Mon, 2013-01-14 at 18:48 +0500, Shakeel, Muhammad wrote:
> From: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> 
> * Add archiver tasks correctly for any 'IMAGE_PKGTYPE'.
> 
> [YOCTO #3449]
> 
> Signed-off-by: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> Signed-off-by: Christopher Larson <chris_larson@mentor.com>
> ---
>  meta/classes/archive-configured-source.bbclass |   15 +++++++++++++--
>  meta/classes/archive-original-source.bbclass   |   15 +++++++++++++--
>  meta/classes/archive-patched-source.bbclass    |   15 +++++++++++++--
>  3 files changed, 39 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/classes/archive-configured-source.bbclass b/meta/classes/archive-configured-source.bbclass
> index 64799e3..50838e6 100644
> --- a/meta/classes/archive-configured-source.bbclass
> +++ b/meta/classes/archive-configured-source.bbclass
> @@ -11,12 +11,19 @@ inherit archiver
>  addtask do_archive_configured_sources after do_configure
>  
>  # Get archiving package with temp(logs) and scripts(.bb and inc files)
> -addtask do_archive_scripts_logs after do_package_write_rpm
> +addtask do_archive_scripts_logs
>  
>  # Get dump date and create diff file 
> -addtask do_dumpdata_create_diff_gz after do_package_write_rpm before do_build
> +addtask do_dumpdata_create_diff_gz before do_build
>  
>  python () {
> +    packaging = d.getVar('IMAGE_PKGTYPE', True)
> +    write_package_task = 'do_package_write_' + packaging
> +
> +    deps = d.getVarFlag('do_dumpdata_create_diff_gz', 'deps') or []
> +    deps.append(write_package_task)
> +    d.setVarFlag('do_dumpdata_create_diff_gz', 'deps', deps)
> +
>      if d.getVar('SOURCE_ARCHIVE_PACKAGE_TYPE', True) != 'srpm':
>          """
>          If package type is not 'srpm' then add tasks to move archive packages of
> @@ -30,6 +37,10 @@ python () {
>          recrdeptask += ' do_archive_configured_sources '
>          d.setVarFlag('do_build', 'recrdeptask', recrdeptask)
>  
> +        deps = d.getVarFlag('do_archive_scripts_logs', 'deps') or []
> +        deps.append(write_package_task)
> +        d.setVarFlag('do_archive_scripts_logs', 'deps', deps)

Looking at the patch series, this is code introducing users of the
'deps' flag outside of externalsrc.bbclass (which has some nasty reasons
it currently pokes bitbake internals).

I'd like to not introduce usage of the flag so can you please change
these references to things like:

d.appendVarFlag('do_archive_scripts_logs', 'deps', ' do_package_write_' + packaging)

Use of appendVarFlag applies anywhere you currently do a set, append,
set operation, its just neater code and gives us some scope to optmise
the implementation in bitbake at a later date.

This applies to both this patch and the other one.

Thanks,

Richard

Re: [PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

[Shakeel, Muhammad]

Thanks for your valuable feedback. I will rework the patch as per your suggestion and will send a v3 after testing.

Regards,
--Shakeel
________________________________________
From: Richard Purdie [richard.purdie@linuxfoundation.org]
Sent: Wednesday, January 16, 2013 4:48 PM
To: Shakeel, Muhammad
Cc: openembedded-core@lists.openembedded.org; Larson, Chris
Subject: Re: [OE-core] [PATCH 2/2 v2] archive-*-source.bbclass: Handle all package classes

On Mon, 2013-01-14 at 18:48 +0500, Shakeel, Muhammad wrote:
> From: Muhammad Shakeel <muhammad_shakeel@mentor.com>
>
> * Add archiver tasks correctly for any 'IMAGE_PKGTYPE'.
>
> [YOCTO #3449]
>
> Signed-off-by: Muhammad Shakeel <muhammad_shakeel@mentor.com>
> Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> Signed-off-by: Christopher Larson <chris_larson@mentor.com>
> ---
>  meta/classes/archive-configured-source.bbclass |   15 +++++++++++++--
>  meta/classes/archive-original-source.bbclass   |   15 +++++++++++++--
>  meta/classes/archive-patched-source.bbclass    |   15 +++++++++++++--
>  3 files changed, 39 insertions(+), 6 deletions(-)
>
> diff --git a/meta/classes/archive-configured-source.bbclass b/meta/classes/archive-configured-source.bbclass
> index 64799e3..50838e6 100644
> --- a/meta/classes/archive-configured-source.bbclass
> +++ b/meta/classes/archive-configured-source.bbclass
> @@ -11,12 +11,19 @@ inherit archiver
>  addtask do_archive_configured_sources after do_configure
>
>  # Get archiving package with temp(logs) and scripts(.bb and inc files)
> -addtask do_archive_scripts_logs after do_package_write_rpm
> +addtask do_archive_scripts_logs
>
>  # Get dump date and create diff file
> -addtask do_dumpdata_create_diff_gz after do_package_write_rpm before do_build
> +addtask do_dumpdata_create_diff_gz before do_build
>
>  python () {
> +    packaging = d.getVar('IMAGE_PKGTYPE', True)
> +    write_package_task = 'do_package_write_' + packaging
> +
> +    deps = d.getVarFlag('do_dumpdata_create_diff_gz', 'deps') or []
> +    deps.append(write_package_task)
> +    d.setVarFlag('do_dumpdata_create_diff_gz', 'deps', deps)
> +
>      if d.getVar('SOURCE_ARCHIVE_PACKAGE_TYPE', True) != 'srpm':
>          """
>          If package type is not 'srpm' then add tasks to move archive packages of
> @@ -30,6 +37,10 @@ python () {
>          recrdeptask += ' do_archive_configured_sources '
>          d.setVarFlag('do_build', 'recrdeptask', recrdeptask)
>
> +        deps = d.getVarFlag('do_archive_scripts_logs', 'deps') or []
> +        deps.append(write_package_task)
> +        d.setVarFlag('do_archive_scripts_logs', 'deps', deps)

Looking at the patch series, this is code introducing users of the
'deps' flag outside of externalsrc.bbclass (which has some nasty reasons
it currently pokes bitbake internals).

I'd like to not introduce usage of the flag so can you please change
these references to things like:

d.appendVarFlag('do_archive_scripts_logs', 'deps', ' do_package_write_' + packaging)

Use of appendVarFlag applies anywhere you currently do a set, append,
set operation, its just neater code and gives us some scope to optmise
the implementation in bitbake at a later date.

This applies to both this patch and the other one.

Thanks,

Richard