[PATCH] base bbclass: add support for tar.lz

[PATCH] base bbclass: add support for tar.lz

[Koen Kooi]

Some GNU packages (e.g. ddrescue) switched to lzip only, so add support for it.

Needs matching bitbake patch to work properly:
http://lists.openembedded.org/pipermail/bitbake-devel/2014-October/005093.html

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
---
 meta/classes/base.bbclass | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index ff8c633..128297a 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -478,6 +478,11 @@ python () {
     elif "osc://" in srcuri:
         d.appendVarFlag('do_fetch', 'depends', ' osc-native:do_populate_sysroot')
 
+    # *.lz should depends on lzip-native for unpacking
+    # Not endswith because of "*.patch.lz;patch=1". Need bb.fetch.decodeurl in future
+    if '.lz' in srcuri:
+        d.appendVarFlag('do_unpack', 'depends', ' lzip-native:do_populate_sysroot')
+
     # *.lz4 should depends on lz4-native for unpacking
     # Not endswith because of "*.patch.lz4;patch=1". Need bb.fetch.decodeurl in future
     if '.lz4' in srcuri:
-- 
1.9.0

Re: [PATCH] base bbclass: add support for tar.lz

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1068 bytes --]

On 26 October 2014 15:10, Koen Kooi <koen@dominion.thruhere.net> wrote:

> +    # Not endswith because of "*.patch.lz;patch=1". Need
> bb.fetch.decodeurl in future
> +    if '.lz' in srcuri:
>

I think it's time to implement that todo, as oe-core now fails to build:

ERROR: Nothing PROVIDES 'lzip-native'. Close matches:
  zip-native
  lzop-native
  gzip-native
ERROR: Nothing RPROVIDES 'lzop-dev' (but
/home/ross/Yocto/poky/meta/recipes-support/lzop/lzop_1.03.bb RDEPENDS on or
otherwise requires it)
ERROR: No eligible RPROVIDERs exist for 'lzop-dev'
NOTE: Runtime target 'lzop-dev' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['lzop-dev']
ERROR: Nothing RPROVIDES 'lzop' (but
/home/ross/Yocto/poky/meta/recipes-support/lzop/lzop_1.03.bb RDEPENDS on or
otherwise requires it)
ERROR: No eligible RPROVIDERs exist for 'lzop'
NOTE: Runtime target 'lzop' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['lzop']

Because:

SRC_URI = "http://www.lzop.org/download/${BP}.tar.gz \

Ross

[-- Attachment #2: Type: text/html, Size: 1916 bytes --]

Re: [PATCH] base bbclass: add support for tar.lz

[Koen Kooi]

> Op 27 okt. 2014, om 11:59 heeft Burton, Ross <ross.burton@intel.com> het volgende geschreven:
> 
> 
> On 26 October 2014 15:10, Koen Kooi <koen@dominion.thruhere.net> wrote:
> +    # Not endswith because of "*.patch.lz;patch=1". Need bb.fetch.decodeurl in future
> +    if '.lz' in srcuri:
> 
> I think it's time to implement that todo, as oe-core now fails to build:
> 
> ERROR: Nothing PROVIDES 'lzip-native'. Close matches:
>   zip-native
>   lzop-native
>   gzip-native
> ERROR: Nothing RPROVIDES 'lzop-dev' (but /home/ross/Yocto/poky/meta/recipes-support/lzop/lzop_1.03.bb RDEPENDS on or otherwise requires it)
> ERROR: No eligible RPROVIDERs exist for 'lzop-dev'
> NOTE: Runtime target 'lzop-dev' is unbuildable, removing...
> Missing or unbuildable dependency chain was: ['lzop-dev']
> ERROR: Nothing RPROVIDES 'lzop' (but /home/ross/Yocto/poky/meta/recipes-support/lzop/lzop_1.03.bb RDEPENDS on or otherwise requires it)
> ERROR: No eligible RPROVIDERs exist for 'lzop'
> NOTE: Runtime target 'lzop' is unbuildable, removing...
> Missing or unbuildable dependency chain was: ['lzop'] 
> 
> Because:
> 
> SRC_URI = "http://www.lzop.org/download/${BP}.tar.gz \

Not quite, 'lzop' is not the same as 'lzip',the problem is this:

koen@beast:/build/v2014.12/sources$ find . -name "lzip*.bb"
./meta-openembedded/meta-oe/recipes-support/lzip/lzip_1.15.bb

lzip is in meta-oe, not in oe-core. Should I send a patch moving lzip to OE-core?

regards,

Koen

Re: [PATCH] base bbclass: add support for tar.lz

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

On 27 October 2014 11:39, Koen Kooi <koen@dominion.thruhere.net> wrote:

> > SRC_URI = "http://www.lzop.org/download/${BP}.tar.gz \
>
> Not quite, 'lzop' is not the same as 'lzip',the problem is this:
>

The problem is that the lzop SRC_URI contains the string ".lz", so it
incorrectly gains a lzip-native dependency.

Ross

[-- Attachment #2: Type: text/html, Size: 832 bytes --]

Re: [PATCH] base bbclass: add support for tar.lz

[Koen Kooi]

> Op 27 okt. 2014, om 12:54 heeft Burton, Ross <ross.burton@intel.com> het volgende geschreven:
> 
> 
> On 27 October 2014 11:39, Koen Kooi <koen@dominion.thruhere.net> wrote:
> > SRC_URI = "http://www.lzop.org/download/${BP}.tar.gz \
> 
> Not quite, 'lzop' is not the same as 'lzip',the problem is this:
> 
> The problem is that the lzop SRC_URI contains the string ".lz", so it incorrectly gains a lzip-native dependency.


Ah, two different problems :/

Re: [PATCH] base bbclass: add support for tar.lz

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 268 bytes --]

On 27 October 2014 13:06, Koen Kooi <koen@dominion.thruhere.net> wrote:

> Ah, two different problems :/


The problem that lzip-native isn't in oe-core is only a problem when
there's a .lz SRC_URI in oe-core, but the false-positives are a problem now.

Ross

[-- Attachment #2: Type: text/html, Size: 628 bytes --]

Re: [PATCH] base bbclass: add support for tar.lz

[Koen Kooi]

> Op 27 okt. 2014, om 14:15 heeft Burton, Ross <ross.burton@intel.com> het volgende geschreven:
> 
> 
> On 27 October 2014 13:06, Koen Kooi <koen@dominion.thruhere.net> wrote:
> Ah, two different problems :/
> 
> The problem that lzip-native isn't in oe-core is only a problem when there's a .lz SRC_URI in oe-core, but the false-positives are a problem now.

Apart from this dependency problem, bitbake has a problem as well. Does anyone know why bitbake does:

	compressiontool --stdout file.tar.compressed | tar x

instead of:

	tar xf file.tar.compressed 

Tar seems to do the right thing with 'tar xf' after installing lzip on my angstrom system, I don't have and .tar.lz lzops lying around to test with, though.

regards,

Koen

Re: [PATCH] base bbclass: add support for tar.lz

[Richard Purdie]

On Tue, 2014-10-28 at 10:36 +0100, Koen Kooi wrote:
> > Op 27 okt. 2014, om 14:15 heeft Burton, Ross <ross.burton@intel.com> het volgende geschreven:
> > 
> > 
> > On 27 October 2014 13:06, Koen Kooi <koen@dominion.thruhere.net> wrote:
> > Ah, two different problems :/
> > 
> > The problem that lzip-native isn't in oe-core is only a problem when there's a .lz SRC_URI in oe-core, but the false-positives are a problem now.
> 
> Apart from this dependency problem, bitbake has a problem as well. Does anyone know why bitbake does:
> 
> 	compressiontool --stdout file.tar.compressed | tar x
> 
> instead of:
> 
> 	tar xf file.tar.compressed 

The unpack code is rather old, its age being around a decade. A decade
ago, tar without inbuilt decompression support was not uncommon. The
code has remained unchanged...

zlib was added, I know that most tar has bzip2 now. I'm not 100% sure
which formats we could rely on being in tar though. It would also be
interesting to know which performs better on a multicore system.

Cheers,

Richard

Re: [PATCH] base bbclass: add support for tar.lz

[Koen Kooi]

> Op 28 okt. 2014, om 23:22 heeft Richard Purdie <richard.purdie@linuxfoundation.org> het volgende geschreven:
> 
> On Tue, 2014-10-28 at 10:36 +0100, Koen Kooi wrote:
>>> Op 27 okt. 2014, om 14:15 heeft Burton, Ross <ross.burton@intel.com> het volgende geschreven:
>>> 
>>> 
>>> On 27 October 2014 13:06, Koen Kooi <koen@dominion.thruhere.net> wrote:
>>> Ah, two different problems :/
>>> 
>>> The problem that lzip-native isn't in oe-core is only a problem when there's a .lz SRC_URI in oe-core, but the false-positives are a problem now.
>> 
>> Apart from this dependency problem, bitbake has a problem as well. Does anyone know why bitbake does:
>> 
>> 	compressiontool --stdout file.tar.compressed | tar x
>> 
>> instead of:
>> 
>> 	tar xf file.tar.compressed 
> 
> The unpack code is rather old, its age being around a decade. A decade
> ago, tar without inbuilt decompression support was not uncommon. The
> code has remained unchanged...
> 
> zlib was added, I know that most tar has bzip2 now. I'm not 100% sure
> which formats we could rely on being in tar though.

What hapenneded over here was:

tar xf foo.tar.lz
<error>
install lzip
tar xf foo.tar.lz
<success>

So I'm not sure if tar has the knowledge already or not, but installing lzip made things work without rebuilding tar.

> It would also be
> interesting to know which performs better on a multicore system.

Indeed.

regards,

Koen