icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Paul Menzel]

[-- Attachment #1: Type: text/plain, Size: 2105 bytes --]

Dear OE folks,


even with the latest patches send to the list [1].

        Importing binaries from component CORBA_DIST
        ( cd  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
        Archive:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
          End-of-central-directory signature not found.  Either this file is not
          a zipfile, or it constitutes one disk of a multi-part archive.  In the
          latter case the central directory and zipfile comment will be found on
          the last disk(s) of this archive.
        note:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
        unzip:  cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period.
        make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Running the command directly works fine though.

        $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
        Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
          inflating: lib/orb.idl             
          inflating: lib/ir.idl


Thanks,

Paul


[1] http://patches.openembedded.org/patch/4621/

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Robert Schuster]

[-- Attachment #1: Type: text/plain, Size: 2478 bytes --]

Hi,
I'll look into this.

Regards,
Robert

Am 22.05.2011 23:24, schrieb Paul Menzel:
> Dear OE folks,
> 
> 
> even with the latest patches send to the list [1].
> 
>         Importing binaries from component CORBA_DIST
>         ( cd  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
>         Archive:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
>           End-of-central-directory signature not found.  Either this file is not
>           a zipfile, or it constitutes one disk of a multi-part archive.  In the
>           latter case the central directory and zipfile comment will be found on
>           the last disk(s) of this archive.
>         note:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
>         unzip:  cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period.
>         make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9
> 
> Running the command directly works fine though.
> 
>         $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
>         Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
>           inflating: lib/orb.idl             
>           inflating: lib/ir.idl
> 
> 
> Thanks,
> 
> Paul
> 
> 
> [1] http://patches.openembedded.org/patch/4621/
> 
> 
> 
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 270 bytes --]

Re: icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Robert Schuster]

[-- Attachment #1: Type: text/plain, Size: 1663 bytes --]

Hi,

>         Importing binaries from component CORBA_DIST
>         ( cd  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
>         Archive:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
>           End-of-central-directory signature not found.  Either this file is not
>           a zipfile, or it constitutes one disk of a multi-part archive.  In the
>           latter case the central directory and zipfile comment will be found on
>           the last disk(s) of this archive.
>         note:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
>         unzip:  cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period.
>         make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9
Did you notice that somehow the makefile tries to unpack the file
'/oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip'. I
smell some shell-quoting problem or similar.

Regards,
Robert


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 270 bytes --]

Re: icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Paul Menzel]

[-- Attachment #1: Type: text/plain, Size: 3763 bytes --]

Dear Robert,


On Mi, 2011-05-25 at 11:35 +0200, Robert Schuster wrote:

> >         Importing binaries from component CORBA_DIST
> >         ( cd  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
> >         Archive:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
> >           End-of-central-directory signature not found.  Either this file is not
> >           a zipfile, or it constitutes one disk of a multi-part archive.  In the
> >           latter case the central directory and zipfile comment will be found on
> >           the last disk(s) of this archive.
> >         note:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
> >         unzip:  cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period.
> >         make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9
> Did you notice that somehow the makefile tries to unpack the file
> '/oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip'.

It looks strange. But I thought that is some behavior of `unzip` to look
into some default paths.

> I smell some shell-quoting problem or similar.

I found the following parts but do not know how this can fail.

        /oe/build-minimal-eglibc/minimal-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0$ grep -Ri "Importing binaries" *
        icedtea6-1.7.10/openjdk/jdk/make/common/internal/ImportComponents.gmk:  $(ECHO) "Importing binaries from component $1"; \
        icedtea6-1.7.10/openjdk-ecj/jdk/make/common/internal/ImportComponents.gmk:  $(ECHO) "Importing binaries from component $1"; \
        openjdk-src-dir/jdk/make/common/internal/ImportComponents.gmk:  $(ECHO) "Importing binaries from component $1"; \
        temp/log.do_compile:Importing binaries from component CORBA_DIST
        temp/log.do_compile.22501:Importing binaries from component CORBA_DIST

looking at there is

        # Importing optional component bin files (for install image)
        define import-one-binaries
        @if [ "$($1)" != "" -a -f $($1)/$(BIN_ZIP_FILE) ] ; then \
          $(ECHO) "Importing binaries from component $1"; \
          $(call Unzipper,$2,$($1)/$(BIN_ZIP_FILE)); \
        fi
        endef

and the function/method `Unzipper` is defined as follows.

        # Unzip zip file $2 into directory $1 (if $2 exists)
        #   Warning: $2 must be absolute path not relative
        define Unzipper
        ( \
          $(MKDIR) -p $1; \
          $(ECHO) "( $(CD) $1 && $(UNZIP) -o $2 )"; \
          ( $(CD) $1 && $(UNZIP) -o $2 ) \
        )
        endef

I do not see where this fails. By the way, to use the native `unzip` I have the following in my recipe (as send to the list [2]).

        EXTRA_OEMAKE = ' \
                        ZIPEXE="${STAGING_BINDIR_NATIVE}/zip" \
                        UNZIP="${STAGING_BINDIR_NATIVE}/unzip" \
                       '

Thanks,

Paul


[1] icedtea6-1.7.10/openjdk/jdk/make/common/internal/ImportComponents.gmk
[2] http://patches.openembedded.org/patch/4621/

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Robert Schuster]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

Hi Paul,

>         # Unzip zip file $2 into directory $1 (if $2 exists)
>         #   Warning: $2 must be absolute path not relative
>         define Unzipper
>         ( \
>           $(MKDIR) -p $1; \
>           $(ECHO) "( $(CD) $1 && $(UNZIP) -o $2 )"; \
>           ( $(CD) $1 && $(UNZIP) -o $2 ) \
>         )
>         endef
> 
> I do not see where this fails. By the way, to use the native `unzip` I have the following in my recipe (as send to the list [2]).
Wow, this is really weird! There is a file
openjdk-ecj/jdk/make/common/shared/Defs-utils.gmk containing all the
tool variable definitions - including UNZIP. So I removed the
EXTRA_OEMAKE part for UNZIP and added the path to our native unzip
executable directly into Defs-utils.gmk and guess what. That works!

This is really weird because the 'Unzipper' command actually echos its
arguments and there $2 is printed perfectly. But when it is accessed a
second time in the actual call it is wrong.

/me puzzled ....

Regards,
Robert


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 270 bytes --]

Re: icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Robert Schuster]

[-- Attachment #1: Type: text/plain, Size: 3501 bytes --]

Hi Paul,
greetings from the GDB cave ....

I now what is going wrong with unzip and ... whoa this is so weird ...

Am 22.05.2011 23:24, schrieb Paul Menzel:
>         Importing binaries from component CORBA_DIST
>         ( cd  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
>         Archive:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
>           End-of-central-directory signature not found.  Either this file is not
>           a zipfile, or it constitutes one disk of a multi-part archive.  In the
>           latter case the central directory and zipfile comment will be found on
>           the last disk(s) of this archive.
>         note:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
>         unzip:  cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period.
>         make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9
> 
> Running the command directly works fine though.
> 
>         $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
>         Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
>           inflating: lib/orb.idl             
>           inflating: lib/ir.idl
The thing is. You are tweaking the IcedTea build by declaring the
makefile variable UNZIP through the command-line. For programs that are
called from this make process the variables turn into environment
variables ...
And this is where the shit hits the fan: As it turns out Info-Zip's
unzip understand an arcane way of declaring the file to be unzipped.
Through an environment variable called: UNZIP (!!!)

To find this out I did nothing less than running an OE-called unzip and
a manually called unzip process run under gdb and compare what they do
in the beginning step by step.

Here a shorter explanation of what I mean:

UNZIP=xyz /home/rschus-oe/panda/tmp/sysroots/x86_64-linux/usr/bin/unzip
blablafoo
unzip:  cannot find or open xyz, xyz.zip or xyz.ZIP.

So apparently UNZIP is in our case set to the zip file executable
already to please the openjdk build.

I don't have an immediate idea how to fix it properly. I'd vote for
patching our unzip not to use that variable (It still has another one
called UNZIP_OPTS for the same use).

Last but not least: Our unzip recipe sucks. It does not enforce OE build
flags and does not prevent that the called makefile strips the binary.
There are also newer version of unzip out there (ours is from 2005).
I'll plan on fixing all of the above.

Regards,
Robert



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 270 bytes --]

Re: icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

[Paul Menzel]

[-- Attachment #1: Type: text/plain, Size: 4293 bytes --]

Dear Robert,


Am Mittwoch, den 25.05.2011, 20:00 +0200 schrieb Robert Schuster:

> greetings from the GDB cave ....
> 
> I now what is going wrong with unzip and ... whoa this is so weird ...
> 
> Am 22.05.2011 23:24, schrieb Paul Menzel:
> >         Importing binaries from component CORBA_DIST
> >         ( cd  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
> >         Archive:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
> >           End-of-central-directory signature not found.  Either this file is not
> >           a zipfile, or it constitutes one disk of a multi-part archive.  In the
> >           latter case the central directory and zipfile comment will be found on
> >           the last disk(s) of this archive.
> >         note:  /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
> >         unzip:  cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period.
> >         make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9
> > 
> > Running the command directly works fine though.
> > 
> >         $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
> >         Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
> >           inflating: lib/orb.idl             
> >           inflating: lib/ir.idl
> The thing is. You are tweaking the IcedTea build by declaring the
> makefile variable UNZIP through the command-line. For programs that are
> called from this make process the variables turn into environment
> variables ...
> And this is where the shit hits the fan: As it turns out Info-Zip's
> unzip understand an arcane way of declaring the file to be unzipped.
> Through an environment variable called: UNZIP (!!!)
> 
> To find this out I did nothing less than running an OE-called unzip and
> a manually called unzip process run under gdb and compare what they do
> in the beginning step by step.

Great analysis. I could not have done it and only used `gdb` to create a
backtrace for the developers.

> Here a shorter explanation of what I mean:
> 
> UNZIP=xyz /home/rschus-oe/panda/tmp/sysroots/x86_64-linux/usr/bin/unzip
> blablafoo
> unzip:  cannot find or open xyz, xyz.zip or xyz.ZIP.
> 
> So apparently UNZIP is in our case set to the zip file executable
> already to please the openjdk build.
> 
> I don't have an immediate idea how to fix it properly. I'd vote for
> patching our unzip not to use that variable (It still has another one
> called UNZIP_OPTS for the same use).

Well as far as I understand the manual, `UNZIP_OPTS` should be used for
VMS, but could also be used for other operating systems.

Actually I would think to solve this, by patching OpenJDK to use another
variable and report that upstream. I suggest for example UNZIP_EXE for
unzip executable.

I do not know how if the unzip developers have decided on this issue to
rename remove that variable and how many programs use this feature.

> Last but not least: Our unzip recipe sucks. It does not enforce OE build
> flags and does not prevent that the called makefile strips the binary.
> There are also newer version of unzip out there (ours is from 2005).
> I'll plan on fixing all of the above.

That would be awesome.

I am sorry for opening Pandora’s box.


Thanks,

Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 205 bytes --]