[PATCH 1/3] bitbake: add file-native to ASSUME_PROVIDED

[PATCH 1/3] bitbake: add file-native to ASSUME_PROVIDED

[Ross Burton]

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/conf/bitbake.conf | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 9ddd1c0..06971da 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -166,6 +166,7 @@ DATETIME = "${DATE}${TIME}"
 ASSUME_PROVIDED = "\
     bzip2-native \
     chrpath-native \
+    file-native \
     git-native \
     grep-native \
     diffstat-native \
-- 
2.1.4

[PATCH 2/3] sanity: check that the host has file installed

[Ross Burton]

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/classes/sanity.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index 34f8618..a1fed36 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -3,7 +3,7 @@
 #
 
 SANITY_REQUIRED_UTILITIES ?= "patch diffstat makeinfo git bzip2 tar \
-    gzip gawk chrpath wget cpio perl"
+    gzip gawk chrpath wget cpio perl filex"
 
 def bblayers_conf_file(d):
     return os.path.join(d.getVar('TOPDIR', True), 'conf/bblayers.conf')
-- 
2.1.4

[PATCH 3/3] file: don't replace host file when built natively

[Ross Burton]

To avoid races over the file binaries being replaced mid-build we made
file-native an assume-provided binary and check for it at startup, but target
file still needs a native file binary of the same version to compile the magic
data.

The least invasive way of doing this is to build a native file but use
NATIVE_PACKAGE_PATH_SUFFIX to ensure it isn't on PATH, and tell the target build
where to find the native binary.  To ensure that the host libmagic and the
libmagic provided by file-replacement-native don't cause build races, don't
install libmagic if building natively.

[ YOCTO #8144 ]

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/file/file/host-file.patch  | 32 ++++++++++++++++++++++++
 meta/recipes-devtools/file/file/no-library.patch | 28 +++++++++++++++++++++
 meta/recipes-devtools/file/file_5.24.bb          | 11 +++++++-
 3 files changed, 70 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-devtools/file/file/host-file.patch
 create mode 100644 meta/recipes-devtools/file/file/no-library.patch

diff --git a/meta/recipes-devtools/file/file/host-file.patch b/meta/recipes-devtools/file/file/host-file.patch
new file mode 100644
index 0000000..a7efbdc
--- /dev/null
+++ b/meta/recipes-devtools/file/file/host-file.patch
@@ -0,0 +1,32 @@
+Upstream-Status: Submitted (http://bugs.gw.com/view.php?id=485)
+Signed-off-by: Ross Burton <ross.burton@intel.com>
+
+From 3cde199d03b39632361c275cd30fa0612a03138b Mon Sep 17 00:00:00 2001
+From: Ross Burton <ross.burton@intel.com>
+Date: Mon, 19 Oct 2015 10:30:57 +0100
+Subject: [PATCH 2/2] When using the host file, respect FILE_COMPILE
+
+If we're cross-compiling and not using the file binary that was just built,
+execute the binary that we've been told to use (via FILE_COMPILE) when checking
+the version instead of assuming that "file" is correct as the actual compile
+uses FILE_COMPILE so different binaries may be used.
+---
+ magic/Makefile.am | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/magic/Makefile.am b/magic/Makefile.am
+index 89ac844..67067fe 100644
+--- a/magic/Makefile.am
++++ b/magic/Makefile.am
+@@ -293,7 +293,7 @@ ${MAGIC}: $(EXTRA_DIST) $(FILE_COMPILE_DEP)
+ 	@(if expr "${FILE_COMPILE}" : '.*/.*' > /dev/null; then \
+ 	    echo "Using ${FILE_COMPILE} to generate ${MAGIC}" > /dev/null; \
+ 	  else \
+-	    v=$$(file --version | sed -e s/file-// -e q); \
++	    v=$$(${FILE_COMPILE} --version | sed -e s/file-// -e q); \
+ 	    if [ "$$v" != "${PACKAGE_VERSION}" ]; then \
+ 		echo "Cannot use the installed version of file ($$v) to"; \
+ 		echo "cross-compile file ${PACKAGE_VERSION}"; \
+-- 
+2.1.4
+
diff --git a/meta/recipes-devtools/file/file/no-library.patch b/meta/recipes-devtools/file/file/no-library.patch
new file mode 100644
index 0000000..f1a26d4
--- /dev/null
+++ b/meta/recipes-devtools/file/file/no-library.patch
@@ -0,0 +1,28 @@
+We don't build a full file-native binary but rely on the host file binary.
+file-replacement-native is only needed to build the target file binary so to
+avoid problems with the host libmagic vs file-replacement-native, don't install
+libmagic.
+
+Upstream-Status: Inappropriate
+Signed-off-by: Ross Burton <ross.burton@intel.com>
+
+diff --git a/src/Makefile.am b/src/Makefile.am
+index 575899f..052193a 100644
+--- a/src/Makefile.am
++++ b/src/Makefile.am
+@@ -1,13 +1,12 @@
+ MAGIC = $(pkgdatadir)/magic
+-lib_LTLIBRARIES = libmagic.la
+-include_HEADERS = magic.h
++noinst_LTLIBRARIES = libmagic.la
+ 
+ bin_PROGRAMS = file
+ 
+ AM_CPPFLAGS = -DMAGIC='"$(MAGIC)"'
+ AM_CFLAGS = $(CFLAG_VISIBILITY) @WARNINGS@
+ 
+-libmagic_la_SOURCES = magic.c apprentice.c softmagic.c ascmagic.c \
++libmagic_la_SOURCES = magic.c magic.h apprentice.c softmagic.c ascmagic.c \
+ 	encoding.c compress.c is_tar.c readelf.c print.c fsmagic.c \
+ 	funcs.c file.h readelf.h tar.h apptype.c \
+ 	file_opts.h elfclass.h mygetopt.h cdf.c cdf_time.c readcdf.c cdf.h
diff --git a/meta/recipes-devtools/file/file_5.24.bb b/meta/recipes-devtools/file/file_5.24.bb
index 08b95d7..c1d4578 100644
--- a/meta/recipes-devtools/file/file_5.24.bb
+++ b/meta/recipes-devtools/file/file_5.24.bb
@@ -8,19 +8,26 @@ SECTION = "console/utils"
 LICENSE = "BSD"
 LIC_FILES_CHKSUM = "file://COPYING;beginline=2;md5=6a7382872edb68d33e1a9398b6e03188"
 
-DEPENDS = "zlib file-native"
+DEPENDS = "zlib file-replacement-native"
 DEPENDS_class-native = "zlib-native"
 
 SRC_URI = "git://github.com/file/file.git \
         file://debian-742262.patch \
         file://0001-Add-P-prompt-into-Usage-info.patch \
+        file://host-file.patch \
         "
 
+# file-native is generally assume-provided so don't install the library as this
+# is for file-replacement-native.
+SRC_URI_append_class-native = "file://no-library.patch"
+
 SRCREV = "3c0874be4d3232d672b20f513451a39cfd7c585a"
 S = "${WORKDIR}/git"
 
 inherit autotools
 
+EXTRA_OEMAKE_append_class-target = "-e FILE_COMPILE=${STAGING_BINDIR_NATIVE}/file-native/file"
+
 FILES_${PN} += "${datadir}/misc/*.mgc"
 
 do_install_append_class-native() {
@@ -34,3 +41,5 @@ do_install_append_class-nativesdk() {
 }
 
 BBCLASSEXTEND = "native nativesdk"
+PROVIDES_append_class-native = " file-replacement-native"
+NATIVE_PACKAGE_PATH_SUFFIX = "/${PN}"
-- 
2.1.4

Re: [PATCH 2/3] sanity: check that the host has file installed

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

On Mon, Oct 19, 2015 at 11:05:28AM +0100, Ross Burton wrote:
> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
>  meta/classes/sanity.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 34f8618..a1fed36 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
> @@ -3,7 +3,7 @@
>  #
>  
>  SANITY_REQUIRED_UTILITIES ?= "patch diffstat makeinfo git bzip2 tar \
> -    gzip gawk chrpath wget cpio perl"
> +    gzip gawk chrpath wget cpio perl filex"

file or filex?

>  
>  def bblayers_conf_file(d):
>      return os.path.join(d.getVar('TOPDIR', True), 'conf/bblayers.conf')
> -- 
> 2.1.4
> 
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [PATCH 2/3] sanity: check that the host has file installed

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 185 bytes --]

On 19 October 2015 at 13:09, Martin Jansa <martin.jansa@gmail.com> wrote:

> file or filex?
>

Well that was embarassing.  I wonder how that test change made it into git.

Ross

[-- Attachment #2: Type: text/html, Size: 573 bytes --]

Re: [PATCH 2/3] sanity: check that the host has file installed

[Paul Eggleton]

On Monday 19 October 2015 13:12:56 Burton, Ross wrote:
> On 19 October 2015 at 13:09, Martin Jansa <martin.jansa@gmail.com> wrote:
> > file or filex?
> 
> Well that was embarassing.  I wonder how that test change made it into git.

Since this test isn't run on every build, it can be broken or unsatisfied and 
the build will still start. Given how quick this check is and that people have 
stumbled across the results of the check not having run, I wonder whether that 
should be changed.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH 2/3] sanity: check that the host has file installed

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 470 bytes --]

On 19 October 2015 at 14:34, Paul Eggleton <paul.eggleton@linux.intel.com>
wrote:

> > Well that was embarassing.  I wonder how that test change made it into
> git.
>
> Since this test isn't run on every build, it can be broken or unsatisfied
> and
> the build will still start. Given how quick this check is and that people
> have
> stumbled across the results of the check not having run, I wonder whether
> that
> should be changed.


Agreed.

Ross

[-- Attachment #2: Type: text/html, Size: 854 bytes --]

Re: [PATCH 1/3] bitbake: add file-native to ASSUME_PROVIDED

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]

So this breaks rpm-native v4, which needs file-native as it links to
libmagic.

Not sure what to do here.  If we install file-native directly into the
sysroot we get occasional races on the file binary  causing build
failures.  If we make file-native ASSUME_PROVIDED then rpm (and subversion)
change their behaviour depending on the presence of libmagic-dev on the
host (well, rpm4 requires libmagic, rpm5 and subversion change their
configuration).

Thoughts?

Ross

On 19 October 2015 at 11:05, Ross Burton <ross.burton@intel.com> wrote:

> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
>  meta/conf/bitbake.conf | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 9ddd1c0..06971da 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -166,6 +166,7 @@ DATETIME = "${DATE}${TIME}"
>  ASSUME_PROVIDED = "\
>      bzip2-native \
>      chrpath-native \
> +    file-native \
>      git-native \
>      grep-native \
>      diffstat-native \
> --
> 2.1.4
>
>

[-- Attachment #2: Type: text/html, Size: 1621 bytes --]

Re: [PATCH 1/3] bitbake: add file-native to ASSUME_PROVIDED

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

On 19 October 2015 at 16:37, Burton, Ross <ross.burton@intel.com> wrote:

> Not sure what to do here.  If we install file-native directly into the
> sysroot we get occasional races on the file binary  causing build
> failures.  If we make file-native ASSUME_PROVIDED then rpm (and subversion)
> change their behaviour depending on the presence of libmagic-dev on the
> host (well, rpm4 requires libmagic, rpm5 and subversion change their
> configuration).
>

Two proposals:

1) Tell RPM and Subversion to explicitly enable libmagic support so they
fail if libmagic-dev isn't installed on the host, and document that
libmagic-dev is a build requirement
2) Change file-replacement-native to install libmagic, and add
file-replacement-native as a dependency to subversion-native and rpm-native.

Does anyone have a strong opinion either way?

Ross

[-- Attachment #2: Type: text/html, Size: 1377 bytes --]

Re: [PATCH 1/3] bitbake: add file-native to ASSUME_PROVIDED

[Richard Purdie]

On Mon, 2015-10-19 at 17:11 +0100, Burton, Ross wrote:
> 
> On 19 October 2015 at 16:37, Burton, Ross <ross.burton@intel.com>
> wrote:
>         Not sure what to do here.  If we install file-native directly
>         into the sysroot we get occasional races on the file binary
>          causing build failures.  If we make file-native
>         ASSUME_PROVIDED then rpm (and subversion) change their
>         behaviour depending on the presence of libmagic-dev on the
>         host (well, rpm4 requires libmagic, rpm5 and subversion change
>         their configuration).
>         
> 
> Two proposals:
> 
> 
> 1) Tell RPM and Subversion to explicitly enable libmagic support so
> they fail if libmagic-dev isn't installed on the host, and document
> that libmagic-dev is a build requirement
> 2) Change file-replacement-native to install libmagic, and add
> file-replacement-native as a dependency to subversion-native and
> rpm-native.
>
> Does anyone have a strong opinion either way?

I tend to become nervous about -dev requirements so whilst ugly, I'm
leaning towards 2) which is what we've done for other similar issues
IIRC.

Cheers,

Richard

Re: [PATCH 1/3] bitbake: add file-native to ASSUME_PROVIDED

[Alex Franco]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

#2 seems more ubiquitous

Franco

On 10/19/2015 11:11 AM, Burton, Ross wrote:
>
> On 19 October 2015 at 16:37, Burton, Ross <ross.burton@intel.com
> <mailto:ross.burton@intel.com>> wrote:
>
>     Not sure what to do here.  If we install file-native directly into
>     the sysroot we get occasional races on the file binary  causing
>     build failures.  If we make file-native ASSUME_PROVIDED then rpm
>     (and subversion) change their behaviour depending on the presence
>     of libmagic-dev on the host (well, rpm4 requires libmagic, rpm5
>     and subversion change their configuration).
>
>
> Two proposals:
>
> 1) Tell RPM and Subversion to explicitly enable libmagic support so
> they fail if libmagic-dev isn't installed on the host, and document
> that libmagic-dev is a build requirement
> 2) Change file-replacement-native to install libmagic, and add
> file-replacement-native as a dependency to subversion-native and
> rpm-native.
>
> Does anyone have a strong opinion either way?
>
> Ross
>
>


[-- Attachment #2: Type: text/html, Size: 2390 bytes --]