[PATCH 0/3] Install external binary packages

[PATCH 0/3] Install external binary packages

[Robert Yang]

Hi folks,

Please see the first commit for details, and I will send a patch to
add the usage to local.conf.sample.extended when this is OK.

// Robert

The following changes since commit 0c6ac62a9fb81146ace64ae7493dcc56b4bed197:

  documentation: remove references to Pimlico (2012-08-15 15:26:27 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib robert/external_install
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/external_install

Robert Yang (3):
  rpm: install external binary packages
  ipk: install external binary packages
  deb: install external binary packages

 meta/classes/external_package.bbclass | 89 +++++++++++++++++++++++++++++++++++
 meta/classes/image.bbclass            |  2 +
 meta/classes/package_deb.bbclass      | 43 ++++++++++-------
 meta/classes/package_ipk.bbclass      | 21 ++++++++-
 meta/classes/package_rpm.bbclass      | 12 +++--
 meta/classes/rootfs_deb.bbclass       |  2 +-
 meta/classes/rootfs_ipk.bbclass       |  2 +-
 meta/classes/rootfs_rpm.bbclass       |  2 +-
 8 files changed, 149 insertions(+), 24 deletions(-)
 create mode 100644 meta/classes/external_package.bbclass

-- 
1.7.11.2

[PATCH 1/3] rpm: install external binary packages

[Robert Yang]

It's been suggested that it would be a useful feature to be able to
easily take a binary from a 3rd party software vendor and integrate
it into an image created by the build system.

* The user can easily use this by:
  # Specify where is the external binary pkg dir
  #EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
  # Specify which pkg will be installed
  #EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."

  Then the pkg1, pkg2 ... would be installed.

  Add an "EXTERNAL_INSTALL_PACKAGE"here since we can't use the existence
  variable (for example, IMAGE_INSTALL), if we add the pkg to the IMAGE_INSTALL,
  it would check whether the pkg is in the PROVIDES, and this is an external
  pkg, it is not in the PROVIDES.

* Main changes:
  - Add external_package.bbclass:
    Add the external package to the repo, the package would be copied
    to deploy/rpm/external/<arch>.
    Use an "external" directory since we don't want the "internal" pkgs
    to be mixed, and it would be easy to remove them.

  - package_rpm.bbclass:
    Create repo for deploy/rpm/external

  - rootfs_rpm.bbclass
    Add the package that would be installed to INSTALL_PACKAGES_RPM, so
    that it would be installed.

[YOCTO #1592]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/external_package.bbclass | 41 +++++++++++++++++++++++++++++++++++
 meta/classes/image.bbclass            |  2 ++
 meta/classes/package_rpm.bbclass      | 12 +++++++---
 meta/classes/rootfs_rpm.bbclass       |  2 +-
 4 files changed, 53 insertions(+), 4 deletions(-)
 create mode 100644 meta/classes/external_package.bbclass

diff --git a/meta/classes/external_package.bbclass b/meta/classes/external_package.bbclass
new file mode 100644
index 0000000..c6d87bb
--- /dev/null
+++ b/meta/classes/external_package.bbclass
@@ -0,0 +1,41 @@
+#
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Add external binary pkg to the repo and install them:
+#
+# Specify where are the external binary pkg dir
+#EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
+# Specify which pkg will be installed
+#EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."
+
+#
+# Add the external binary rpm into the repo, copy the binary rpm files
+# from EXTERNAL_PACKAGE_DIR to ${DEPLOY_DIR_RPM}/external, and put them
+# to the relevant arch dir.
+#
+add_external_rpm () {
+    local supported_archs="$@"
+
+    # Clean the EXTERNAL_DIR_RPM dir and re-copy
+    [ ! -d "${EXTERNAL_DIR_RPM}" ] || rm -fr ${EXTERNAL_DIR_RPM}
+
+    if [ -n "${EXTERNAL_PACKAGE_DIR}" -a -n "${EXTERNAL_INSTALL_PACKAGE}" ]; then
+        echo "Adding external binary rpms to the repo ..."
+        for f in `find ${EXTERNAL_PACKAGE_DIR} -type f  -name '*.rpm'`; do
+            arch="`echo $f | awk -F\. '{print $(NF-1)}'`"
+            found=""
+            for archvar in $supported_archs; do
+                # Only pick up the supported arch's rpm
+                if [ "$arch" == "$archvar" ]; then
+                    [ -d "${EXTERNAL_DIR_RPM}/$arch" ] || mkdir -p ${EXTERNAL_DIR_RPM}/$arch
+                    cp -f $f ${EXTERNAL_DIR_RPM}/$arch/
+                    found="1"
+                    break
+                fi
+            done
+            [ -n "$found" ] || echo "Uknown arch $arch"
+        done
+    fi
+}
+
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 72720f1..283688a 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -5,6 +5,8 @@ inherit imagetest-${IMAGETEST}
 
 inherit populate_sdk_base
 
+inherit external_package
+
 TOOLCHAIN_TARGET_TASK += "${PACKAGE_INSTALL}"
 TOOLCHAIN_TARGET_TASK_ATTEMPTONLY += "${PACKAGE_INSTALL_ATTEMPTONLY}"
 POPULATE_SDK_POST_TARGET_COMMAND += "rootfs_install_complementary populate_sdk; "
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index b58ae85..bd2c9a2 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -7,6 +7,7 @@ RPMBUILD="rpmbuild"
 
 PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
 PKGWRITEDIRSRPM = "${DEPLOY_DIR}/sources/deploy-srpm"
+EXTERNAL_DIR_RPM = "${DEPLOY_DIR_RPM}/external"
 
 python package_rpm_fn () {
     d.setVar('PKGFN', d.getVar('PKG'))
@@ -26,6 +27,9 @@ package_update_index_rpm () {
 		return
 	fi
 
+	# Add external binary pkgs
+	add_external_rpm ${PACKAGE_ARCHS} ${MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
+
 	# Update target packages
 	base_archs="${PACKAGE_ARCHS}"
 	ml_archs="${MULTILIB_PACKAGE_ARCHS}"
@@ -44,9 +48,11 @@ package_update_index_rpm_common () {
 	for archvar in "$@"; do
 		eval archs=\${${archvar}}
 		packagedirs=""
-		for arch in $archs; do
-			packagedirs="${DEPLOY_DIR_RPM}/$arch $packagedirs"
-			rm -rf ${DEPLOY_DIR_RPM}/$arch/solvedb.done
+		for d in ${DEPLOY_DIR_RPM} ${EXTERNAL_DIR_RPM}; do
+			for arch in $archs; do
+				packagedirs="$d/$arch $packagedirs"
+				rm -rf $d/$arch/solvedb.done
+			done
 		done
 
 		cat /dev/null > ${rpmconf_base}-${archvar}.conf
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index c0207d8..7ce694c 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -51,7 +51,7 @@ fakeroot rootfs_rpm_do_rootfs () {
 	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
-	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
 	export INSTALL_PROVIDENAME_RPM=""
-- 
1.7.11.2

[PATCH 2/3] ipk: install external binary packages

[Robert Yang]

Please see the commit message for the rpm one, most of the contents are
similar, except that add an function "add_external_debipk" which are
used by both deb and ipk, the add_external_ipk is just a wrapper.

[YOCTO #2948]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/external_package.bbclass | 44 +++++++++++++++++++++++++++++++++++
 meta/classes/package_ipk.bbclass      | 21 ++++++++++++++++-
 meta/classes/rootfs_ipk.bbclass       |  2 +-
 3 files changed, 65 insertions(+), 2 deletions(-)

diff --git a/meta/classes/external_package.bbclass b/meta/classes/external_package.bbclass
index c6d87bb..1587363 100644
--- a/meta/classes/external_package.bbclass
+++ b/meta/classes/external_package.bbclass
@@ -39,3 +39,47 @@ add_external_rpm () {
     fi
 }
 
+#
+# Add the external binary ipk/deb into the repo, copy the binary files
+# from EXTERNAL_PACKAGE_DIR to ${DEPLOY_DIR_IPK}/external, and put them
+# to the relevant arch dir.
+#
+# $1: EXTERNAL_DIR_IPK or EXTERNAL_DIR_DEB
+# $2, $3, $4...: supported archs
+#
+add_external_debipk () {
+    dir="$1"
+
+    if [ "$dir" != "${EXTERNAL_DIR_DEB}" -a "$dir" != "${EXTERNAL_DIR_IPK}" ]; then
+        echo "Unsupported dir $1"
+        return
+    fi
+
+    shift
+    local supported_archs="$@"
+
+    # Clean the dir and re-copy
+    [ ! -d "$dir" ] || rm -fr $dir
+
+    if [ -n "${EXTERNAL_PACKAGE_DIR}" -a -n "${EXTERNAL_INSTALL_PACKAGE}" ]; then
+        echo "Adding external binary ${IMAGE_PKGTYPE} to the repo ..."
+        for f in `find ${EXTERNAL_PACKAGE_DIR} -type f  -name "*.${IMAGE_PKGTYPE}"`; do
+            arch="`echo $f | sed 's/.*_\(.*\)\.'"${IMAGE_PKGTYPE}"'$/\1/'`"
+            found=""
+            for archvar in $supported_archs; do
+                # Only pick up the supported arch
+                if [ "$arch" == "$archvar" ]; then
+                    [ -d "$dir/$arch" ] || mkdir -p $dir/$arch
+                    cp -f $f $dir/$arch/
+                    found="1"
+                    break
+                fi
+            done
+            [ -n "$found" ] || echo "Uknown arch $arch"
+        done
+    fi
+}
+
+add_external_ipk () {
+	add_external_debipk $@
+}
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index a297a1f..12e493d 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -6,6 +6,7 @@ IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
 IPKGCONF_SDK =  "${WORKDIR}/opkg-sdk.conf"
 
 PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
+EXTERNAL_DIR_IPK = "${DEPLOY_DIR_IPK}/external"
 
 # Program to be used to build opkg packages
 OPKGBUILDCMD ??= "opkg-build"
@@ -203,7 +204,7 @@ package_update_index_ipk () {
 		return
 	fi
 
-	packagedirs="${DEPLOY_DIR_IPK}"
+	packagedirs="${DEPLOY_DIR_IPK} ${EXTERNAL_DIR_IPK}"
 	for arch in $ipkgarchs; do
 		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
@@ -213,6 +214,12 @@ package_update_index_ipk () {
 		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
 
+	all_archs="$ipkgarchs $multilib_archs"
+	add_external_ipk "${EXTERNAL_DIR_IPK}" $all_archs
+	for arch in $all_archs; do
+		packagedirs="$packagedirs ${EXTERNAL_DIR_IPK}/$arch"
+	done
+
 	for pkgdir in $packagedirs; do
 		if [ -e $pkgdir/ ]; then
 			touch $pkgdir/Packages
@@ -229,19 +236,31 @@ package_update_index_ipk () {
 package_generate_ipkg_conf () {
 	package_generate_archlist
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_SDK}
+	if [ -e ${EXTERNAL_DIR_IPK} ]; then
+		echo "src external file:${EXTERNAL_DIR_IPK}" >> ${IPKGCONF_SDK}
+	fi
 	ipkgarchs="${SDK_PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
 		fi
+		if [ -e ${EXTERNAL_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src external-$arch file:${EXTERNAL_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
+		fi
 	done
 
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
+	if [ -e ${EXTERNAL_DIR_IPK} ]; then
+		echo "src external file:${EXTERNAL_DIR_IPK}" >> ${IPKGCONF_TARGET}
+	fi
 	ipkgarchs="${ALL_MULTILIB_PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
 		fi
+		if [ -e ${EXTERNAL_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src external-$arch file:${EXTERNAL_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
+		fi
 	done
 }
 
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index 6cdd8f6..edea67f 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -62,7 +62,7 @@ fakeroot rootfs_ipk_do_rootfs () {
 
 	export INSTALL_ROOTFS_IPK="${IMAGE_ROOTFS}"
 	export INSTALL_CONF_IPK="${IPKGCONF_TARGET}"
-	export INSTALL_PACKAGES_IPK="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_IPK="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 
 	#post install
 	export D=${IMAGE_ROOTFS}
-- 
1.7.11.2

[PATCH 3/3] deb: install external binary packages

[Robert Yang]

Please see the commit message for the rpm one, most of the contents are
similar.

[YOCTO #2949]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/external_package.bbclass |  4 ++++
 meta/classes/package_deb.bbclass      | 43 +++++++++++++++++++++--------------
 meta/classes/rootfs_deb.bbclass       |  2 +-
 3 files changed, 31 insertions(+), 18 deletions(-)

diff --git a/meta/classes/external_package.bbclass b/meta/classes/external_package.bbclass
index 1587363..618ae72 100644
--- a/meta/classes/external_package.bbclass
+++ b/meta/classes/external_package.bbclass
@@ -83,3 +83,7 @@ add_external_debipk () {
 add_external_ipk () {
 	add_external_debipk $@
 }
+
+add_external_deb () {
+	add_external_debipk $@
+}
diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 48511df..477b90c 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -9,6 +9,7 @@ IMAGE_PKGTYPE ?= "deb"
 DPKG_ARCH ?= "${TARGET_ARCH}" 
 
 PKGWRITEDIRDEB = "${WORKDIR}/deploy-debs"
+EXTERNAL_DIR_DEB = "${DEPLOY_DIR_DEB}/external"
 
 python package_deb_fn () {
     d.setVar('PKGFN', d.getVar('PKG'))
@@ -72,19 +73,25 @@ package_update_index_deb () {
 		return
 	fi
 
-	for arch in ${PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}; do
-		if [ -e ${DEPLOY_DIR_DEB}/$arch ]; then
+	all_archs="${PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}"
+
+	[ ! -d ${EXTERNAL_INSTALL_DIRS} ] || add_external_deb ${EXTERNAL_DIR_DEB} $all_archs
+
+	for arch in $all_archs; do
+		if [ -e ${DEPLOY_DIR_DEB}/$arch -o -e ${EXTERNAL_DIR_DEB}/$arch ]; then
 			debarchs="$debarchs $arch"
 		fi
 	done
 
 	for arch in $debarchs; do
-		if [ ! -d ${DEPLOY_DIR_DEB}/$arch ]; then
-			continue;
-		fi
-		cd ${DEPLOY_DIR_DEB}/$arch
-		dpkg-scanpackages . | gzip > Packages.gz
-		echo "Label: $arch" > Release
+		for d in ${DEPLOY_DIR_DEB} ${EXTERNAL_DIR_DEB}; do
+			if [ ! -d $d/$arch ]; then
+				continue;
+			fi
+			cd $d/$arch
+			dpkg-scanpackages . | gzip > Packages.gz
+			echo "Label: $arch" > Release
+		done
 	done
 }
 
@@ -114,16 +121,18 @@ package_install_internal_deb () {
 
 	priority=1
 	for arch in $archs; do
-		if [ ! -d ${DEPLOY_DIR_DEB}/$arch ]; then
-			continue;
-		fi
+		for d in ${DEPLOY_DIR_DEB} ${EXTERNAL_DIR_DEB}; do
+			if [ ! -d $d/$arch ]; then
+				continue;
+			fi
 
-		echo "deb file:${DEPLOY_DIR_DEB}/$arch/ ./" >> ${STAGING_ETCDIR_NATIVE}/apt/sources.list.rev
-		(echo "Package: *"
-		echo "Pin: release l=$arch"
-		echo "Pin-Priority: $(expr 800 + $priority)"
-		echo) >> ${STAGING_ETCDIR_NATIVE}/apt/preferences
-		priority=$(expr $priority + 5)
+			echo "deb file:$d/$arch/ ./" >> ${STAGING_ETCDIR_NATIVE}/apt/sources.list.rev
+			(echo "Package: *"
+			echo "Pin: release l=$arch"
+			echo "Pin-Priority: $(expr 800 + $priority)"
+			echo) >> ${STAGING_ETCDIR_NATIVE}/apt/preferences
+			priority=$(expr $priority + 5)
+		done
 	done
 
 	tac ${STAGING_ETCDIR_NATIVE}/apt/sources.list.rev > ${STAGING_ETCDIR_NATIVE}/apt/sources.list
diff --git a/meta/classes/rootfs_deb.bbclass b/meta/classes/rootfs_deb.bbclass
index 750a8ca..0c7f5f2 100644
--- a/meta/classes/rootfs_deb.bbclass
+++ b/meta/classes/rootfs_deb.bbclass
@@ -34,7 +34,7 @@ fakeroot rootfs_deb_do_rootfs () {
 	export INSTALL_ROOTFS_DEB="${IMAGE_ROOTFS}"
 	export INSTALL_BASEARCH_DEB="${DPKG_ARCH}"
 	export INSTALL_ARCHS_DEB="${PACKAGE_ARCHS}"
-	export INSTALL_PACKAGES_NORMAL_DEB="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_NORMAL_DEB="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_DEB="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_DEB="${LINGUAS_INSTALL}"
 	export INSTALL_TASK_DEB="rootfs"
-- 
1.7.11.2

Re: [PATCH 1/3] rpm: install external binary packages

[Robert Yang]

On 08/15/2012 11:32 PM, Robert Yang wrote:
>
> diff --git a/meta/classes/external_package.bbclass b/meta/classes/external_package.bbclass
> new file mode 100644
> index 0000000..c6d87bb
> --- /dev/null
> +++ b/meta/classes/external_package.bbclass
> @@ -0,0 +1,41 @@
> +#
> +# ex:ts=4:sw=4:sts=4:et
> +# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
> +#
> +# Add external binary pkg to the repo and install them:
> +#
> +# Specify where are the external binary pkg dir
> +#EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
> +# Specify which pkg will be installed
> +#EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."
> +
> +#
> +# Add the external binary rpm into the repo, copy the binary rpm files
> +# from EXTERNAL_PACKAGE_DIR to ${DEPLOY_DIR_RPM}/external, and put them
> +# to the relevant arch dir.
> +#
> +add_external_rpm () {
> +    local supported_archs="$@"
> +

I updated this patch and the "PATCH 2/3" a little since the "dash" doesn't
support the local var="$@", so changed it to:

local supported_archs
supported_archs="$@"

and pushed to the repo:

   git://git.pokylinux.org/poky-contrib robert/external_install
   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/external_install

// Robert

> +    # Clean the EXTERNAL_DIR_RPM dir and re-copy
> +    [ ! -d "${EXTERNAL_DIR_RPM}" ] || rm -fr ${EXTERNAL_DIR_RPM}
> +
> +    if [ -n "${EXTERNAL_PACKAGE_DIR}" -a -n "${EXTERNAL_INSTALL_PACKAGE}" ]; then
> +        echo "Adding external binary rpms to the repo ..."
> +        for f in `find ${EXTERNAL_PACKAGE_DIR} -type f  -name '*.rpm'`; do
> +            arch="`echo $f | awk -F\. '{print $(NF-1)}'`"
> +            found=""
> +            for archvar in $supported_archs; do
> +                # Only pick up the supported arch's rpm
> +                if [ "$arch" == "$archvar" ]; then
> +                    [ -d "${EXTERNAL_DIR_RPM}/$arch" ] || mkdir -p ${EXTERNAL_DIR_RPM}/$arch
> +                    cp -f $f ${EXTERNAL_DIR_RPM}/$arch/
> +                    found="1"
> +                    break
> +                fi
> +            done
> +            [ -n "$found" ] || echo "Uknown arch $arch"
> +        done
> +    fi
> +}
> +
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 72720f1..283688a 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -5,6 +5,8 @@ inherit imagetest-${IMAGETEST}
>
>   inherit populate_sdk_base
>
> +inherit external_package
> +
>   TOOLCHAIN_TARGET_TASK += "${PACKAGE_INSTALL}"
>   TOOLCHAIN_TARGET_TASK_ATTEMPTONLY += "${PACKAGE_INSTALL_ATTEMPTONLY}"
>   POPULATE_SDK_POST_TARGET_COMMAND += "rootfs_install_complementary populate_sdk; "
> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
> index b58ae85..bd2c9a2 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -7,6 +7,7 @@ RPMBUILD="rpmbuild"
>
>   PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
>   PKGWRITEDIRSRPM = "${DEPLOY_DIR}/sources/deploy-srpm"
> +EXTERNAL_DIR_RPM = "${DEPLOY_DIR_RPM}/external"
>
>   python package_rpm_fn () {
>       d.setVar('PKGFN', d.getVar('PKG'))
> @@ -26,6 +27,9 @@ package_update_index_rpm () {
>   		return
>   	fi
>
> +	# Add external binary pkgs
> +	add_external_rpm ${PACKAGE_ARCHS} ${MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
> +
>   	# Update target packages
>   	base_archs="${PACKAGE_ARCHS}"
>   	ml_archs="${MULTILIB_PACKAGE_ARCHS}"
> @@ -44,9 +48,11 @@ package_update_index_rpm_common () {
>   	for archvar in "$@"; do
>   		eval archs=\${${archvar}}
>   		packagedirs=""
> -		for arch in $archs; do
> -			packagedirs="${DEPLOY_DIR_RPM}/$arch $packagedirs"
> -			rm -rf ${DEPLOY_DIR_RPM}/$arch/solvedb.done
> +		for d in ${DEPLOY_DIR_RPM} ${EXTERNAL_DIR_RPM}; do
> +			for arch in $archs; do
> +				packagedirs="$d/$arch $packagedirs"
> +				rm -rf $d/$arch/solvedb.done
> +			done
>   		done
>
>   		cat /dev/null > ${rpmconf_base}-${archvar}.conf
> diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
> index c0207d8..7ce694c 100644
> --- a/meta/classes/rootfs_rpm.bbclass
> +++ b/meta/classes/rootfs_rpm.bbclass
> @@ -51,7 +51,7 @@ fakeroot rootfs_rpm_do_rootfs () {
>   	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
>   	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
>   	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
> -	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL}"
> +	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
>   	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
>   	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
>   	export INSTALL_PROVIDENAME_RPM=""
>

Re: [PATCH 2/3] ipk: install external binary packages

[Koen Kooi]

Op 15 aug. 2012, om 17:32 heeft Robert Yang <liezhi.yang@windriver.com> het volgende geschreven:

> Please see the commit message for the rpm one, 

No, you copy-paste that message into this commit message.

Re: [PATCH 2/3] ipk: install external binary packages

[Robert Yang]

On 08/16/2012 04:03 PM, Koen Kooi wrote:
>
> Op 15 aug. 2012, om 17:32 heeft Robert Yang <liezhi.yang@windriver.com> het volgende geschreven:
>
>> Please see the commit message for the rpm one,
>
> No, you copy-paste that message into this commit message.
>

Thanks, sounds better, V2 is coming.

// Robert

_______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

[PATCH 2/3] ipk: install external binary packages

[Robert Yang]

It's been suggested that it would be a useful feature to be able to
easily take a binary from a 3rd party software vendor and integrate
it into an image created by the build system.

* The user can easily use this by:
  # Specify where is the external binary pkg dir
  #EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
  # Specify which pkg will be installed
  #EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."

  Then the pkg1, pkg2 ... would be installed.

  Add an "EXTERNAL_INSTALL_PACKAGE"here since we can't use the existence
  variable (for example, IMAGE_INSTALL), if we add the pkg to the IMAGE_INSTALL,
  it would check whether the pkg is in the PROVIDES, and this is an external
  pkg, it is not in the PROVIDES.

* Main changes:
  - Add external_package.bbclass:
    Add the external package to the repo, the package would be copied
    to deploy/ipk/external/<arch>, the add_external_debipk is used by
    ipk and deb, the add_external_ipk is just a wrapper.

    Use an "external" directory since we don't want the "internal" pkgs
    to be mixed, and it would be easy to remove them.

  - package_ipk.bbclass:
    Create repo for deploy/ipk/external

  - rootfs_ipk.bbclass
    Add the package that would be installed to INSTALL_PACKAGES_IPK, so
    that it would be installed.

[YOCTO #2948]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/external_package.bbclass | 45 +++++++++++++++++++++++++++++++++++
 meta/classes/package_ipk.bbclass      | 21 +++++++++++++++-
 meta/classes/rootfs_ipk.bbclass       |  2 +-
 3 files changed, 66 insertions(+), 2 deletions(-)

diff --git a/meta/classes/external_package.bbclass b/meta/classes/external_package.bbclass
index e032bec..c21a5c6 100644
--- a/meta/classes/external_package.bbclass
+++ b/meta/classes/external_package.bbclass
@@ -40,3 +40,48 @@ add_external_rpm () {
     fi
 }
 
+#
+# Add the external binary ipk/deb into the repo, copy the binary files
+# from EXTERNAL_PACKAGE_DIR to ${DEPLOY_DIR_IPK}/external, and put them
+# to the relevant arch dir.
+#
+# $1: EXTERNAL_DIR_IPK or EXTERNAL_DIR_DEB
+# $2, $3, $4...: supported archs
+#
+add_external_debipk () {
+    dir="$1"
+
+    if [ "$dir" != "${EXTERNAL_DIR_DEB}" -a "$dir" != "${EXTERNAL_DIR_IPK}" ]; then
+        echo "Unsupported dir $1"
+        return
+    fi
+
+    shift
+    local supported_archs
+    supported_archs="$@"
+
+    # Clean the dir and re-copy
+    [ ! -d "$dir" ] || rm -fr $dir
+
+    if [ -n "${EXTERNAL_PACKAGE_DIR}" -a -n "${EXTERNAL_INSTALL_PACKAGE}" ]; then
+        echo "Adding external binary ${IMAGE_PKGTYPE} to the repo ..."
+        for f in `find ${EXTERNAL_PACKAGE_DIR} -type f  -name "*.${IMAGE_PKGTYPE}"`; do
+            arch="`echo $f | sed 's/.*_\(.*\)\.'"${IMAGE_PKGTYPE}"'$/\1/'`"
+            found=""
+            for archvar in $supported_archs; do
+                # Only pick up the supported arch
+                if [ "$arch" == "$archvar" ]; then
+                    [ -d "$dir/$arch" ] || mkdir -p $dir/$arch
+                    cp -f $f $dir/$arch/
+                    found="1"
+                    break
+                fi
+            done
+            [ -n "$found" ] || echo "Uknown arch $arch"
+        done
+    fi
+}
+
+add_external_ipk () {
+	add_external_debipk $@
+}
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index a297a1f..12e493d 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -6,6 +6,7 @@ IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
 IPKGCONF_SDK =  "${WORKDIR}/opkg-sdk.conf"
 
 PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
+EXTERNAL_DIR_IPK = "${DEPLOY_DIR_IPK}/external"
 
 # Program to be used to build opkg packages
 OPKGBUILDCMD ??= "opkg-build"
@@ -203,7 +204,7 @@ package_update_index_ipk () {
 		return
 	fi
 
-	packagedirs="${DEPLOY_DIR_IPK}"
+	packagedirs="${DEPLOY_DIR_IPK} ${EXTERNAL_DIR_IPK}"
 	for arch in $ipkgarchs; do
 		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
@@ -213,6 +214,12 @@ package_update_index_ipk () {
 		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
 
+	all_archs="$ipkgarchs $multilib_archs"
+	add_external_ipk "${EXTERNAL_DIR_IPK}" $all_archs
+	for arch in $all_archs; do
+		packagedirs="$packagedirs ${EXTERNAL_DIR_IPK}/$arch"
+	done
+
 	for pkgdir in $packagedirs; do
 		if [ -e $pkgdir/ ]; then
 			touch $pkgdir/Packages
@@ -229,19 +236,31 @@ package_update_index_ipk () {
 package_generate_ipkg_conf () {
 	package_generate_archlist
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_SDK}
+	if [ -e ${EXTERNAL_DIR_IPK} ]; then
+		echo "src external file:${EXTERNAL_DIR_IPK}" >> ${IPKGCONF_SDK}
+	fi
 	ipkgarchs="${SDK_PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
 		fi
+		if [ -e ${EXTERNAL_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src external-$arch file:${EXTERNAL_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
+		fi
 	done
 
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
+	if [ -e ${EXTERNAL_DIR_IPK} ]; then
+		echo "src external file:${EXTERNAL_DIR_IPK}" >> ${IPKGCONF_TARGET}
+	fi
 	ipkgarchs="${ALL_MULTILIB_PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
 		fi
+		if [ -e ${EXTERNAL_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src external-$arch file:${EXTERNAL_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
+		fi
 	done
 }
 
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index 6cdd8f6..edea67f 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -62,7 +62,7 @@ fakeroot rootfs_ipk_do_rootfs () {
 
 	export INSTALL_ROOTFS_IPK="${IMAGE_ROOTFS}"
 	export INSTALL_CONF_IPK="${IPKGCONF_TARGET}"
-	export INSTALL_PACKAGES_IPK="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_IPK="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 
 	#post install
 	export D=${IMAGE_ROOTFS}
-- 
1.7.11.2

Re: [PATCH 2/3] ipk: install external binary packages

[Koen Kooi]

Op 16 aug. 2012, om 10:27 heeft Robert Yang <liezhi.yang@windriver.com> het volgende geschreven:

> It's been suggested that it would be a useful feature to be able to
> easily take a binary from a 3rd party software vendor and integrate
> it into an image created by the build system.

So why not write a recipe for such packages? That way you can fix a lot more of packaging screwups (it's a vendor binary, screwups are a given) and don't need to patch a ton of classes.

Re: [PATCH 2/3] ipk: install external binary packages

[Robert Yang]

On 08/16/2012 05:40 PM, Koen Kooi wrote:
>
> Op 16 aug. 2012, om 10:27 heeft Robert Yang <liezhi.yang@windriver.com> het volgende geschreven:
>
>> It's been suggested that it would be a useful feature to be able to
>> easily take a binary from a 3rd party software vendor and integrate
>> it into an image created by the build system.
>
> So why not write a recipe for such packages? That way you can fix a lot more of packaging screwups (it's a vendor binary, screwups are a given) and don't need to patch a ton of classes.
>

I think that this is easier than use recipe for such packages, here are my
rough thoughts:

If we do it in a recipe, we should tell the recipe:

1) Where are the binary pkgs

2) Which recipe will be installed

These 2 are similar to EXTERNAL_PACKAGE_DIR and EXTERNAL_INSTALL_PACKAGE which
I have used.

3) Write the do_fetch to copy the pkgs to $WORKDIR

4) Write the do_install to uncompress the binary pkgs if we want to fix the
    screwups (I don't know the exactly meaning of screwups, I think it is
    similar to "trouble"), and I don't know what can we fix for a binary pkg,
    the common errors that we meet are the dependencies and conflicts, but it
    seems that we can do nothing on them. If we can't fix the pkg, it seems not
    very different from patch the class.

And from the user side, he needs to add the recipe to the image, then
specify where are the binary pkgs and which needs to be installed. It is not
as easy as the current implementation.

// Robert




_______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>