[PATCH 0/5] Fix the SDK generation

[PATCH 0/5] Fix the SDK generation

[Mark Hatle]

This is an attempt to fix a number of issues with the SDK generation.  It has been
tested using bitbake meta-toolchain-sdk.

Note, patches:

 4/5: package_ipk: SDK generation workaround
 5/5: update-alternatives: Add alternatives as a runtime provide

4/5 might be reasonable to implement immediately to resolve this issue.. However
5/5 is absolutely incorrect and that is noted within the patch header.  I'd
like comments on the issue to understand how to fix the problem.


The following changes since commit 2a41a311ddda11713296391050f3c2c1b2c1d3d3:

  arch-armv7a.inc: fix armv7a-vfp-neon -> armv7a compat case (2011-08-01 16:49:03 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib mhatle/oecore/sdk-fix-WIP
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/oecore/sdk-fix-WIP

Mark Hatle (5):
  rootfs_rpm: Cleanup and minor bug fixes
  bitbake.conf: Add SDK_PACKAGE_ARCHS
  populate_sdk_*: Sync SDK and regular rootfs functions
  package_ipk: SDK generation workaround
  update-alternatives: Add alternatives as a runtime provide

 meta/classes/package_deb.bbclass         |    6 +-
 meta/classes/package_ipk.bbclass         |   43 +++++------
 meta/classes/package_rpm.bbclass         |  120 ++++++++++++------------------
 meta/classes/populate_sdk.bbclass        |    9 ++
 meta/classes/populate_sdk_deb.bbclass    |   35 ++++-----
 meta/classes/populate_sdk_rpm.bbclass    |   89 +++++++++++++++++------
 meta/classes/rootfs_rpm.bbclass          |   30 ++++----
 meta/classes/update-alternatives.bbclass |    2 +
 meta/conf/bitbake.conf                   |    1 +
 9 files changed, 179 insertions(+), 156 deletions(-)

-- 
1.7.3.4

[PATCH 1/5] rootfs_rpm: Cleanup and minor bug fixes

[Mark Hatle]

in packages_rpm.bbclass:

Cleanup the way package_update_index_rpm works.  Break this out into two
functions, one for common functionality and one for controlling target vs
sdk package processing.

Similar change to package_generate_rpm_conf.

Usage of IMAGE_ROOTFS should actually be "target_rootfs", this fixes a problem
with the SDK case.

in rootfs_rpm.bbclass:

Minor re-order of the values, and avoid multiple uses of IMAGE_ROOTFS in order
to make the merge with the SDK behavior easier.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/package_rpm.bbclass |  128 ++++++++++++++++---------------------
 meta/classes/rootfs_rpm.bbclass  |   17 +++---
 2 files changed, 64 insertions(+), 81 deletions(-)

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 950f665..0f16613 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -25,31 +25,41 @@ package_update_index_rpm () {
 		return
 	fi
 
-	base_package_archs="${PACKAGE_ARCHS}"
-	ml_package_archs="${MULTILIB_PACKAGE_ARCHS}"
+	# Update target packages
+	base_archs="${PACKAGE_ARCHS}"
+	ml_archs="${MULTILIB_PACKAGE_ARCHS}"
+	package_update_index_rpm_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
+
+	# Update SDK packages
+	base_archs=""
+	for arch in ${PACKAGE_ARCHS}; do
+		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
+		extension="-nativesdk"
+		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
+		    extension=""
+		fi
+		base_archs="$base_archs $sdkarch$extension"
+	done
+	package_update_index_rpm_common "${RPMCONF_HOST_BASE}" base_archs
+}
+
+package_update_index_rpm_common () {
+	rpmconf_base="$1"
+	shift
 
-	for archvar in base_package_archs ml_package_archs; do
+	for archvar in "$@"; do
 		eval archs=\${${archvar}}
 		packagedirs=""
-		packagedirs_sdk=""
 		for arch in $archs; do
-			sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-			extension="-nativesdk"
-			if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-			    extension=""
-			fi
 			packagedirs="${DEPLOY_DIR_RPM}/$arch $packagedirs"
-			packagedirs_sdk="${DEPLOY_DIR_RPM}/$sdkarch$extension $packagedirs_sdk"
-
 			rm -rf ${DEPLOY_DIR_RPM}/$arch/solvedb
-			rm -rf ${DEPLOY_DIR_RPM}/$sdkarch$extension/solvedb
 		done
 
-		cat /dev/null > ${RPMCONF_TARGET_BASE}-${archvar}.conf
+		cat /dev/null > ${rpmconf_base}-${archvar}.conf
 		for pkgdir in $packagedirs; do
 			if [ -e $pkgdir/ ]; then
 				echo "Generating solve db for $pkgdir..."
-				echo $pkgdir/solvedb >> ${RPMCONF_TARGET_BASE}-${archvar}.conf
+				echo $pkgdir/solvedb >> ${rpmconf_base}-${archvar}.conf
 				if [ -d $pkgdir/solvedb ]; then
 					# We've already processed this and it's a duplicate
 					continue
@@ -65,27 +75,6 @@ package_update_index_rpm () {
 					$pkgdir/solvedb/manifest
 			fi
 		done
-
-		cat /dev/null > ${RPMCONF_HOST_BASE}-${archvar}.conf
-		for pkgdir in $packagedirs_sdk; do
-			if [ -e $pkgdir/ ]; then
-				echo "Generating solve db for $pkgdir..."
-				echo $pkgdir/solvedb >> ${RPMCONF_HOST_BASE}-${archvar}.conf
-				if [ -d $pkgdir/solvedb ]; then
-					# We've already processed this and it's a duplicate
-					continue
-				fi	
-				mkdir -p $pkgdir/solvedb
-				echo "# Dynamically generated solve manifest" >> $pkgdir/solvedb/manifest
-				find $pkgdir -maxdepth 1 -type f >> $pkgdir/solvedb/manifest
-				${RPM} -i --replacepkgs --replacefiles --oldpackage \
-					-D "_dbpath $pkgdir/solvedb" --justdb \
-					--noaid --nodeps --noorder --noscripts --notriggers --noparentdirs --nolinktos --stats \
-					--ignoresize --nosignature --nodigest \
-					-D "__dbi_txn create nofsync" \
-					$pkgdir/solvedb/manifest
-			fi
-		done
 	done
 }
 
@@ -94,45 +83,38 @@ package_update_index_rpm () {
 # generated depsolver db's...
 #
 package_generate_rpm_conf () {
-	printf "_solve_dbpath " > ${RPMCONF_TARGET_BASE}.macro
-	o_colon_t=false
-	o_colon_h=false
+	# Update target packages
+	package_generate_rpm_conf_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
 
-	for archvar in base_package_archs ml_package_archs; do
-		printf "_solve_dbpath " > ${RPMCONF_TARGET_BASE}-${archvar}.macro
-		colon=false
-		for each in `cat ${RPMCONF_TARGET_BASE}-${archvar}.conf` ; do
-			if [ "$o_colon_t" == true ]; then
-				printf ":" >> ${RPMCONF_TARGET_BASE}.macro
-			fi
-			if [ "$colon" == true ]; then
-				printf ":" >> ${RPMCONF_TARGET_BASE}-${archvar}.macro
-			fi
-			printf "%s" $each >> ${RPMCONF_TARGET_BASE}.macro
-			o_colon_t=true
-			printf "%s" $each >> ${RPMCONF_TARGET_BASE}-${archvar}.macro
-			colon=true
-		done
-		printf "\n" >> ${RPMCONF_TARGET_BASE}-${archvar}.macro
+	# Update SDK packages
+	package_generate_rpm_conf_common "${RPMCONF_HOST_BASE}" base_archs
+}
+
+package_generate_rpm_conf_common() {
+	rpmconf_base="$1"
+	shift
+
+	printf "_solve_dbpath " > ${rpmconf_base}.macro
+	o_colon=false
 
-		printf "_solve_dbpath " > ${RPMCONF_HOST_BASE}-${archvar}.macro
+	for archvar in "$@"; do
+		printf "_solve_dbpath " > ${rpmconf_base}-${archvar}.macro
 		colon=false
-		for each in `cat ${RPMCONF_HOST_BASE}-${archvar}.conf` ; do
-			if [ "$o_colon_h" == true ]; then
-				printf ":" >> ${RPMCONF_HOST_BASE}.macro
+		for each in `cat ${rpmconf_base}-${archvar}.conf` ; do
+			if [ "$o_colon" == true ]; then
+				printf ":" >> ${rpmconf_base}.macro
 			fi
 			if [ "$colon" == true ]; then
-				printf ":" >> ${RPMCONF_HOST_BASE}-${archvar}.macro
+				printf ":" >> ${rpmconf_base}-${archvar}.macro
 			fi
-			printf "%s" $each >> ${RPMCONF_HOST_BASE}.macro
-			o_colon_h=true
-			printf "%s" $each >> ${RPMCONF_HOST_BASE}-${archvar}.macro
+			printf "%s" $each >> ${rpmconf_base}.macro
+			o_colon=true
+			printf "%s" $each >> ${rpmconf_base}-${archvar}.macro
 			colon=true
 		done
-		printf "\n" >> ${RPMCONF_HOST_BASE}-${archvar}.macro
+		printf "\n" >> ${rpmconf_base}-${archvar}.macro
 	done
-	printf "\n" >> ${RPMCONF_TARGET_BASE}.macro
-	printf "\n" >> ${RPMCONF_HOST_BASE}.macro
+	printf "\n" >> ${rpmconf_base}.macro
 }
 
 rpm_log_check() {
@@ -211,7 +193,7 @@ package_install_internal_rpm () {
 	mkdir -p ${target_rootfs}/etc/rpm/sysinfo
 	echo "/" >${target_rootfs}/etc/rpm/sysinfo/Dirnames
 	if [ ! -z "$providename" ]; then
-		>>${target_rootfs}/etc/rpm/sysinfo/Providename
+		cat /dev/null > ${target_rootfs}/etc/rpm/sysinfo/Providename
 		for provide in $providename ; do
 			echo $provide >> ${target_rootfs}/etc/rpm/sysinfo/Providename
 		done
@@ -227,10 +209,10 @@ package_install_internal_rpm () {
 			for pkg in ${package_linguas}; do
 				echo "Processing $pkg..."
 
-				archvar=base_package_archs
+				archvar=base_archs
 				ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
 				if [ "${ml_pkg}" != "${pkg}" ]; then
-					archvar=ml_package_archs
+					archvar=ml_archs
 				fi
 
 				pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
@@ -238,7 +220,7 @@ package_install_internal_rpm () {
 					echo "Unable to find package $pkg ($ml_pkg)!"
 					exit 1
 				fi
-				echo $pkg_name >> ${IMAGE_ROOTFS}/install/install.manifest
+				echo $pkg_name >> ${target_rootfs}/install/install.manifest
 			done
 		fi
 	fi
@@ -247,10 +229,10 @@ package_install_internal_rpm () {
 		for pkg in ${package_to_install} ; do
 			echo "Processing $pkg..."
 
-			archvar=base_package_archs
+			archvar=base_archs
 			ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
 			if [ "${ml_pkg}" != "${pkg}" ]; then
-				archvar=ml_package_archs
+				archvar=ml_archs
 			fi
 
 			pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
@@ -258,7 +240,7 @@ package_install_internal_rpm () {
 				echo "Unable to find package $pkg ($ml_pkg)!"
 				exit 1
 			fi
-			echo $pkg_name >> ${IMAGE_ROOTFS}/install/install.manifest
+			echo $pkg_name >> ${target_rootfs}/install/install.manifest
 		done
 	fi
 
@@ -303,7 +285,7 @@ package_install_internal_rpm () {
 		# Dump the full set of recommends...
 		${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
 			--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-			-D "_dbpath ${IMAGE_ROOTFS}/install" -D "`cat ${confbase}.macro`" \
+			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
 			-D "__dbi_txn create nofsync private" \
 			-qa --qf "[%{RECOMMENDS}\n]" | sort -u > ${target_rootfs}/install/recommend
 		# Did we add more to the list?
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index dfe4612..4f627a5 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -52,12 +52,16 @@ fakeroot rootfs_rpm_do_rootfs () {
 
 	#createrepo "${DEPLOY_DIR_RPM}"
 
+	# install packages
+	# This needs to work in the same way as populate_sdk_rpm.bbclass!
+	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
+
 	# Setup base system configuration
-	mkdir -p ${IMAGE_ROOTFS}/etc/rpm/
+	mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
 
-	mkdir -p ${IMAGE_ROOTFS}${rpmlibdir}
-	mkdir -p ${IMAGE_ROOTFS}${rpmlibdir}/log
-	cat > ${IMAGE_ROOTFS}${rpmlibdir}/DB_CONFIG << EOF
+	mkdir -p ${INSTALL_ROOTFS_RPM}${rpmlibdir}
+	mkdir -p ${INSTALL_ROOTFS_RPM}${rpmlibdir}/log
+	cat > ${INSTALL_ROOTFS_RPM}${rpmlibdir}/DB_CONFIG << EOF
 # ================ Environment
 set_data_dir            .
 set_create_dir          .
@@ -81,15 +85,13 @@ mutex_set_max           163840
 # ================ Replication
 EOF
 
-	#install pacakges
-	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
 	export INSTALL_PROVIDENAME_RPM=""
-	export INSTALL_TASK_RPM="populate_sdk"
+	export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
 
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
@@ -221,5 +223,4 @@ python () {
             ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
-
 }
-- 
1.7.3.4

[PATCH 2/5] bitbake.conf: Add SDK_PACKAGE_ARCHS

[Mark Hatle]

Add SDK_PACKAGE_ARCHS to avoid the sed operations we were previously doing
inside of the variouns populate_sdk functions and related items.

Also add documentation to populate_sdk to explain when the various functions
are expected to be doing.

Finally fix a bug in populate_sdk_rpm where the wrong value was being set,
noticed while working on this change.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/package_deb.bbclass      |    6 +----
 meta/classes/package_ipk.bbclass      |   37 +++++++++++++++-----------------
 meta/classes/package_rpm.bbclass      |   10 +--------
 meta/classes/populate_sdk.bbclass     |    9 ++++++++
 meta/classes/populate_sdk_deb.bbclass |   13 +----------
 meta/classes/populate_sdk_rpm.bbclass |   23 ++++++++------------
 meta/conf/bitbake.conf                |    1 +
 7 files changed, 39 insertions(+), 60 deletions(-)

diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 8e363f8..48aa232 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -78,14 +78,10 @@ package_update_index_deb () {
 		return
 	fi
 
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
+	for arch in ${PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}; do
 		if [ -e ${DEPLOY_DIR_DEB}/$arch ]; then
 			debarchs="$debarchs $arch"
 		fi
-		if [ -e ${DEPLOY_DIR_DEB}/$sdkarch-nativesdk ]; then
-			debarchs="$debarchs $sdkarch-nativesdk"
-		fi
 	done
 
 	for arch in $debarchs; do
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 16574e9..3768889 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -161,7 +161,7 @@ ipk_log_check() {
 package_update_index_ipk () {
 	set -x
 
-	ipkgarchs="${PACKAGE_ARCHS}"
+	ipkgarchs="${PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}"
 
 	if [ ! -z "${DEPLOY_KEEP_PACKAGES}" ]; then
 		return
@@ -169,8 +169,7 @@ package_update_index_ipk () {
 
 	packagedirs="${DEPLOY_DIR_IPK}"
 	for arch in $ipkgarchs; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch ${DEPLOY_DIR_IPK}/$sdkarch-nativesdk"
+		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
 
 	multilib_archs="${MULTILIB_ARCHS}"
@@ -193,21 +192,20 @@ package_update_index_ipk () {
 #
 package_generate_ipkg_conf () {
 	package_generate_archlist
-	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_SDK}
+	ipkgarchs="${SDK_PACKAGE_ARCHS}"
+	for arch in $ipkgarchs; do
+		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
+		fi
+	done
+
+	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
 	ipkgarchs="${PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
 		fi
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension=-nativesdk
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		if [ -e ${DEPLOY_DIR_IPK}/$sdkarch$extension/Packages ] ; then
-		        echo "src oe-$sdkarch$extension file:${DEPLOY_DIR_IPK}/$sdkarch$extension" >> ${IPKGCONF_SDK}
-		fi
 	done
 
 	multilib_archs="${MULTILIB_ARCHS}"
@@ -219,16 +217,17 @@ package_generate_ipkg_conf () {
 }
 
 package_generate_archlist () {
+	ipkgarchs="${SDK_PACKAGE_ARCHS}"
+	priority=1
+	for arch in $ipkgarchs; do
+		echo "arch $arch $priority" >> ${IPKGCONF_SDK}
+		priority=$(expr $priority + 5)
+	done
+
 	ipkgarchs="${PACKAGE_ARCHS}"
 	priority=1
 	for arch in $ipkgarchs; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
 		echo "arch $arch $priority" >> ${IPKGCONF_TARGET}
-		extension=-nativesdk
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		echo "arch $sdkarch$extension $priority" >> ${IPKGCONF_SDK}
 		priority=$(expr $priority + 5)
 	done
 
@@ -237,8 +236,6 @@ package_generate_archlist () {
 		echo "arch $arch $priority" >> ${IPKGCONF_TARGET}
 		priority=$(expr $priority + 5)
 	done
-
-
 }
 
 python do_package_ipk () {
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 0f16613..abedc68 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -31,15 +31,7 @@ package_update_index_rpm () {
 	package_update_index_rpm_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
 
 	# Update SDK packages
-	base_archs=""
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension="-nativesdk"
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		base_archs="$base_archs $sdkarch$extension"
-	done
+	base_archs="${SDK_PACKAGE_ARCHS}"
 	package_update_index_rpm_common "${RPMCONF_HOST_BASE}" base_archs
 }
 
diff --git a/meta/classes/populate_sdk.bbclass b/meta/classes/populate_sdk.bbclass
index 089ed9a..0f3591b 100644
--- a/meta/classes/populate_sdk.bbclass
+++ b/meta/classes/populate_sdk.bbclass
@@ -22,6 +22,15 @@ fakeroot do_populate_sdk() {
 	rm -rf ${SDK_OUTPUT}
 	mkdir -p ${SDK_OUTPUT}
 
+	# populate_sdk_<image> is required to construct two images:
+	#  SDK_ARCH-nativesdk - contains the cross compiler and associated tooling
+	#  target - contains a target rootfs configured for the SDK usage
+	#
+	# the output of populate_sdk_<image> should end up in ${SDK_OUTPUT} it is made
+	# up of:
+	#  ${SDK_OUTPUT}/<sdk_arch-nativesdk pkgs>
+	#  ${SDK_OUTPUT}/${SDKTARGETSYSROOT}/<target pkgs>
+
 	populate_sdk_${IMAGE_PKGTYPE}
 
 	# Don't ship any libGL in the SDK
diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass
index a5b6384..25d90c7 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -34,18 +34,7 @@ fakeroot populate_sdk_deb () {
 	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}"
 	export INSTALL_BASEARCH_DEB="${SDK_ARCH}"
 	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
-	INSTALL_ARCHS_DEB=""
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension="-nativesdk"
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		if [ -e ${DEPLOY_DIR_DEB}/$sdkarch$extension ]; then
-			INSTALL_ARCHS_DEB="$INSTALL_ARCHS_DEB $sdkarch$extension"
-		fi
-	done
-	export INSTALL_ARCHS_DEB
+	export INSTALL_ARCHS_DEB="${SDK_PACKAGE_ARCHS}"
 
 	package_install_internal_deb
 	populate_sdk_post_deb ${SDK_OUTPUT}/${SDKPATHNATIVE}
diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass
index da308c1..560e40c 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -34,16 +34,16 @@ fakeroot populate_sdk_rpm () {
 
 	#install target
 	export INSTALL_ROOTFS_RPM="${SDK_OUTPUT}/${SDKTARGETSYSROOT}"
-	export INSTALL_PLATFORM_RPM="${TARGET_ARCHS}"
+	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${TOOLCHAIN_TARGET_TASK}"
 
 	# List must be prefered to least preferred order
-	INSTALL_PLATFORM_RPM=""
+	INSTALL_PLATFORM_EXTRA_RPM=""
 	for each_arch in ${PACKAGE_ARCHS} ; do
-		INSTALL_PLATFORM_RPM="$each_arch $INSTALL_PLATFORM_RPM"
+		INSTALL_PLATFORM_EXTRA_RPM="$each_arch $INSTALL_PLATFORM_EXTRA_RPM"
 	done
-	export INSTALL_PLATFORM_RPM
+	export INSTALL_PLATFORM_EXTRA_RPM
 
 	package_install_internal_rpm
 	populate_sdk_post_rpm ${INSTALL_ROOTFS_RPM}
@@ -53,16 +53,11 @@ fakeroot populate_sdk_rpm () {
 	export INSTALL_PLATFORM_RPM="${SDK_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_HOST_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${TOOLCHAIN_HOST_TASK}"
-	export INSTALL_PLATFORM_EXTRA_RPM=""
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension="-nativesdk"
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		if [ -e ${DEPLOY_DIR_RPM}/$sdkarch$extension ]; then
-			INSTALL_PLATFORM_EXTRA_RPM="$sdkarch $INSTALL_PLATFORM_EXTRA_RPM"
-		fi
+
+	# List must be prefered to least preferred order
+	INSTALL_PLATFORM_EXTRA_RPM=""
+	for each_arch in ${SDK_PACKAGE_ARCHS} ; do
+		INSTALL_PLATFORM_EXTRA_RPM="$each_arch $INSTALL_PLATFORM_EXTRA_RPM"
 	done
 	export INSTALL_PLATFORM_EXTRA_RPM
 
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 9c80f11..ac5d1f3 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -100,6 +100,7 @@ SDK_VENDOR = "-oesdk"
 SDK_SYS = "${SDK_ARCH}${SDK_VENDOR}${@['-' + bb.data.getVar('SDK_OS', d, 1), ''][bb.data.getVar('SDK_OS', d, 1) == ('' or 'custom')]}"
 SDK_PREFIX = "${SDK_SYS}-"
 SDK_CC_ARCH = "${BUILD_CC_ARCH}"
+SDK_PACKAGE_ARCHS = "all any noarch ${SDK_ARCH}-nativesdk"
 
 PACKAGE_ARCH = "${TUNE_PKGARCH}"
 MACHINE_ARCH = "${@[bb.data.getVar('TUNE_PKGARCH', d, 1), bb.data.getVar('MACHINE', d, 1)][bool(bb.data.getVar('MACHINE', d, 1))].replace('-', '_')}"
-- 
1.7.3.4

[PATCH 3/5] populate_sdk_*: Sync SDK and regular rootfs functions

[Mark Hatle]

In order for things to be easier to maintain in the future, sync up
the sdk and rootfs versions.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/populate_sdk_deb.bbclass |   24 ++++++++----
 meta/classes/populate_sdk_rpm.bbclass |   68 ++++++++++++++++++++++++++++----
 meta/classes/rootfs_rpm.bbclass       |   15 +++----
 3 files changed, 82 insertions(+), 25 deletions(-)

diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass
index 25d90c7..1bf1b95 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -11,30 +11,38 @@ populate_sdk_post_deb () {
 
 fakeroot populate_sdk_deb () {
 
+	# update index
 	package_update_index_deb
 
-	export INSTALL_TASK_DEB="populate_sdk"
-	export INSTALL_PACKAGES_LINGUAS_DEB=""
-	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
-
-	#install target
+	## install target ##
+	# This needs to work in the same way as rootfs_deb.bbclass
 	echo "Installing TARGET packages"
+
+	mkdir -p ${IMAGE_ROOTFS}/var/dpkg/alternatives
+
 	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}/${SDKTARGETSYSROOT}"
 	export INSTALL_BASEARCH_DEB="${DPKG_ARCH}"
 	export INSTALL_ARCHS_DEB="${PACKAGE_ARCHS}"
 	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_TARGET_TASK}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
+	export PACKAGES_LINGUAS_DEB=""
+	export INSTALL_TASK_DEB="populate_sdk-target"
 
 	package_install_internal_deb
+
 	populate_sdk_post_deb ${INSTALL_ROOTFS_DEB}
 
 	populate_sdk_log_check populate_sdk
 
-	#install host
-	echo "Installing HOST packages"
+	## install nativesdk ##
+	echo "Installing NATIVESDK packages"
 	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}"
 	export INSTALL_BASEARCH_DEB="${SDK_ARCH}"
-	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
 	export INSTALL_ARCHS_DEB="${SDK_PACKAGE_ARCHS}"
+	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
+	export PACKAGES_LINGUAS_DEB=""
+	export INSTALL_TASK_DEB="populate_sdk-nativesdk"
 
 	package_install_internal_deb
 	populate_sdk_post_deb ${SDK_OUTPUT}/${SDKPATHNATIVE}
diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass
index 560e40c..5fb0234 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -26,21 +26,49 @@ fakeroot populate_sdk_rpm () {
 	package_update_index_rpm
 	package_generate_rpm_conf
 
-	export INSTALL_PACKAGES_ATTEMPTONLY_RPM=""
-	export INSTALL_PACKAGES_LINGUAS_RPM=""
-	export INSTALL_PROVIDENAME_RPM="/bin/sh"
-	export INSTALL_TASK_RPM="populate_sdk"
-
-
-	#install target
+	## install target ##
+	# This needs to work in the same way as rootfs_rpm.bbclass!
+	#
 	export INSTALL_ROOTFS_RPM="${SDK_OUTPUT}/${SDKTARGETSYSROOT}"
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${TOOLCHAIN_TARGET_TASK}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_RPM=""
+	export INSTALL_PACKAGES_LINGUAS_RPM=""
+	export INSTALL_PROVIDENAME_RPM=""
+	export INSTALL_TASK_RPM="populate_sdk-target"
+
+	# Setup base system configuration
+	mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
+	mkdir -p ${INSTALL_ROOTFS_RPM}${rpmlibdir}
+	mkdir -p ${INSTALL_ROOTFS_RPM}${rpmlibdir}/log
+	cat > ${INSTALL_ROOTFS_RPM}${rpmlibdir}/DB_CONFIG << EOF
+# ================ Environment
+set_data_dir            .
+set_create_dir          .
+set_lg_dir              ./log
+set_tmp_dir             ./tmp
+
+# -- thread_count must be >= 8
+set_thread_count        64
+
+# ================ Logging
+
+# ================ Memory Pool
+set_mp_mmapsize         268435456
+
+# ================ Locking
+set_lk_max_locks        16384
+set_lk_max_lockers      16384
+set_lk_max_objects      16384
+mutex_set_max           163840
+
+# ================ Replication
+EOF
 
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
-	for each_arch in ${PACKAGE_ARCHS} ; do
+	for each_arch in ${MULTILIB_PACKAGE_ARCHS} ${PACKAGE_ARCHS} ; do
 		INSTALL_PLATFORM_EXTRA_RPM="$each_arch $INSTALL_PLATFORM_EXTRA_RPM"
 	done
 	export INSTALL_PLATFORM_EXTRA_RPM
@@ -48,11 +76,16 @@ fakeroot populate_sdk_rpm () {
 	package_install_internal_rpm
 	populate_sdk_post_rpm ${INSTALL_ROOTFS_RPM}
 
-	#install host
+	## install nativesdk ##
+	echo "Installing NATIVESDK packages"
 	export INSTALL_ROOTFS_RPM="${SDK_OUTPUT}"
 	export INSTALL_PLATFORM_RPM="${SDK_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_HOST_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${TOOLCHAIN_HOST_TASK}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_RPM=""
+	export INSTALL_PACKAGES_LINGUAS_RPM=""
+	export INSTALL_PROVIDENAME_RPM="/bin/sh"
+	export INSTALL_TASK_RPM="populate_sdk_rpm-nativesdk"
 
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
@@ -80,3 +113,20 @@ fakeroot populate_sdk_rpm () {
 		resolve_package_rpm foo ${RPMCONF_TARGET_BASE}.conf || true
 	fi
 }
+
+python () {
+    ml_package_archs = ""
+    multilibs = d.getVar('MULTILIBS', True) or ""
+    for ext in multilibs.split():
+        eext = ext.split(':')
+        if len(eext) > 1 and eext[0] == 'multilib':
+            localdata = bb.data.createCopy(d)
+            overrides = localdata.getVar("OVERRIDES", False) + ":virtclass-multilib-" + eext[1]
+            localdata.setVar("OVERRIDES", overrides)
+            # TEMP: OVERRIDES isn't working right
+            localdata.setVar("DEFAULTTUNE", localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False) or "")
+            ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
+            #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
+    bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
+}
+
diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index 4f627a5..7c06064 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -55,6 +55,13 @@ fakeroot rootfs_rpm_do_rootfs () {
 	# install packages
 	# This needs to work in the same way as populate_sdk_rpm.bbclass!
 	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
+	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
+	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
+	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
+	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
+	export INSTALL_PROVIDENAME_RPM=""
+	export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
 
 	# Setup base system configuration
 	mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
@@ -85,14 +92,6 @@ mutex_set_max           163840
 # ================ Replication
 EOF
 
-	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
-	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
-	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
-	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
-	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
-	export INSTALL_PROVIDENAME_RPM=""
-	export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
-
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
 	for each_arch in ${MULTILIB_PACKAGE_ARCHS} ${PACKAGE_ARCHS}; do
-- 
1.7.3.4

[PATCH 4/5] package_ipk: SDK generation workaround

[Mark Hatle]

During SDK generation the multilib_sanity_check function is being called,
however it is not available unless we've been called from the image.bbclass.

Disable the check if MULTILIB_CHECK_FILE (also set in image.bbclass) doesn't
exist or is empty.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/package_ipk.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 3768889..8ecf511 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -127,8 +127,10 @@ package_install_internal_ipk() {
 	fi
 
 	package_tryout_install_multilib_ipk
-	#sanity check
-	multilib_sanity_check  ${target_rootfs} ${multilib_tryout_dirs}|| exit 1
+	if [ ! -z "${MULTILIB_CHECK_FILE}" ]; then
+		#sanity check
+		multilib_sanity_check ${target_rootfs} ${multilib_tryout_dirs} || exit 1
+	fi
 
 	if [ ! -z "${package_multilib}" ]; then
 		opkg-cl ${ipkg_args} install ${package_multilib}
-- 
1.7.3.4

[PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Mark Hatle]

The following allows RPM to generate the SDK image, however without it
we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because
they can not have filenames within their RPROVIDES.  I'm looking for some
type of a resolution to the issue, the only thing I can think of is to
add a way to manually add a FILERPROVIDE for the items.  This will require
changes to the way FILERPROVIDE is currently generated... but I'm not sure
how we can automatically generate the FILERPROVIDE values without the use of
python...

Any suggestions?

-----

Add the automatic alternatives as runtime provides.

This was discovered when programers were requiring "/bin/sh" and there
was nothing within the system providing "/bin/sh", however bash had an
alternative for /bin/sh that would satisfy this dependency.

NOTE: THIS BREAKS IPK AND DEB PACKAGE GENERATION!  IT SHOULD NOT BE
MERGED!

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/classes/update-alternatives.bbclass |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/meta/classes/update-alternatives.bbclass b/meta/classes/update-alternatives.bbclass
index ba81221..4cd680e 100644
--- a/meta/classes/update-alternatives.bbclass
+++ b/meta/classes/update-alternatives.bbclass
@@ -40,6 +40,8 @@
 ALTERNATIVE_PRIORITY = "10"
 ALTERNATIVE_LINK = "${bindir}/${ALTERNATIVE_NAME}"
 
+RPROVIDES_${PN} += "${@(d.getVar('ALTERNATIVE_LINK', True) or '') + ' ' + (d.getVar('ALTERNATIVE_LINKS', True) or '')}"
+
 update_alternatives_postinst() {
 update-alternatives --install ${ALTERNATIVE_LINK} ${ALTERNATIVE_NAME} ${ALTERNATIVE_PATH} ${ALTERNATIVE_PRIORITY}
 }
-- 
1.7.3.4

Re: [PATCH 0/5] Fix the SDK generation

[Richard Purdie]

On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
> This is an attempt to fix a number of issues with the SDK generation.  It has been
> tested using bitbake meta-toolchain-sdk.
> 
> Note, patches:
> 
>  4/5: package_ipk: SDK generation workaround
>  5/5: update-alternatives: Add alternatives as a runtime provide
> 
> 4/5 might be reasonable to implement immediately to resolve this issue.. However
> 5/5 is absolutely incorrect and that is noted within the patch header.  I'd
> like comments on the issue to understand how to fix the problem.

I've merged 1-4 to master, 4 on the grounds that it improves the
situation substantially.

For 5, I suspect we'll need to process the path using python, I can't
really see a way to avoid it. I assume / is an illegal character for deb
anywhere, not just at the start of the dependency name?

Cheers,

Richard

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Phil Blundell]

On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
> The following allows RPM to generate the SDK image, however without it
> we get a failure because the system has nothing that provides /bin/sh.
> 
> Unfortunately the patch causes failures with ipk and deb packages because
> they can not have filenames within their RPROVIDES.  I'm looking for some
> type of a resolution to the issue, the only thing I can think of is to
> add a way to manually add a FILERPROVIDE for the items.  This will require
> changes to the way FILERPROVIDE is currently generated... but I'm not sure
> how we can automatically generate the FILERPROVIDE values without the use of
> python...
> 
> Any suggestions?

It's never really been the intent that update-alternatives should put
the name of the link being provided into RPROVIDES.  If you want to
solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
+= "virtual-bourne-shell" or something to bash and busybox is probably
the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing
here, though.  Something like:

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
[ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
(d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps.  I'm not sure that the resulting
virtual names will be very pretty though.

p.

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Mark Hatle]

On 8/2/11 8:46 AM, Phil Blundell wrote:
> On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
>> The following allows RPM to generate the SDK image, however without it
>> we get a failure because the system has nothing that provides /bin/sh.
>>
>> Unfortunately the patch causes failures with ipk and deb packages because
>> they can not have filenames within their RPROVIDES.  I'm looking for some
>> type of a resolution to the issue, the only thing I can think of is to
>> add a way to manually add a FILERPROVIDE for the items.  This will require
>> changes to the way FILERPROVIDE is currently generated... but I'm not sure
>> how we can automatically generate the FILERPROVIDE values without the use of
>> python...
>>
>> Any suggestions?
> 
> It's never really been the intent that update-alternatives should put
> the name of the link being provided into RPROVIDES.  If you want to
> solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
> += "virtual-bourne-shell" or something to bash and busybox is probably
> the easiest way of doing that.
> 
> I wouldn't be entirely opposed to the concept of what you're proposing
> here, though.  Something like:
> 
> RPROVIDES_${PN} += "${@' '.join(map(lambda x:
> legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
> [ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
> (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"
> 
> might be what you want, perhaps.  I'm not sure that the resulting
> virtual names will be very pretty though.

Hmm.. Coming from the RPM world, the virtual-path- because we can't just
"provide" a file in the system seems a bit wonky to me.  But it should be able
to work.  For RPM at least, we'd want a reversing function to turn virtual paths
back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can
get it to work reasonably well.

--Mark

> p.
> 
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Richard Purdie]

On Tue, 2011-08-02 at 09:49 -0500, Mark Hatle wrote:
> On 8/2/11 8:46 AM, Phil Blundell wrote:
> > On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
> >> The following allows RPM to generate the SDK image, however without it
> >> we get a failure because the system has nothing that provides /bin/sh.
> >>
> >> Unfortunately the patch causes failures with ipk and deb packages because
> >> they can not have filenames within their RPROVIDES.  I'm looking for some
> >> type of a resolution to the issue, the only thing I can think of is to
> >> add a way to manually add a FILERPROVIDE for the items.  This will require
> >> changes to the way FILERPROVIDE is currently generated... but I'm not sure
> >> how we can automatically generate the FILERPROVIDE values without the use of
> >> python...
> >>
> >> Any suggestions?
> > 
> > It's never really been the intent that update-alternatives should put
> > the name of the link being provided into RPROVIDES.  If you want to
> > solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
> > += "virtual-bourne-shell" or something to bash and busybox is probably
> > the easiest way of doing that.
> > 
> > I wouldn't be entirely opposed to the concept of what you're proposing
> > here, though.  Something like:
> > 
> > RPROVIDES_${PN} += "${@' '.join(map(lambda x:
> > legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
> > [ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
> > (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"
> > 
> > might be what you want, perhaps.  I'm not sure that the resulting
> > virtual names will be very pretty though.
> 
> Hmm.. Coming from the RPM world, the virtual-path- because we can't just
> "provide" a file in the system seems a bit wonky to me.  But it should be able
> to work.  For RPM at least, we'd want a reversing function to turn virtual paths
> back into real paths.
> 
> If I have time today, I'll try to implement a proof of concept and see if I can
> get it to work reasonably well.

Just to be clear for Phil's benefit, RPM natively supports file based
dependencies, so a dependency of "/bin/sh" is automatically fulfilled by
a package which contains "/bin/sh". Some dependencies such as the
shebang in scripts are automatically added to packages and resolved by
rpm.

I did chase down this bug a bit and it seems that if you "bitbake
meta-toolchain-game" you hit an error about /bin/sh being missing but if
you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is
due to busybox shipping a /bin/sh.

The question is therefore how to handle this on the deb/ipk side and
ensure we get consistency between the behaviours of the different
backends. I thought with the rpm filedeps code in do_package, we were
adding things like /bin/sh dependencies to the other package formats but
now I'm not so sure. 

Cheers,

Richard

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Mark Hatle]

On 8/3/11 7:20 AM, Richard Purdie wrote:
> On Tue, 2011-08-02 at 09:49 -0500, Mark Hatle wrote:
>> On 8/2/11 8:46 AM, Phil Blundell wrote:
>>> On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
>>>> The following allows RPM to generate the SDK image, however without it
>>>> we get a failure because the system has nothing that provides /bin/sh.
>>>>
>>>> Unfortunately the patch causes failures with ipk and deb packages because
>>>> they can not have filenames within their RPROVIDES.  I'm looking for some
>>>> type of a resolution to the issue, the only thing I can think of is to
>>>> add a way to manually add a FILERPROVIDE for the items.  This will require
>>>> changes to the way FILERPROVIDE is currently generated... but I'm not sure
>>>> how we can automatically generate the FILERPROVIDE values without the use of
>>>> python...
>>>>
>>>> Any suggestions?
>>>
>>> It's never really been the intent that update-alternatives should put
>>> the name of the link being provided into RPROVIDES.  If you want to
>>> solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
>>> += "virtual-bourne-shell" or something to bash and busybox is probably
>>> the easiest way of doing that.
>>>
>>> I wouldn't be entirely opposed to the concept of what you're proposing
>>> here, though.  Something like:
>>>
>>> RPROVIDES_${PN} += "${@' '.join(map(lambda x:
>>> legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
>>> [ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
>>> (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"
>>>
>>> might be what you want, perhaps.  I'm not sure that the resulting
>>> virtual names will be very pretty though.
>>
>> Hmm.. Coming from the RPM world, the virtual-path- because we can't just
>> "provide" a file in the system seems a bit wonky to me.  But it should be able
>> to work.  For RPM at least, we'd want a reversing function to turn virtual paths
>> back into real paths.
>>
>> If I have time today, I'll try to implement a proof of concept and see if I can
>> get it to work reasonably well.
> 
> Just to be clear for Phil's benefit, RPM natively supports file based
> dependencies, so a dependency of "/bin/sh" is automatically fulfilled by
> a package which contains "/bin/sh". Some dependencies such as the
> shebang in scripts are automatically added to packages and resolved by
> rpm.
> 
> I did chase down this bug a bit and it seems that if you "bitbake
> meta-toolchain-game" you hit an error about /bin/sh being missing but if
> you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is
> due to busybox shipping a /bin/sh.
> 
> The question is therefore how to handle this on the deb/ipk side and
> ensure we get consistency between the behaviours of the different
> backends. I thought with the rpm filedeps code in do_package, we were
> adding things like /bin/sh dependencies to the other package formats but
> now I'm not so sure. 

Due to deb/ipk not handling file based dependencies, they are filtered out on
the creation of the deb/ipk packages.  The original intention was to use them,
but it wasn't possible at the time.  Simply adding a ton of file-based
dependencies seemed like a huge mistake as well.  (We'd have to add virtual
provides for all of the virtual requirements....)

We could certainly select a few specific requirements and scan for and use those
to catch obvious issues, such as perl, sh, bash, env/python... but it's still
only a partial solution to the real issue.

--Mark

> Cheers,
> 
> Richard
> 
> 
> 
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Richard Purdie]

On Wed, 2011-08-03 at 09:41 -0500, Mark Hatle wrote:
> On 8/3/11 7:20 AM, Richard Purdie wrote:
> > On Tue, 2011-08-02 at 09:49 -0500, Mark Hatle wrote:
> >> On 8/2/11 8:46 AM, Phil Blundell wrote:
> >>> On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
> >>>> The following allows RPM to generate the SDK image, however without it
> >>>> we get a failure because the system has nothing that provides /bin/sh.
> >>>>
> >>>> Unfortunately the patch causes failures with ipk and deb packages because
> >>>> they can not have filenames within their RPROVIDES.  I'm looking for some
> >>>> type of a resolution to the issue, the only thing I can think of is to
> >>>> add a way to manually add a FILERPROVIDE for the items.  This will require
> >>>> changes to the way FILERPROVIDE is currently generated... but I'm not sure
> >>>> how we can automatically generate the FILERPROVIDE values without the use of
> >>>> python...
> >>>>
> >>>> Any suggestions?
> >>>
> >>> It's never really been the intent that update-alternatives should put
> >>> the name of the link being provided into RPROVIDES.  If you want to
> >>> solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
> >>> += "virtual-bourne-shell" or something to bash and busybox is probably
> >>> the easiest way of doing that.
> >>>
> >>> I wouldn't be entirely opposed to the concept of what you're proposing
> >>> here, though.  Something like:
> >>>
> >>> RPROVIDES_${PN} += "${@' '.join(map(lambda x:
> >>> legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
> >>> [ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
> >>> (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"
> >>>
> >>> might be what you want, perhaps.  I'm not sure that the resulting
> >>> virtual names will be very pretty though.
> >>
> >> Hmm.. Coming from the RPM world, the virtual-path- because we can't just
> >> "provide" a file in the system seems a bit wonky to me.  But it should be able
> >> to work.  For RPM at least, we'd want a reversing function to turn virtual paths
> >> back into real paths.
> >>
> >> If I have time today, I'll try to implement a proof of concept and see if I can
> >> get it to work reasonably well.
> > 
> > Just to be clear for Phil's benefit, RPM natively supports file based
> > dependencies, so a dependency of "/bin/sh" is automatically fulfilled by
> > a package which contains "/bin/sh". Some dependencies such as the
> > shebang in scripts are automatically added to packages and resolved by
> > rpm.
> > 
> > I did chase down this bug a bit and it seems that if you "bitbake
> > meta-toolchain-game" you hit an error about /bin/sh being missing but if
> > you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is
> > due to busybox shipping a /bin/sh.
> > 
> > The question is therefore how to handle this on the deb/ipk side and
> > ensure we get consistency between the behaviours of the different
> > backends. I thought with the rpm filedeps code in do_package, we were
> > adding things like /bin/sh dependencies to the other package formats but
> > now I'm not so sure. 
> 
> Due to deb/ipk not handling file based dependencies, they are filtered out on
> the creation of the deb/ipk packages.  The original intention was to use them,
> but it wasn't possible at the time.  Simply adding a ton of file-based
> dependencies seemed like a huge mistake as well.  (We'd have to add virtual
> provides for all of the virtual requirements....)
> 
> We could certainly select a few specific requirements and scan for and use those
> to catch obvious issues, such as perl, sh, bash, env/python... but it's still
> only a partial solution to the real issue.

Short term I'm tempted to buy us some time and do this (rpm specific):

package_rpm: Ensure alternatives links are reflected in rpm package dependencies

Currently, if a file is provided as an alternative link within the package, rpm
doesn't see the dependency. This works out badly for dependencies such as /bin/sh
which scripts might require.

Since rpm detects and adds these dependencies we do need to ensure the dependency
information in the packages is correct. This patch does so for the rpm backend
ensuring internal consistency whilst the approach for addressing this problem in
the other package backends is considered.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index abedc68..c44fdcc 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -501,6 +501,10 @@ python write_specfile () {
 		splitrconflicts  = bb.data.getVar('RCONFLICTS', localdata, True) or ""
 		splitrobsoletes  = []
 
+		# For now we need to manually supplement RPROVIDES with any update-alternatives links
+		if pkg == d.getVar("PN", True):
+			splitrprovides = splitrprovides + (d.getVar('ALTERNATIVE_LINK', True) or '') + (d.getVar('ALTERNATIVE_LINKS', True) or '')
+
 		# Gather special src/first package data
 		if srcname == splitname:
 			srcrdepends    = splitrdepends

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Mark Hatle]

On 8/3/11 10:40 AM, Richard Purdie wrote:
> On Wed, 2011-08-03 at 09:41 -0500, Mark Hatle wrote:
>> On 8/3/11 7:20 AM, Richard Purdie wrote:
>>> On Tue, 2011-08-02 at 09:49 -0500, Mark Hatle wrote:
>>>> On 8/2/11 8:46 AM, Phil Blundell wrote:
>>>>> On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
>>>>>> The following allows RPM to generate the SDK image, however without it
>>>>>> we get a failure because the system has nothing that provides /bin/sh.
>>>>>>
>>>>>> Unfortunately the patch causes failures with ipk and deb packages because
>>>>>> they can not have filenames within their RPROVIDES.  I'm looking for some
>>>>>> type of a resolution to the issue, the only thing I can think of is to
>>>>>> add a way to manually add a FILERPROVIDE for the items.  This will require
>>>>>> changes to the way FILERPROVIDE is currently generated... but I'm not sure
>>>>>> how we can automatically generate the FILERPROVIDE values without the use of
>>>>>> python...
>>>>>>
>>>>>> Any suggestions?
>>>>>
>>>>> It's never really been the intent that update-alternatives should put
>>>>> the name of the link being provided into RPROVIDES.  If you want to
>>>>> solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
>>>>> += "virtual-bourne-shell" or something to bash and busybox is probably
>>>>> the easiest way of doing that.
>>>>>
>>>>> I wouldn't be entirely opposed to the concept of what you're proposing
>>>>> here, though.  Something like:
>>>>>
>>>>> RPROVIDES_${PN} += "${@' '.join(map(lambda x:
>>>>> legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
>>>>> [ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
>>>>> (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"
>>>>>
>>>>> might be what you want, perhaps.  I'm not sure that the resulting
>>>>> virtual names will be very pretty though.
>>>>
>>>> Hmm.. Coming from the RPM world, the virtual-path- because we can't just
>>>> "provide" a file in the system seems a bit wonky to me.  But it should be able
>>>> to work.  For RPM at least, we'd want a reversing function to turn virtual paths
>>>> back into real paths.
>>>>
>>>> If I have time today, I'll try to implement a proof of concept and see if I can
>>>> get it to work reasonably well.
>>>
>>> Just to be clear for Phil's benefit, RPM natively supports file based
>>> dependencies, so a dependency of "/bin/sh" is automatically fulfilled by
>>> a package which contains "/bin/sh". Some dependencies such as the
>>> shebang in scripts are automatically added to packages and resolved by
>>> rpm.
>>>
>>> I did chase down this bug a bit and it seems that if you "bitbake
>>> meta-toolchain-game" you hit an error about /bin/sh being missing but if
>>> you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is
>>> due to busybox shipping a /bin/sh.
>>>
>>> The question is therefore how to handle this on the deb/ipk side and
>>> ensure we get consistency between the behaviours of the different
>>> backends. I thought with the rpm filedeps code in do_package, we were
>>> adding things like /bin/sh dependencies to the other package formats but
>>> now I'm not so sure. 
>>
>> Due to deb/ipk not handling file based dependencies, they are filtered out on
>> the creation of the deb/ipk packages.  The original intention was to use them,
>> but it wasn't possible at the time.  Simply adding a ton of file-based
>> dependencies seemed like a huge mistake as well.  (We'd have to add virtual
>> provides for all of the virtual requirements....)
>>
>> We could certainly select a few specific requirements and scan for and use those
>> to catch obvious issues, such as perl, sh, bash, env/python... but it's still
>> only a partial solution to the real issue.
> 
> Short term I'm tempted to buy us some time and do this (rpm specific):
> 
> package_rpm: Ensure alternatives links are reflected in rpm package dependencies
> 
> Currently, if a file is provided as an alternative link within the package, rpm
> doesn't see the dependency. This works out badly for dependencies such as /bin/sh
> which scripts might require.
> 
> Since rpm detects and adds these dependencies we do need to ensure the dependency
> information in the packages is correct. This patch does so for the rpm backend
> ensuring internal consistency whilst the approach for addressing this problem in
> the other package backends is considered.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
> index abedc68..c44fdcc 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -501,6 +501,10 @@ python write_specfile () {
>  		splitrconflicts  = bb.data.getVar('RCONFLICTS', localdata, True) or ""
>  		splitrobsoletes  = []
>  
> +		# For now we need to manually supplement RPROVIDES with any update-alternatives links
> +		if pkg == d.getVar("PN", True):
> +			splitrprovides = splitrprovides + (d.getVar('ALTERNATIVE_LINK', True) or '') + (d.getVar('ALTERNATIVE_LINKS', True) or '')
> +

Will the "+" add a space in this case, otherwise we need additional spaces added.

But this is a good solution to the issue.  It's RPM specific (for now) until we
decide if we have to address the other packaging systems.

--Mark

>  		# Gather special src/first package data
>  		if srcname == splitname:
>  			srcrdepends    = splitrdepends
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

[Richard Purdie]

On Wed, 2011-08-03 at 11:03 -0500, Mark Hatle wrote:
> On 8/3/11 10:40 AM, Richard Purdie wrote:
> > On Wed, 2011-08-03 at 09:41 -0500, Mark Hatle wrote:
> >> On 8/3/11 7:20 AM, Richard Purdie wrote:
> >>> On Tue, 2011-08-02 at 09:49 -0500, Mark Hatle wrote:
> >>>> On 8/2/11 8:46 AM, Phil Blundell wrote:
> >>>>> On Mon, 2011-08-01 at 19:17 -0500, Mark Hatle wrote:
> >>>>>> The following allows RPM to generate the SDK image, however without it
> >>>>>> we get a failure because the system has nothing that provides /bin/sh.
> >>>>>>
> >>>>>> Unfortunately the patch causes failures with ipk and deb packages because
> >>>>>> they can not have filenames within their RPROVIDES.  I'm looking for some
> >>>>>> type of a resolution to the issue, the only thing I can think of is to
> >>>>>> add a way to manually add a FILERPROVIDE for the items.  This will require
> >>>>>> changes to the way FILERPROVIDE is currently generated... but I'm not sure
> >>>>>> how we can automatically generate the FILERPROVIDE values without the use of
> >>>>>> python...
> >>>>>>
> >>>>>> Any suggestions?
> >>>>>
> >>>>> It's never really been the intent that update-alternatives should put
> >>>>> the name of the link being provided into RPROVIDES.  If you want to
> >>>>> solve the specific problem with /bin/sh then just adding RPROVIDES_${PN}
> >>>>> += "virtual-bourne-shell" or something to bash and busybox is probably
> >>>>> the easiest way of doing that.
> >>>>>
> >>>>> I wouldn't be entirely opposed to the concept of what you're proposing
> >>>>> here, though.  Something like:
> >>>>>
> >>>>> RPROVIDES_${PN} += "${@' '.join(map(lambda x:
> >>>>> legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '',
> >>>>> [ d.getVar('ALTERNATIVE_LINK', True) or '' ] +
> >>>>> (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"
> >>>>>
> >>>>> might be what you want, perhaps.  I'm not sure that the resulting
> >>>>> virtual names will be very pretty though.
> >>>>
> >>>> Hmm.. Coming from the RPM world, the virtual-path- because we can't just
> >>>> "provide" a file in the system seems a bit wonky to me.  But it should be able
> >>>> to work.  For RPM at least, we'd want a reversing function to turn virtual paths
> >>>> back into real paths.
> >>>>
> >>>> If I have time today, I'll try to implement a proof of concept and see if I can
> >>>> get it to work reasonably well.
> >>>
> >>> Just to be clear for Phil's benefit, RPM natively supports file based
> >>> dependencies, so a dependency of "/bin/sh" is automatically fulfilled by
> >>> a package which contains "/bin/sh". Some dependencies such as the
> >>> shebang in scripts are automatically added to packages and resolved by
> >>> rpm.
> >>>
> >>> I did chase down this bug a bit and it seems that if you "bitbake
> >>> meta-toolchain-game" you hit an error about /bin/sh being missing but if
> >>> you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is
> >>> due to busybox shipping a /bin/sh.
> >>>
> >>> The question is therefore how to handle this on the deb/ipk side and
> >>> ensure we get consistency between the behaviours of the different
> >>> backends. I thought with the rpm filedeps code in do_package, we were
> >>> adding things like /bin/sh dependencies to the other package formats but
> >>> now I'm not so sure. 
> >>
> >> Due to deb/ipk not handling file based dependencies, they are filtered out on
> >> the creation of the deb/ipk packages.  The original intention was to use them,
> >> but it wasn't possible at the time.  Simply adding a ton of file-based
> >> dependencies seemed like a huge mistake as well.  (We'd have to add virtual
> >> provides for all of the virtual requirements....)
> >>
> >> We could certainly select a few specific requirements and scan for and use those
> >> to catch obvious issues, such as perl, sh, bash, env/python... but it's still
> >> only a partial solution to the real issue.
> > 
> > Short term I'm tempted to buy us some time and do this (rpm specific):
> > 
> > package_rpm: Ensure alternatives links are reflected in rpm package dependencies
> > 
> > Currently, if a file is provided as an alternative link within the package, rpm
> > doesn't see the dependency. This works out badly for dependencies such as /bin/sh
> > which scripts might require.
> > 
> > Since rpm detects and adds these dependencies we do need to ensure the dependency
> > information in the packages is correct. This patch does so for the rpm backend
> > ensuring internal consistency whilst the approach for addressing this problem in
> > the other package backends is considered.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
> > index abedc68..c44fdcc 100644
> > --- a/meta/classes/package_rpm.bbclass
> > +++ b/meta/classes/package_rpm.bbclass
> > @@ -501,6 +501,10 @@ python write_specfile () {
> >  		splitrconflicts  = bb.data.getVar('RCONFLICTS', localdata, True) or ""
> >  		splitrobsoletes  = []
> >  
> > +		# For now we need to manually supplement RPROVIDES with any update-alternatives links
> > +		if pkg == d.getVar("PN", True):
> > +			splitrprovides = splitrprovides + (d.getVar('ALTERNATIVE_LINK', True) or '') + (d.getVar('ALTERNATIVE_LINKS', True) or '')
> > +
> 
> Will the "+" add a space in this case, otherwise we need additional spaces added.

It needs more spaces. I realised that after I posted it.

> But this is a good solution to the issue.  It's RPM specific (for now) until we
> decide if we have to address the other packaging systems.

I think I'll likely merge this to fix the immediate issues and we can
think about better solutions to this...

Cheers,

Richard