[PATCH 0/1][v3] Automatically generate package repos for rpmand deb [bug #1024]

[PATCH 0/1][v3] Automatically generate package repos for rpmand deb [bug #1024]

[Dexuan Cui]

From: Dexuan Cui <dexuan.cui@intel.com>

This was made to address http://bugzilla.yoctoproject.org/show_bug.cgi?id=1024.

Thanks a lot to RP and Saul's suggestions, the patch becomes much cleaner now.

As to DEB, I found currently in target, some important config files, like
/etc/apt/sources.list, for deb/apt are missing. As a result, we can't
install/remove package in target at all.
I tried to resolve this by add "deb http://192.168.7.1/deb/all/ ./" into
target's /etc/apt/sources.list, but later "apt-get update" failes -- it
tries to find a file named Packages.gz, but the direcory only has
Packages.bz2 -- not sure why apt-get here doesn't try to find Packages.bz2
when it can't find Packages.gz and do_rootfs does succeed with Packages.bz2.
Later after I tried to manually generate Packages.gz from Packages.bz2 and did
"mkdir -p /var/cache/apt/archives/partial/; mkdir -p /var/log/apt/",
"apt-get update" and "apt-get remove" finally can work, but "apt-get install"
could still fail: "package architecture (arm) does not match system (armel)".
The issues need more effort to be fixed.

Pull URL: git://git.pokylinux.org/poky-contrib.git
  Branch: dcui/master
  Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dcui/master

Thanks,
    Dexuan Cui <dexuan.cui@intel.com>
---


Dexuan Cui (1):
  package-index.bb: add support for deb and rpm.

 meta/classes/package_deb.bbclass        |    9 +++++++++
 meta/classes/package_ipk.bbclass        |    9 +++++++++
 meta/classes/package_rpm.bbclass        |    9 +++++++++
 meta/recipes-core/meta/package-index.bb |    6 ++----
 4 files changed, 29 insertions(+), 4 deletions(-)

-- 
1.7.2

[PATCH 1/1][RFC][v3] package-index.bb: add support for rpm and deb.

[Dexuan Cui]

From: Dexuan Cui <dexuan.cui@intel.com>

[YOCTO #1024]
Currently package-index.bb only supports ipk. This commit adds the support
for rpm and deb, too.

------------------------------
How to generate and use repos:

1) run "bitbake package-index" after building some target,
e.g., core-image-sato-sdk;

2) export ${DEPLOY_DIR_RPM}, ${DEPLOY_DIR_IPK} and ${DEPLOY_DIR_DEB} by a
webserver on the host, assuming the host IP is 192.168.7.1, at
http://192.168.7.1/rpm
http://192.168.7.1/ipk
http://192.168.7.1/deb

3) inside the target, according to the packaging system (rpm, ipk or deb) used
when we generate the target image, we can use different ways to manage
packages:

3.1) RPM
    run "zypper addrepo http://192.168.7.1/rpm main; zypper refresh"
    to retrieve info about the repo; next, we can use "zypper install/remove"
    to manage packages.

3.2) IPK
    add the repo info into opkg config file, i.e., in
    /etc/opkg/arch.conf, we can add something like
    "src i586 http://192.168.7.1/ipk/i586", and next, we run "opkg update" to
    make opkg update the list of available packages. And later, we can use
    "opkg install/remove" to manage packages.

3.3) DEB
    Currently in target, some important config files, like
/var/lib/dpkg/status and /etc/apt/sources.list, for deb/apt are missing. So
we can't install/remove package in target at present.

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/classes/package_deb.bbclass        |    9 +++++++++
 meta/classes/package_ipk.bbclass        |    9 +++++++++
 meta/classes/package_rpm.bbclass        |    9 +++++++++
 meta/recipes-core/meta/package-index.bb |    6 ++----
 4 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 4faeb4a..f9ef32c 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -431,3 +431,12 @@ python do_package_write_deb () {
 do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}"
 addtask package_write_deb before do_package_write after do_package
 
+
+PACKAGEINDEXES += "package_index_deb;"
+PACKAGEINDEXDEPS += "dpkg-native:do_populate_sysroot"
+PACKAGEINDEXDEPS += "apt-native:do_populate_sysroot"
+
+package_index_deb() {
+    package_update_index_deb
+}
+
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 3c2472b..97de17a 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -386,3 +386,12 @@ python do_package_write_ipk () {
 }
 do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}"
 addtask package_write_ipk before do_package_write after do_package
+
+PACKAGEINDEXES += "package_index_ipk;"
+PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot"
+PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot"
+
+package_index_ipk() {
+    package_update_index_ipk
+}
+
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 1cf9f79..390d17c 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -815,3 +815,12 @@ python do_package_write_rpm () {
 do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}"
 addtask package_write_rpm before do_package_write after do_package
 
+PACKAGEINDEXES += "package_index_rpm;"
+PACKAGEINDEXDEPS += "rpm-native:do_populate_sysroot"
+PACKAGEINDEXDEPS += "createrepo-native:do_populate_sysroot"
+
+package_index_rpm() {
+    package_update_index_rpm
+    createrepo "${DEPLOY_DIR_RPM}"
+}
+
diff --git a/meta/recipes-core/meta/package-index.bb b/meta/recipes-core/meta/package-index.bb
index 3c642cb..bf3922f 100644
--- a/meta/recipes-core/meta/package-index.bb
+++ b/meta/recipes-core/meta/package-index.bb
@@ -19,13 +19,11 @@ do_package_write_deb[noexec] = "1"
 do_populate_sysroot[noexec] = "1"
 
 do_package_index[nostamp] = "1"
-do_package_index[dirs] = "${DEPLOY_DIR_IPK}"
-do_package_index[depends] += "opkg-utils-native:do_populate_sysroot"
-do_package_index[depends] += "opkg-native:do_populate_sysroot"
+do_package_index[depends] += "${PACKAGEINDEXDEPS}"
 
 do_package_index() {
 	set -ex
-	package_update_index_ipk
+	${PACKAGEINDEXES}
 	set +ex
 }
 addtask do_package_index before do_build
-- 
1.7.2

Re: [PATCH 1/1][RFC][v3] package-index.bb: add support for rpm and deb.

[Saul Wold]

On 05/18/2011 06:10 AM, Dexuan Cui wrote:
> From: Dexuan Cui<dexuan.cui@intel.com>
>
> [YOCTO #1024]
> Currently package-index.bb only supports ipk. This commit adds the support
> for rpm and deb, too.
>
> ------------------------------
> How to generate and use repos:
>
> 1) run "bitbake package-index" after building some target,
> e.g., core-image-sato-sdk;
>
> 2) export ${DEPLOY_DIR_RPM}, ${DEPLOY_DIR_IPK} and ${DEPLOY_DIR_DEB} by a
> webserver on the host, assuming the host IP is 192.168.7.1, at
> http://192.168.7.1/rpm
> http://192.168.7.1/ipk
> http://192.168.7.1/deb
>
> 3) inside the target, according to the packaging system (rpm, ipk or deb) used
> when we generate the target image, we can use different ways to manage
> packages:
>
> 3.1) RPM
>      run "zypper addrepo http://192.168.7.1/rpm main; zypper refresh"
>      to retrieve info about the repo; next, we can use "zypper install/remove"
>      to manage packages.
>
> 3.2) IPK
>      add the repo info into opkg config file, i.e., in
>      /etc/opkg/arch.conf, we can add something like
>      "src i586 http://192.168.7.1/ipk/i586", and next, we run "opkg update" to
>      make opkg update the list of available packages. And later, we can use
>      "opkg install/remove" to manage packages.
>
> 3.3) DEB
>      Currently in target, some important config files, like
> /var/lib/dpkg/status and /etc/apt/sources.list, for deb/apt are missing. So
> we can't install/remove package in target at present.
>
> Signed-off-by: Dexuan Cui<dexuan.cui@intel.com>
> ---
>   meta/classes/package_deb.bbclass        |    9 +++++++++
>   meta/classes/package_ipk.bbclass        |    9 +++++++++
>   meta/classes/package_rpm.bbclass        |    9 +++++++++
>   meta/recipes-core/meta/package-index.bb |    6 ++----
>   4 files changed, 29 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
> index 4faeb4a..f9ef32c 100644
> --- a/meta/classes/package_deb.bbclass
> +++ b/meta/classes/package_deb.bbclass
> @@ -431,3 +431,12 @@ python do_package_write_deb () {
>   do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}"
>   addtask package_write_deb before do_package_write after do_package
>
> +
> +PACKAGEINDEXES += "package_index_deb;"
> +PACKAGEINDEXDEPS += "dpkg-native:do_populate_sysroot"
> +PACKAGEINDEXDEPS += "apt-native:do_populate_sysroot"
> +
> +package_index_deb() {
> +    package_update_index_deb
> +}
> +
> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> index 3c2472b..97de17a 100644
> --- a/meta/classes/package_ipk.bbclass
> +++ b/meta/classes/package_ipk.bbclass
> @@ -386,3 +386,12 @@ python do_package_write_ipk () {
>   }
>   do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}"
>   addtask package_write_ipk before do_package_write after do_package
> +
> +PACKAGEINDEXES += "package_index_ipk;"
> +PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot"
> +PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot"
> +
> +package_index_ipk() {
> +    package_update_index_ipk
> +}
> +
> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
> index 1cf9f79..390d17c 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -815,3 +815,12 @@ python do_package_write_rpm () {
>   do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}"
>   addtask package_write_rpm before do_package_write after do_package
>
> +PACKAGEINDEXES += "package_index_rpm;"
> +PACKAGEINDEXDEPS += "rpm-native:do_populate_sysroot"
> +PACKAGEINDEXDEPS += "createrepo-native:do_populate_sysroot"
> +
> +package_index_rpm() {
> +    package_update_index_rpm
> +    createrepo "${DEPLOY_DIR_RPM}"
> +}
> +
> diff --git a/meta/recipes-core/meta/package-index.bb b/meta/recipes-core/meta/package-index.bb
> index 3c642cb..bf3922f 100644
> --- a/meta/recipes-core/meta/package-index.bb
> +++ b/meta/recipes-core/meta/package-index.bb
> @@ -19,13 +19,11 @@ do_package_write_deb[noexec] = "1"
>   do_populate_sysroot[noexec] = "1"
>
>   do_package_index[nostamp] = "1"
> -do_package_index[dirs] = "${DEPLOY_DIR_IPK}"
> -do_package_index[depends] += "opkg-utils-native:do_populate_sysroot"
> -do_package_index[depends] += "opkg-native:do_populate_sysroot"
> +do_package_index[depends] += "${PACKAGEINDEXDEPS}"
>
>   do_package_index() {
>   	set -ex
> -	package_update_index_ipk
> +	${PACKAGEINDEXES}
>   	set +ex
>   }
>   addtask do_package_index before do_build

Updated version of this with direct PACKAGEINDEXDEP was pulled into oe-core

Thanks for working through this.

Sau!