From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mail.openembedded.org (Postfix) with ESMTP id 8A6416F170 for ; Fri, 21 Feb 2014 08:24:47 +0000 (UTC) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP; 21 Feb 2014 00:24:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,517,1389772800"; d="scan'208";a="485357466" Received: from lpalcu-linux.rb.intel.com (HELO lpalcu-linux) ([10.237.105.45]) by fmsmga002.fm.intel.com with ESMTP; 21 Feb 2014 00:24:46 -0800 Date: Fri, 21 Feb 2014 10:24:45 +0200 From: Laurentiu Palcu To: Hongxu Jia Message-ID: <20140221082445.GG30981@lpalcu-linux> References: MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: saul.wold@intel.com, openembedded-core@lists.openembedded.org Subject: Re: [PATCH V3 0/5] manifest.py/package_manager.py/rootfs.py: support ipk incremental image generation X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Feb 2014 08:24:50 -0000 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Reviewed-by: Laurentiu Palcu laurentiu On Fri, Feb 21, 2014 at 02:38:16PM +0800, Hongxu Jia wrote: > Change in V2: > - Move dump_install_solution to Manifest class as create_full; > - Move load_old_install_solution to Manifest class as parse_full_manifest; > - Handle the output of dummy install in Python rather than shell; > - Fix typos > - Rename _file_duplicate with _file_equal to avoid confusion; > > Change in V3: > - While create_full calls dummy_install, pass an instance to PM > as an argument rather than instantiate another PM object; > - Rename _create_incremental with _remove_extra_packages; > - Move the call of _remove_extra_packages below the call of > self.pm.handle_bad_recommendations(); > - Remove redundant self.pm.update in _remove_extra_packages; > - Rename _remove_existing_image with _remove_old_rootfs; > > Test Cases > > --------------------------------------------- > Case 1: Rebuild in place > 1. Edit local.conf, enable multilib and ipk incremental image generation > ... > 46 MACHINE ?= "qemux86-64" > 254 require conf/multilib.conf > 255 MULTILIBS = "multilib:lib32" > 256 DEFAULTTUNE_virtclass-multilib-lib32 = "x86" > 257 > 258 IMAGE_INSTALL_append = " lib32-gzip" > 260 IMAGE_FEATURES_append = " package-management ssh-server-dropbear " > 262 > 263 INC_IPK_IMAGE_GEN = "1" > ... > bb core-image-minimal > > 2. Edit local.conf, add some spaces in IMAGE_INSTALL_append > vim local.conf > ... > 258 IMAGE_INSTALL_append = " lib32-gzip " > ... > > 3. bb core-image-minimal, the image should not be rebuilt > vim log.do_rootfs: > ... > 16 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. > 17 Package opkg-collateral (1.0-r2) installed in root is up to date. > 18 Package run-postinsts (1.0-r9) installed in root is up to date. > 19 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. > 20 Package opkg (1:0.2.0-r0) installed in root is up to date. > 21 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. > ... > 53 NOTE: Package lib32-gzip (1.6-r7) installed in root is up to date. > ... > > vim installed_pkgs.txt > ... > lib32-gzip x86 > ... > > 4. runqemu qemux86-64, ssh could work and lib32-gzip is installed > On host: > runqemu qemux86-64 > ssh root@192.168.7.2 and run 'opkg-cl list-installed' > ... > lib32-gzip - 1.6-r7 > ... > > --------------------------------------------- > Case 2: Decremental Rebuild > 1. Edit local.conf, enable multilib and ipk incremental image generation > ... > 46 MACHINE ?= "qemux86-64" > 254 require conf/multilib.conf > 255 MULTILIBS = "multilib:lib32" > 256 DEFAULTTUNE_virtclass-multilib-lib32 = "x86" > 257 > 258 IMAGE_INSTALL_append = " lib32-gzip openssl" > 260 IMAGE_FEATURES_append = " package-management ssh-server-dropbear " > 262 > 263 INC_IPK_IMAGE_GEN = "1" > ... > bb core-image-minimal > > 2. Edit local.conf, remove 'lib32-gzip openssl' > vim local.conf > ... > 258 # IMAGE_INSTALL_append = " lib32-gzip openssl" > ... > > 3. bb core-image-minimal, lib32-gzip and openssl should be removed. > vim log.do_rootfs: > ... > 20 NOTE: Removing package lib32-eglibc from root... > 21 Removing package lib32-gzip from root... > 22 Removing package lib32-update-alternatives-opkg from root... > 23 Removing package libcrypto1.0.0 from root... > 24 Removing package libssl1.0.0 from root... > 25 Removing package openssl from root... > 26 Removing package openssl-conf from root... > ... > 31 Package opkg-collateral (1.0-r2) installed in root is up to date. > 32 Package run-postinsts (1.0-r9) installed in root is up to date. > 33 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. > 34 Package opkg (1:0.2.0-r0) installed in root is up to date. > 35 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. > ... > > --------------------------------------------- > Case 3: Incremental Rebuild > 1. Edit local.conf, enable multilib and ipk incremental image generation > ... > 46 MACHINE ?= "qemux86-64" > 254 require conf/multilib.conf > 255 MULTILIBS = "multilib:lib32" > 256 DEFAULTTUNE_virtclass-multilib-lib32 = "x86" > 257 > 258 # IMAGE_INSTALL_append = " lib32-gzip openssl" > 260 IMAGE_FEATURES_append = " package-management ssh-server-dropbear " > 262 > 263 INC_IPK_IMAGE_GEN = "1" > ... > bb core-image-minimal > > 2. Edit local.conf, append 'lib32-gzip openssl' to IMAGE_INSTALL > vim local.conf > ... > 258 IMAGE_INSTALL_append = " lib32-gzip openssl" > ... > > 3. bb core-image-minimal, lib32-gzip and openssl should be added. > vim log.do_rootfs: > ... > 20 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. > 21 Package opkg-collateral (1.0-r2) installed in root is up to date. > 22 Package run-postinsts (1.0-r9) installed in root is up to date. > 23 Installing openssl (1.0.1e-r15.0) to root... > ... > 31 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. > 32 Package opkg (1:0.2.0-r0) installed in root is up to date. > 33 Package poky-feed-config-opkg (1.0-r2) installed in root is up to date. > ... > 54 NOTE: Installing the following packages: lib32-gzip > ... > > --------------------------------------------- > Case 4: Rebuild in place and Upgrade package > 1. Edit local.conf, enable ipk incremental image generation > ... > 258 IMAGE_INSTALL_append = " gzip" > 263 INC_IPK_IMAGE_GEN = "1" > ... > bb core-image-minimal > > 2. Edit local.conf, upgrade gzip's PR > ... > 258 PR_pn-gzip = "r1" > ... > > 3. bb core-image-minimal, gzip should be upgraded to r1 > vim log.do_rootfs: > ... > 21 NOTE: Package packagegroup-core-ssh-dropbear (1.0-r1) installed in root is up to date. > 22 Package opkg-collateral (1.0-r2) installed in root is up to date. > 23 Package run-postinsts (1.0-r9) installed in root is up to date. > 24 Package packagegroup-core-boot (1.0-r11) installed in root is up to date. > 25 Package opkg (1:0.2.0-r0) installed in root is up to date. > 26 Upgrading gzip on root from 1.6-r0 to 1.6-r1... > ... > > --------------------------------------------- > Case 5: Test BAD_RECOMMENDATIONS > 1. Edit local.conf, enable ipk incremental image generation > ... > 258 IMAGE_INSTALL_append = " util-linux" > 263 INC_IPK_IMAGE_GEN = "1" > ... > bb core-image-minimal > > The util-linux-fdisk is in util-linux's RRECOMMENDS > vim installed_pkgs.txt > ... > 41 util-linux-fdisk core2-64 > ... > > 2. Edit local.conf, add some spaces to IMAGE_INSTALL and add > util-linux-fdisk to core-image-minimal's BAD_RECOMMENDATIONS > ... > 260 IMAGE_INSTALL_append = " util-linux " > 265 BAD_RECOMMENDATIONS_pn-core-image-minimal += "util-linux-fdisk" > ... > > 3. bb core-image-minimal, util-linux-fdisk should not be installed > vim log.do_rootfs: > ... > 34 NOTE: Requested ignored recommendation util-linux-fdisk is not a package > ... > 119 util-linux: ignoring recommendation for util-linux-fdisk at user request > 120 util-linux: unsatisfied recommendation for util-linux-fdisk > ... > > The util-linux-fdisk doesn't exist in installed_pkgs.txt any more. > > --------------------------------------------- > Case 6: Test NO_RECOMMENDATIONS > 1. Edit local.conf, enable ipk incremental image generation > ... > 258 IMAGE_INSTALL_append = " util-linux" > 263 INC_IPK_IMAGE_GEN = "1" > ... > bb core-image-minimal > > The util-linux-fdisk util-linux-cfdisk util-linux-sfdisk > util-linux-mount util-linux-readprofile util-linux-mkfs are > in util-linux's RRECOMMENDS > > vim installed_pkgs.txt > ... > 4 util-linux-mkfs core2-64 > 18 util-linux-sfdisk core2-64 > 20 util-linux-mount core2-64 > 38 util-linux-cfdisk core2-64 > 41 util-linux-fdisk core2-64 > 46 util-linux-readprofile core2-64 > ... > > 2. Edit local.conf, add some spaces to IMAGE_INSTALL and > assign core-image-minimal's NO_RECOMMENDATIONS with 1 > ... > 260 IMAGE_INSTALL_append = " util-linux " > 267 NO_RECOMMENDATIONS_pn-core-image-minimal = "1" > ... > > 3. bb core-image-minimal, The six packages should not be installed > vim log.do_rootfs: > ... > 34 NOTE: Requested ignored recommendation util-linux-fdisk is not a package > ... > 113 util-linux: ignoring recommendation for util-linux-sfdisk at user request > 114 util-linux: unsatisfied recommendation for util-linux-sfdisk > 115 util-linux: ignoring recommendation for util-linux-cfdisk at user request > 116 util-linux: unsatisfied recommendation for util-linux-cfdisk > 117 util-linux: ignoring recommendation for util-linux-mount at user request > 118 util-linux: unsatisfied recommendation for util-linux-mount > 119 util-linux: ignoring recommendation for util-linux-fdisk at user request > 120 util-linux: unsatisfied recommendation for util-linux-fdisk > 121 util-linux: ignoring recommendation for util-linux-mkfs at user request > 122 util-linux: unsatisfied recommendation for util-linux-mkfs > 123 util-linux: ignoring recommendation for util-linux-readprofile at user request > 124 util-linux: unsatisfied recommendation for util-linux-readprofile > ... > > They don't exist in installed_pkgs.txt any more. > > //Hongxu > > The following changes since commit bb0c26960d3b05070346cdfdfd05d6fd4ad0a7b1: > > wic: Make exec_native_command() fail if a command isn't found (2014-02-20 14:28:13 +0000) > > are available in the git repository at: > > git://git.pokylinux.org/poky-contrib hongxu/ipk-inc-gen > http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/ipk-inc-gen > > Hongxu Jia (5): > manifest.py: add create_full for OpkgManifest class > package_manager.py: support ipk incremental image generation > rootfs.py: support ipk incremental image generation > rootfs.py: support BAD_RECOMMENDATIONS for ipk incremental image > generation > rootfs.py: tweak _multilib_sanity_test for ipk incremental image > generation > > meta/lib/oe/manifest.py | 57 +++++++++++++++++- > meta/lib/oe/package_manager.py | 81 ++++++++++++++++++++++++-- > meta/lib/oe/rootfs.py | 128 +++++++++++++++++++++++++++++++++++++++-- > 3 files changed, 254 insertions(+), 12 deletions(-) > > -- > 1.8.1.2 >