[PATCH 0/3] Empty-image functionality

[PATCH 0/3] Empty-image functionality

[Alex Franco]

Enable creation of core-empty-image (a recipe that inherits image
without any extra packages) or similar empty images, whether RPM,
DEB or IPK packaging is selected.

[YOCTO #7664]

Alex Franco (3):
  Empty image: core-image-empty recipe, Rootfs fix
  Empty image: package list splitting and iteration
  Empty image: filesystem allocation

 meta/classes/image_types.bbclass             | 10 +++++++++-
 meta/classes/license.bbclass                 |  2 +-
 meta/lib/oe/package_manager.py               |  5 +++--
 meta/lib/oe/rootfs.py                        |  8 ++++++--
 meta/recipes-core/images/core-image-empty.bb |  7 +++++++
 5 files changed, 26 insertions(+), 6 deletions(-)
 create mode 100644 meta/recipes-core/images/core-image-empty.bb

-- 
2.5.0

[PATCH 1/3] Empty image: core-image-empty recipe, Rootfs fix

[Alex Franco]

Added a core-image-empty recipe, as well as a DpkgOpkgRootfs test
which skips the package post installs step, if PACKAGE_INSTALL is
empty.

[YOCTO #7664]

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
 meta/lib/oe/rootfs.py                        | 6 +++++-
 meta/recipes-core/images/core-image-empty.bb | 7 +++++++
 2 files changed, 12 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-core/images/core-image-empty.bb

diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 8c8244c..f8cc6eb 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -585,7 +585,11 @@ class DpkgOpkgRootfs(Rootfs):
 
         pkg_list = []
 
-        pkgs = self._get_pkgs_postinsts(status_file)
+        pkgs = None
+        if not self.d.getVar('PACKAGE_INSTALL', True).strip():
+            bb.note("Building empty image")
+        else:
+            pkgs = self._get_pkgs_postinsts(status_file)
         if pkgs:
             root = "__packagegroup_postinst__"
             pkgs[root] = pkgs.keys()
diff --git a/meta/recipes-core/images/core-image-empty.bb b/meta/recipes-core/images/core-image-empty.bb
new file mode 100644
index 0000000..550567c
--- /dev/null
+++ b/meta/recipes-core/images/core-image-empty.bb
@@ -0,0 +1,7 @@
+SUMMARY = "An empty image."
+IMAGE_INSTALL = ""
+IMAGE_LINGUAS = ""
+PACKAGE_INSTALL = ""
+LICENSE = "MIT"
+
+inherit image
-- 
2.5.0

Re: [PATCH 1/3] Empty image: core-image-empty recipe, Rootfs fix

[Paul Eggleton]

Hi Alex,

On Wednesday 26 August 2015 16:57:33 Alex Franco wrote:
> Added a core-image-empty recipe, as well as a DpkgOpkgRootfs test
> which skips the package post installs step, if PACKAGE_INSTALL is
> empty.
> 
> [YOCTO #7664]
> 
> Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
> ---
>  meta/lib/oe/rootfs.py                        | 6 +++++-
>  meta/recipes-core/images/core-image-empty.bb | 7 +++++++
>  2 files changed, 12 insertions(+), 1 deletion(-)
>  create mode 100644 meta/recipes-core/images/core-image-empty.bb
> 
> diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
> index 8c8244c..f8cc6eb 100644
> --- a/meta/lib/oe/rootfs.py
> +++ b/meta/lib/oe/rootfs.py
> @@ -585,7 +585,11 @@ class DpkgOpkgRootfs(Rootfs):
> 
>          pkg_list = []
> 
> -        pkgs = self._get_pkgs_postinsts(status_file)
> +        pkgs = None
> +        if not self.d.getVar('PACKAGE_INSTALL', True).strip():
> +            bb.note("Building empty image")
> +        else:
> +            pkgs = self._get_pkgs_postinsts(status_file)
>          if pkgs:
>              root = "__packagegroup_postinst__"
>              pkgs[root] = pkgs.keys()
> diff --git a/meta/recipes-core/images/core-image-empty.bb
> b/meta/recipes-core/images/core-image-empty.bb new file mode 100644
> index 0000000..550567c
> --- /dev/null
> +++ b/meta/recipes-core/images/core-image-empty.bb
> @@ -0,0 +1,7 @@
> +SUMMARY = "An empty image."
> +IMAGE_INSTALL = ""
> +IMAGE_LINGUAS = ""
> +PACKAGE_INSTALL = ""
> +LICENSE = "MIT"
> +
> +inherit image

Could you please put this image recipe into meta-selftest instead (call it oe-
selftest-empty-image.bb) and add an oe-selftest test to ensure it builds and 
is empty?

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH 1/3] Empty image: core-image-empty recipe, Rootfs fix

[Alex Franco]

Yes indeed! Sorry, I didn't see this before!

Alex

On 08/29/2015 06:18 AM, Paul Eggleton wrote:
> Hi Alex,
>
> On Wednesday 26 August 2015 16:57:33 Alex Franco wrote:
>> Added a core-image-empty recipe, as well as a DpkgOpkgRootfs test
>> which skips the package post installs step, if PACKAGE_INSTALL is
>> empty.
>>
>> [YOCTO #7664]
>>
>> Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
>> ---
>>   meta/lib/oe/rootfs.py                        | 6 +++++-
>>   meta/recipes-core/images/core-image-empty.bb | 7 +++++++
>>   2 files changed, 12 insertions(+), 1 deletion(-)
>>   create mode 100644 meta/recipes-core/images/core-image-empty.bb
>>
>> diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
>> index 8c8244c..f8cc6eb 100644
>> --- a/meta/lib/oe/rootfs.py
>> +++ b/meta/lib/oe/rootfs.py
>> @@ -585,7 +585,11 @@ class DpkgOpkgRootfs(Rootfs):
>>
>>           pkg_list = []
>>
>> -        pkgs = self._get_pkgs_postinsts(status_file)
>> +        pkgs = None
>> +        if not self.d.getVar('PACKAGE_INSTALL', True).strip():
>> +            bb.note("Building empty image")
>> +        else:
>> +            pkgs = self._get_pkgs_postinsts(status_file)
>>           if pkgs:
>>               root = "__packagegroup_postinst__"
>>               pkgs[root] = pkgs.keys()
>> diff --git a/meta/recipes-core/images/core-image-empty.bb
>> b/meta/recipes-core/images/core-image-empty.bb new file mode 100644
>> index 0000000..550567c
>> --- /dev/null
>> +++ b/meta/recipes-core/images/core-image-empty.bb
>> @@ -0,0 +1,7 @@
>> +SUMMARY = "An empty image."
>> +IMAGE_INSTALL = ""
>> +IMAGE_LINGUAS = ""
>> +PACKAGE_INSTALL = ""
>> +LICENSE = "MIT"
>> +
>> +inherit image
> Could you please put this image recipe into meta-selftest instead (call it oe-
> selftest-empty-image.bb) and add an oe-selftest test to ensure it builds and
> is empty?
>
> Thanks,
> Paul
>

[PATCH 2/3] Empty image: package list splitting and iteration

[Alex Franco]

A few short fixes to splitting/iteration done over package lists
in license.bbclass, package_manager.py and rootfs.py.

[YOCTO #7664]

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
 meta/classes/license.bbclass   | 2 +-
 meta/lib/oe/package_manager.py | 5 +++--
 meta/lib/oe/rootfs.py          | 2 +-
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 32e172a..c616a20 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -39,7 +39,7 @@ python license_create_manifest() {
         return 0
 
     pkg_dic = {}
-    for pkg in image_list_installed_packages(d).split("\n"):
+    for pkg in image_list_installed_packages(d).splitlines():
         pkg_info = os.path.join(d.getVar('PKGDATA_DIR', True),
                                 'runtime-reverse', pkg)
         pkg_name = os.path.basename(os.readlink(pkg_info))
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 2ab1d78..ef917f1 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -995,9 +995,10 @@ class RpmPM(PackageManager):
     '''
     def install(self, pkgs, attempt_only=False):
 
-        bb.note("Installing the following packages: %s" % ' '.join(pkgs))
-        if attempt_only and len(pkgs) == 0:
+        if not pkgs:
+            bb.note("There are no packages to install")
             return
+        bb.note("Installing the following packages: %s" % ' '.join(pkgs))
         pkgs = self._pkg_translate_oe_to_smart(pkgs, attempt_only)
 
         if not attempt_only:
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index f8cc6eb..0bfc217 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -235,7 +235,7 @@ class Rootfs(object):
                 installed_pkgs_dir = self.d.expand('${WORKDIR}/installed_pkgs.txt')
                 pkgs_to_remove = list()
                 with open(installed_pkgs_dir, "r+") as installed_pkgs:
-                    pkgs_installed = installed_pkgs.read().split('\n')
+                    pkgs_installed = installed_pkgs.read().splitlines()
                     for pkg_installed in pkgs_installed[:]:
                         pkg = pkg_installed.split()[0]
                         if pkg in ["update-rc.d",
-- 
2.5.0

[PATCH 3/3] Empty image: filesystem allocation

[Alex Franco]

Increase sparse image block size when ROOTFS_SIZE is smaller than
the minimum needed for ext4 to fit into it.

[YOCTO #7664]

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
---
 meta/classes/image_types.bbclass | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 35ceb7b..61ce222 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -49,8 +49,16 @@ oe_mkext234fs () {
 		extra_imagecmd=$@
 	fi
 
+	# If generating an empty image the size of the sparse block should be large
+	# enough to allocate an ext4 filesystem using 4096 bytes per inode, this is
+	# about 60K, so dd needs a minimum count of 60, with bs=1024 (bytes per IO)
+	eval local COUNT=\"0\"
+	eval local MIN_COUNT=\"60\"
+	if [ $ROOTFS_SIZE -lt $MIN_COUNT ]; then
+		eval COUNT=\"$MIN_COUNT\"
+	fi
 	# Create a sparse image block
-	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE count=0 bs=1k
+	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE count=$COUNT bs=1024
 	mkfs.$fstype -F $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype -d ${IMAGE_ROOTFS}
 }
 
-- 
2.5.0

Re: [PATCH 3/3] Empty image: filesystem allocation

[Benjamin Esquivel]

Hi Alejandro,

On Wed, 2015-08-26 at 16:57 -0500, Alex Franco wrote:
> Increase sparse image block size when ROOTFS_SIZE is smaller than
> the minimum needed for ext4 to fit into it.
> 
> [YOCTO #7664]
> 
> Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
> ---
>  meta/classes/image_types.bbclass | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image_types.bbclass
> b/meta/classes/image_types.bbclass
> index 35ceb7b..61ce222 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -49,8 +49,16 @@ oe_mkext234fs () {
>  		extra_imagecmd=$@
>  	fi
>  
> +	# If generating an empty image the size of the sparse block
> should be large
> +	# enough to allocate an ext4 filesystem using 4096 bytes per
> inode, this is
> +	# about 60K, so dd needs a minimum count of 60, with bs=1024
> (bytes per IO)
> +	eval local COUNT=\"0\"
> +	eval local MIN_COUNT=\"60\"
> +	if [ $ROOTFS_SIZE -lt $MIN_COUNT ]; then
> +		eval COUNT=\"$MIN_COUNT\"
> +	fi
>  	# Create a sparse image block
> -	dd if=/dev/zero
> of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE
> count=0 bs=1k
> +	dd if=/dev/zero
> of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE
> count=$COUNT bs=1024
Variable references enclosed in {} are less prone to misinterpretation.
>  	mkfs.$fstype -F $extra_imagecmd
> ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype -d ${IMAGE_ROOTFS}
>  }
>  
> -- 
> 2.5.0
> 

Re: [PATCH 3/3] Empty image: filesystem allocation

[Richard Purdie]

On Fri, 2015-09-11 at 12:26 -0500, Benjamin Esquivel wrote:
> Hi Alejandro,
> 
> On Wed, 2015-08-26 at 16:57 -0500, Alex Franco wrote:
> > Increase sparse image block size when ROOTFS_SIZE is smaller than
> > the minimum needed for ext4 to fit into it.
> > 
> > [YOCTO #7664]
> > 
> > Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
> > ---
> >  meta/classes/image_types.bbclass | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/image_types.bbclass
> > b/meta/classes/image_types.bbclass
> > index 35ceb7b..61ce222 100644
> > --- a/meta/classes/image_types.bbclass
> > +++ b/meta/classes/image_types.bbclass
> > @@ -49,8 +49,16 @@ oe_mkext234fs () {
> >  		extra_imagecmd=$@
> >  	fi
> >  
> > +	# If generating an empty image the size of the sparse block
> > should be large
> > +	# enough to allocate an ext4 filesystem using 4096 bytes per
> > inode, this is
> > +	# about 60K, so dd needs a minimum count of 60, with bs=1024
> > (bytes per IO)
> > +	eval local COUNT=\"0\"
> > +	eval local MIN_COUNT=\"60\"
> > +	if [ $ROOTFS_SIZE -lt $MIN_COUNT ]; then
> > +		eval COUNT=\"$MIN_COUNT\"
> > +	fi
> >  	# Create a sparse image block
> > -	dd if=/dev/zero
> > of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE
> > count=0 bs=1k
> > +	dd if=/dev/zero
> > of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE
> > count=$COUNT bs=1024
> Variable references enclosed in {} are less prone to misinterpretation.

They are however more likely to get expanded by bitbake before you'd
expect them to be unfortunately :(

Cheers,

Richard


> >  	mkfs.$fstype -F $extra_imagecmd
> > ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype -d ${IMAGE_ROOTFS}
> >  }
> >  
> > -- 
> > 2.5.0
> > 

Re: [PATCH 0/3] Empty-image functionality

[Aníbal Limón]

Acked-by: Aníbal Limón <anibal.limon@linux.intel.com>

On 26/08/15 16:57, Alex Franco wrote:
> Enable creation of core-empty-image (a recipe that inherits image
> without any extra packages) or similar empty images, whether RPM,
> DEB or IPK packaging is selected.
>
> [YOCTO #7664]
>
> Alex Franco (3):
>    Empty image: core-image-empty recipe, Rootfs fix
>    Empty image: package list splitting and iteration
>    Empty image: filesystem allocation
>
>   meta/classes/image_types.bbclass             | 10 +++++++++-
>   meta/classes/license.bbclass                 |  2 +-
>   meta/lib/oe/package_manager.py               |  5 +++--
>   meta/lib/oe/rootfs.py                        |  8 ++++++--
>   meta/recipes-core/images/core-image-empty.bb |  7 +++++++
>   5 files changed, 26 insertions(+), 6 deletions(-)
>   create mode 100644 meta/recipes-core/images/core-image-empty.bb
>