[meta-rockchip][PATCH] user-selectable wic format

[meta-rockchip][PATCH] user-selectable wic format

[Trevor Woerner]

Allow the user to choose their preferred wic image format.

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
---
 conf/machine/include/rockchip-wic.inc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc
index dab61d83ed2c..eb895cd0b4ad 100644
--- a/conf/machine/include/rockchip-wic.inc
+++ b/conf/machine/include/rockchip-wic.inc
@@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
 
 SPL_BINARY ?= "idbloader.img"
 
-IMAGE_FSTYPES += "wic wic.bmap"
+WIC_FSTYPE ?= "wic"
+IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"
 WKS_FILE ?= "rockchip.wks"
 WKS_FILE_DEPENDS ?= " \
 	e2fsprogs-native \
-- 
2.44.0.478.g7774cfed6261

Re: [yocto-patches] [meta-rockchip][PATCH] user-selectable wic format

[Quentin Schulz]

Hi Trevor,

On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote:
> Allow the user to choose their preferred wic image format.
> 

Can you provide some use case for this?

> Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> ---
>   conf/machine/include/rockchip-wic.inc | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc
> index dab61d83ed2c..eb895cd0b4ad 100644
> --- a/conf/machine/include/rockchip-wic.inc
> +++ b/conf/machine/include/rockchip-wic.inc
> @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
>   
>   SPL_BINARY ?= "idbloader.img"
>   
> -IMAGE_FSTYPES += "wic wic.bmap"
> +WIC_FSTYPE ?= "wic"
> +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"

Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying 
to achieve here?

If so, shouldn't we also not build wic.bmap there?

Cheers,
Quentin

Re: [yocto-patches] [meta-rockchip][PATCH] user-selectable wic format

[Trevor Woerner]

On Tue 2024-06-11 @ 04:31:26 PM, Quentin Schulz via lists.yoctoproject.org wrote:
> Hi Trevor,
> 
> On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote:
> > Allow the user to choose their preferred wic image format.
> > 
> 
> Can you provide some use case for this?

In order to build a wic.xz image the build has to first create a wic image,
then compress it as an extra step. On a slow machine this extra step takes
a noticeable amount of time.

On my local, slow build machine I prefer wic images since it saves build time
not having to do the compression and since everything is local, there's no
over-the-internet xfer time to consider.

When I build using some remote, fast build machine, the extra time spent doing
the extra step of compressing is recuperated by the savings in transfer time
retrieving the image from the remote. Therefore I prefer to make wic.xz
images. Building wic images on remote machines would lead to very long xfer
times to download the image artifact.

If I simply do:

	IMAGE_FSTYPES += "wic.xz"

in my conf/local.conf then MACHINEs that don't normally build wic images will
fail. So the only way to build wic.xz images on remote builds is to tweak the
meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always
carrying this tweak for every remote build that I do.

So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to
wic.xz but that wouldn't suit my use-case since I could then have to tweak it
again for any local builds. Besides, I could never guess what preferred wic
version others would prefer and I don't want to be changing it every other
week.

So this patch maintains the existing behaviour completely, and when I do a
build on a remote machine I can simply set

	WIC_FSTYPE = "wic.xz"

in my conf/local.conf without having to tweak the layer.

And if others want compression but would prefer bz2 or gzip (or whatever) then
then can specify it however they wish.

> > Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> > ---
> >   conf/machine/include/rockchip-wic.inc | 3 ++-
> >   1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc
> > index dab61d83ed2c..eb895cd0b4ad 100644
> > --- a/conf/machine/include/rockchip-wic.inc
> > +++ b/conf/machine/include/rockchip-wic.inc
> > @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
> >   SPL_BINARY ?= "idbloader.img"
> > -IMAGE_FSTYPES += "wic wic.bmap"
> > +WIC_FSTYPE ?= "wic"
> > +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"
> 
> Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to
> achieve here?
> 
> If so, shouldn't we also not build wic.bmap there?

The goal isn't to not build a wic image, the point is to allow the user to
specify which type of wic image (i.e. with or without compression and if with
compression, which one?).

I guess I could do:

	WIC_COMPRESSION ?= ""
	IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap"

And then specify:

	WIC_COMPRESSION = ".xz"

in my conf/local.conf?

Re: [yocto-patches] [meta-rockchip][PATCH] user-selectable wic format

[Quentin Schulz]

Hi Trevor,

On 6/11/24 5:13 PM, Trevor Woerner via lists.yoctoproject.org wrote:
> On Tue 2024-06-11 @ 04:31:26 PM, Quentin Schulz via lists.yoctoproject.org wrote:
>> Hi Trevor,
>>
>> On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote:
>>> Allow the user to choose their preferred wic image format.
>>>
>>
>> Can you provide some use case for this?
> 
> In order to build a wic.xz image the build has to first create a wic image,
> then compress it as an extra step. On a slow machine this extra step takes
> a noticeable amount of time.
> 
> On my local, slow build machine I prefer wic images since it saves build time
> not having to do the compression and since everything is local, there's no
> over-the-internet xfer time to consider.
> 
> When I build using some remote, fast build machine, the extra time spent doing
> the extra step of compressing is recuperated by the savings in transfer time
> retrieving the image from the remote. Therefore I prefer to make wic.xz
> images. Building wic images on remote machines would lead to very long xfer
> times to download the image artifact.
> 
> If I simply do:
> 
> 	IMAGE_FSTYPES += "wic.xz"
> 
> in my conf/local.conf then MACHINEs that don't normally build wic images will
> fail. So the only way to build wic.xz images on remote builds is to tweak the
> meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always
> carrying this tweak for every remote build that I do.
> 
> So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to
> wic.xz but that wouldn't suit my use-case since I could then have to tweak it
> again for any local builds. Besides, I could never guess what preferred wic
> version others would prefer and I don't want to be changing it every other
> week.
> 
> So this patch maintains the existing behaviour completely, and when I do a
> build on a remote machine I can simply set
> 
> 	WIC_FSTYPE = "wic.xz"
> 
> in my conf/local.conf without having to tweak the layer.
> 
> And if others want compression but would prefer bz2 or gzip (or whatever) then
> then can specify it however they wish.
> 
>>> Signed-off-by: Trevor Woerner <twoerner@gmail.com>
>>> ---
>>>    conf/machine/include/rockchip-wic.inc | 3 ++-
>>>    1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc
>>> index dab61d83ed2c..eb895cd0b4ad 100644
>>> --- a/conf/machine/include/rockchip-wic.inc
>>> +++ b/conf/machine/include/rockchip-wic.inc
>>> @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
>>>    SPL_BINARY ?= "idbloader.img"
>>> -IMAGE_FSTYPES += "wic wic.bmap"
>>> +WIC_FSTYPE ?= "wic"
>>> +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"
>>
>> Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to
>> achieve here?
>>
>> If so, shouldn't we also not build wic.bmap there?
> 
> The goal isn't to not build a wic image, the point is to allow the user to
> specify which type of wic image (i.e. with or without compression and if with
> compression, which one?).
> 
> I guess I could do:
> 
> 	WIC_COMPRESSION ?= ""
> 	IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap"
> 
> And then specify:
> 
> 	WIC_COMPRESSION = ".xz"
> 
> in my conf/local.conf?
> 

Thanks for taking the time to explain your use case :)

I was about to suggest this :)

Should we allow multiple values in WIC_COMPRESSION?

Also maybe it should be named "EXTENSION" instead since we have the dot 
in there?

Additionally, I don't see wic.xz in IMAGE_TYPES in 
https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/image_types.bbclass, 
is xz really working? If so, why do we need to specify wic.gz and all 
the others in IMAGE_TYPES? Maybe we do not?

Cheers,
Quentin

Re: [yocto-patches] [meta-rockchip][PATCH] user-selectable wic format

[Trevor Woerner]

On Tue 2024-06-11 @ 05:28:42 PM, Quentin Schulz via lists.yoctoproject.org wrote:
> Hi Trevor,
> 
> On 6/11/24 5:13 PM, Trevor Woerner via lists.yoctoproject.org wrote:
> > On Tue 2024-06-11 @ 04:31:26 PM, Quentin Schulz via lists.yoctoproject.org wrote:
> > > Hi Trevor,
> > > 
> > > On 6/11/24 4:15 PM, Trevor Woerner via lists.yoctoproject.org wrote:
> > > > Allow the user to choose their preferred wic image format.
> > > > 
> > > 
> > > Can you provide some use case for this?
> > 
> > In order to build a wic.xz image the build has to first create a wic image,
> > then compress it as an extra step. On a slow machine this extra step takes
> > a noticeable amount of time.
> > 
> > On my local, slow build machine I prefer wic images since it saves build time
> > not having to do the compression and since everything is local, there's no
> > over-the-internet xfer time to consider.
> > 
> > When I build using some remote, fast build machine, the extra time spent doing
> > the extra step of compressing is recuperated by the savings in transfer time
> > retrieving the image from the remote. Therefore I prefer to make wic.xz
> > images. Building wic images on remote machines would lead to very long xfer
> > times to download the image artifact.
> > 
> > If I simply do:
> > 
> > 	IMAGE_FSTYPES += "wic.xz"
> > 
> > in my conf/local.conf then MACHINEs that don't normally build wic images will
> > fail. So the only way to build wic.xz images on remote builds is to tweak the
> > meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always
> > carrying this tweak for every remote build that I do.
> > 
> > So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to
> > wic.xz but that wouldn't suit my use-case since I could then have to tweak it
> > again for any local builds. Besides, I could never guess what preferred wic
> > version others would prefer and I don't want to be changing it every other
> > week.
> > 
> > So this patch maintains the existing behaviour completely, and when I do a
> > build on a remote machine I can simply set
> > 
> > 	WIC_FSTYPE = "wic.xz"
> > 
> > in my conf/local.conf without having to tweak the layer.
> > 
> > And if others want compression but would prefer bz2 or gzip (or whatever) then
> > then can specify it however they wish.
> > 
> > > > Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> > > > ---
> > > >    conf/machine/include/rockchip-wic.inc | 3 ++-
> > > >    1 file changed, 2 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc
> > > > index dab61d83ed2c..eb895cd0b4ad 100644
> > > > --- a/conf/machine/include/rockchip-wic.inc
> > > > +++ b/conf/machine/include/rockchip-wic.inc
> > > > @@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
> > > >    SPL_BINARY ?= "idbloader.img"
> > > > -IMAGE_FSTYPES += "wic wic.bmap"
> > > > +WIC_FSTYPE ?= "wic"
> > > > +IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"
> > > 
> > > Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to
> > > achieve here?
> > > 
> > > If so, shouldn't we also not build wic.bmap there?
> > 
> > The goal isn't to not build a wic image, the point is to allow the user to
> > specify which type of wic image (i.e. with or without compression and if with
> > compression, which one?).
> > 
> > I guess I could do:
> > 
> > 	WIC_COMPRESSION ?= ""
> > 	IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap"
> > 
> > And then specify:
> > 
> > 	WIC_COMPRESSION = ".xz"
> > 
> > in my conf/local.conf?
> > 
> 
> Thanks for taking the time to explain your use case :)
> 
> I was about to suggest this :)
> 
> Should we allow multiple values in WIC_COMPRESSION?
> 
> Also maybe it should be named "EXTENSION" instead since we have the dot in
> there?

Sounds good.

> Additionally, I don't see wic.xz in IMAGE_TYPES in https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/image_types.bbclass,
> is xz really working?

I have been using "wic.xz" in IMAGE_FSTYPES for years. Works with bmaptool
too.

> If so, why do we need to specify wic.gz and all the
> others in IMAGE_TYPES? Maybe we do not?

I guess that's for others to answer/investigate.