Re: [PATCH] wic: add --reserved-size wks option

[Ed Bartosh <ed.bartosh@linux.intel.com>]

On Tue, Oct 18, 2016 at 12:24:55PM +0200, Maciej Borzęcki wrote:
> On Tue, Oct 18, 2016 at 11:27 AM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> > On Tue, Oct 18, 2016 at 11:24:14AM +0200, Maciej Borzęcki wrote:
> >> On Tue, Oct 18, 2016 at 10:38 AM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> >> > On Tue, Oct 18, 2016 at 10:37:22AM +0200, Maciej Borzęcki wrote:
> >> >> On Tue, Oct 18, 2016 at 9:31 AM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> >> >> > On Mon, Oct 17, 2016 at 04:46:00PM +0200, Maciej Borzęcki wrote:
> >> >> >> On Mon, Oct 17, 2016 at 3:22 PM, Ed Bartosh <ed.bartosh@linux.intel.com> wrote:
> >> >> >> > Hi Maciej,
> >> >> >> >
> >> >> >> > There is already --size and --extra-space options.
> >> >> >> > Can we get the same or similar result by just using them? Do we really
> >> >> >> > need new option for similar purpose?
> >> >> >>
> >> >> >> --reserved-size serves a different purpose, it establishes an upper
> >> >> >> bound on the size of a partition during layout. Unlike
> >> >> >> --size/--extra-space does not depend on the size of the filesystem
> >> >> >> image.
> >> >> >>
> >> >> >> For instance, assume I'm creating an image for SD card/eMMC with a
> >> >> >> fixed partition layout (something simple: boot partition, primary &
> >> >> >> secondary rootfs partitions, some data partition). Because future
> >> >> >> system updates are delivered as filesystem image, I want to make sure
> >> >> >> that there is exactly xxx MBs for my current and future rootfs images
> >> >> >> (regardless of current image size). Neither --size nor --extra-space
> >> >> >> can do that. I could use, say `--size 200 --overhead-factor 1`, but
> >> >> >> this will needlessly create a 200MB rootfs image and if I happen to
> >> >> >> cross the 200MB boundary I will not get an error.
> >> >> >>
> >> >> >> I had a private patch that added --fixed-size to enforce --size, but
> >> >> >> that would still end up creating filesystem image to fill the whole
> >> >> >> space.
> >> >> > I didn't get the difference between enforcing partition size and below
> >> >> > implementation. Can you elaborate a bit?
> >> >>
> >> >> `--fixed-size` was something that I had added to my fork back in 2014,
> >> >> even before `--overhead-factor` came in. The problem is that depending
> >> >> on a project you might want to have more control over how partitions
> >> >> are laid out, or even need to have a fixed layout. Adding
> >> >> `--fixed-size` would had a similar effect to what `--overhead-factor
> >> >> 1` does right now. Combined with `--size` would ensure that rootfs is
> >> >> say, 200MB large. The downside was that wic would actually create a
> >> >> 200MB rootfs, something that is not really necessary. In fact, I only
> >> >> wanted to have 200MB gap so that I have some spare space for future
> >> >> updates (where update is just a rootfs image you dd to the partition).
> >> >>
> >> > Thanks for the explanations. Now I got it - reserved size is not a part
> >> > of partition, it's a gap between partitions.
> >>
> >> I might have not been clear enough when explaining. It's not a gap,
> >> it's just a container of size --reserved-size listed in MBR/GPT.
> >> There's probably a filesystem inside but not necessarily.
> >> Graphically it looks as like this:
> >>
> >>                          --reserved-size
> >>                       |----------------------|
> >>                       v                      v
> >>     +-----------------+----------------------+---------------------+
> >>     |..... stuff .....|xxxxxxxxxx            |..... stuff .........|
> >>     +-----------------+----------------------+---------------------+
> >>                       ^         ^            ^
> >>                       |---------|------------|
> >>                        --size    --extra-space
> >>
> >>
> > Ah, I'm wrong again. It's a partition size limit, but it's not necessary
> > formatted, right? It's only formatted if size == reserved_size.
> >
> >> >
> >> > What's the advantage of creating unusable gap over creating partition of
> >> > the same size that can be used?
> >>
> >> Just convenience.
> >>
> > What's the convenience of having extra space on partition that can't be
> > used for data over having it formatted and used?
> >
> >> > Even if that space is not needed it doesn't harm to have it, does it?
> >>
> >> I have not seen any negative side effects.
> >>
> > I do. If user needs that reserved space it's impossible to get it without
> > reformatting partition. The free space exists, but can't be used.
> 
> That's not the point and is not aligned with use case I'm trying to solve.
> 
> My case is rather simple, I'm creating an image for SD card that is
> deployed in the field. In that particular case, there's a primary and
> a secondary (aka. active and inactive) rootfs partitions that are
> switched whenever a system update comes in. The update is a file
> system image that is copied over to the inactive partition, followed
> by a system reboot.
> 
> What I need is the ability to set a certain size of a partition (say
> 100MB), regardless of current rootfs size (which may be, say 70MB).
> The remaining unused space sets an upper limit on how much the rootfs
> may grow in the future (in this example case, it's 30MB). RIght now
> the best I can do is to describe a partition like this: `part /
> --source rootfs --size 100MB --overhead-factor 1`, hoping that if
> rootfs grows beyond 100MB I will somehow still be able to catch that
> and that the future images remain size compatible.
> 
> The resulting filesystem inside the partition is larger than what
> IMAGE_CMD (ex. IMAGE_CMD_ext4) would give me, because of explicit
> --size in kickstart. I would prefer to have something comparable in
> size just to avoid later surprises, what implies using defaults.
> However, using defaults, means that I cannot control the layout
> because it will likely change each time rootfs size gets changed.
> There is no `--fixed-size` or other option to enforce specific size.
> 
> Summing up, a simple use case that cannot be currently solved using wic.
> 
> BTW. actually we're missing an ability to enforce --size (i.e.
> --fixed-size?) and a method passing an explicit partition offset
> inside the disk image (something useful for `--source rawcopy
> --no-table` partitions, currently solved with `--align`).
> 
I undertood the problem and I agree that wic doesn't provide a solution.

However, instead of making unformatted space I'd propose to format it,
i.e. to have --max-size option that would confict with --size and
specify upper size limit for the partition. All partition will be
formatted and available for data. This is identical to --fixed-size option
you've described. This approach would solve the problem you're
addressing and it would also make additional space usable.

I'd also suggest to rename --size to --min-size and make --size deprecated.

Does this make sense to you?

--
Regards,
Ed