* Re: minimize size of SW update bundle
2017-10-17 13:12 ` minimize size of SW update bundle Robert Berger
@ 2017-10-17 13:28 ` Alexander Kanavin
2017-10-17 14:11 ` Marc Ferland
2017-10-19 7:18 ` Patrick Ohly
2 siblings, 0 replies; 6+ messages in thread
From: Alexander Kanavin @ 2017-10-17 13:28 UTC (permalink / raw)
To: yocto
On 10/17/2017 04:12 PM, Robert Berger wrote:
> Imagine you have only a limited bandwidth to your devices available for
> SW update. So you would like to minimize what's uploaded (over the air)
> to your devices.
>
> How would you do this with yocto?
>
> [1] and [2] might be able to do incremental updates, but maybe there are
> more ideas out there.
>
> Like how would you get a diff between two release builds...
>
> [1] https://wiki.yoctoproject.org/wiki/Meta-swupd
> [2] https://github.com/advancedtelematic/meta-updater
If you want to stick with desktop-style rpm package feeds, then there's
deltarpm, but YP doesn't currently support it, and I don't know if it's
still used by anyone. From what I remember, it did cut down on the
downloads time, but installation became much slower.
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: minimize size of SW update bundle
2017-10-17 13:12 ` minimize size of SW update bundle Robert Berger
2017-10-17 13:28 ` Alexander Kanavin
@ 2017-10-17 14:11 ` Marc Ferland
2017-10-18 20:34 ` Paul Eggleton
2017-10-19 7:18 ` Patrick Ohly
2 siblings, 1 reply; 6+ messages in thread
From: Marc Ferland @ 2017-10-17 14:11 UTC (permalink / raw)
To: Robert Berger; +Cc: yocto@yoctoproject.org
On Tue, Oct 17, 2017 at 9:12 AM, Robert Berger
<yocto.user.mailinglist@gmail.com> wrote:
> Hi,
>
> Imagine you have only a limited bandwidth to your devices available for SW
> update. So you would like to minimize what's uploaded (over the air) to your
> devices.
>
> How would you do this with yocto?
>
> [1] and [2] might be able to do incremental updates, but maybe there are
> more ideas out there.
>
> Like how would you get a diff between two release builds...
>
Some ideas:
- SquashFS with agressive compression settings can be quite
impressive. I was able to reduce my image size from 650MB
(uncompressed) down to maybe 120MB. This was for a complete system
update though.
- The new zstd algo from facebook looks promising (haven't tried it
yet) and might be worth investigating, I think it is supported as a
squashfs decompressor.
- bsdiff can generate diffs from binary files (be carefull if diffing
compressed files, i.e.: gzip --rsyncable).
- there's also casync (never tried it): https://github.com/systemd/casync/
Enjoy!
Marc
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: minimize size of SW update bundle
2017-10-17 14:11 ` Marc Ferland
@ 2017-10-18 20:34 ` Paul Eggleton
0 siblings, 0 replies; 6+ messages in thread
From: Paul Eggleton @ 2017-10-18 20:34 UTC (permalink / raw)
To: yocto
On Wednesday, 18 October 2017 3:11:59 AM NZDT Marc Ferland wrote:
> On Tue, Oct 17, 2017 at 9:12 AM, Robert Berger
> <yocto.user.mailinglist@gmail.com> wrote:
> > Hi,
> >
> > Imagine you have only a limited bandwidth to your devices available for SW
> > update. So you would like to minimize what's uploaded (over the air) to
your
> > devices.
> >
> > How would you do this with yocto?
> >
> > [1] and [2] might be able to do incremental updates, but maybe there are
> > more ideas out there.
> >
> > Like how would you get a diff between two release builds...
> >
>
> Some ideas:
>
> - SquashFS with agressive compression settings can be quite
> impressive. I was able to reduce my image size from 650MB
> (uncompressed) down to maybe 120MB. This was for a complete system
> update though.
> - The new zstd algo from facebook looks promising (haven't tried it
> yet) and might be worth investigating, I think it is supported as a
> squashfs decompressor.
> - bsdiff can generate diffs from binary files (be carefull if diffing
> compressed files, i.e.: gzip --rsyncable).
> - there's also casync (never tried it): https://github.com/systemd/casync/
FWIW I have preliminary recipes for casync and zstd here, but I'm having
trouble with some native dependencies not being populated properly when
casync-native is pulled in by other recipes (or -c addto_recipe_sysroot), so
it still needs work:
http://cgit.openembedded.org/meta-openembedded-contrib/log/?h=paule/casync
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: minimize size of SW update bundle
2017-10-17 13:12 ` minimize size of SW update bundle Robert Berger
2017-10-17 13:28 ` Alexander Kanavin
2017-10-17 14:11 ` Marc Ferland
@ 2017-10-19 7:18 ` Patrick Ohly
2 siblings, 0 replies; 6+ messages in thread
From: Patrick Ohly @ 2017-10-19 7:18 UTC (permalink / raw)
To: Robert Berger, yocto
On Tue, 2017-10-17 at 16:12 +0300, Robert Berger wrote:
> Hi,
>
> Imagine you have only a limited bandwidth to your devices available
> for
> SW update. So you would like to minimize what's uploaded (over the
> air)
> to your devices.
>
> How would you do this with yocto?
>
> [1] and [2] might be able to do incremental updates,
Both can indeed do it.
> Like how would you get a diff between two release builds...
>
> [1] https://wiki.yoctoproject.org/wiki/Meta-swupd
With meta-swupd, set SWUPD_DELTAPACK_VERSIONS to generate binary diffs
against certain older releases. In addition, a single archive gets
created with just the right content for updating from the older
releases to the current one (= better compression, less GET requests).
> [2] https://github.com/advancedtelematic/meta-updater
That's basically OSTree. See https://ostree.readthedocs.io/en/latest/ma
nual/repository-management/#derived-data-static-deltas-and-the-summary-
file for instructions on generating deltas.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
^ permalink raw reply [flat|nested] 6+ messages in thread