Release Candidate Build for yocto-2.4.rc3 now available.

Release Candidate Build for yocto-2.4.rc3 now available.

[pokybuild]

A release candidate build for yocto-2.4.rc3 is now available at:


    https://autobuilder.yocto.io/pub/releases/yocto-2.4.rc3


Please begin QA on this build as soon as possible.


Build hash information: 
meta-intel : c30e4730773d0e660e92837ee841efda2f4d29dc 
meta-qt4 : f313dbee2ac3d5fcc9801407947d3cb6cfb90b5d 
refkit : 6fee176c99d1c28e46ce0d45d4b1eb4be50d06e6 
meta-mingw : 52515d8bee445d728d5fe63bfcbca79b5d8ea250 
meta-qt3 : f33b73a9563f2dfdfd0ee37b61d65d90197a456f 
meta-gplv2 : c7f97f199a6dd54d3f369f0465a6bfd2cfa739b9 
poky : 65d23bd7986615fdfb0f1717b615534a2a14ab80 

\nThis is an automated message from\nThe Yocto Project Autobuilder\nGit: git://git.yoctoproject.org/yocto-autobuilder\nEmail: joshua.g.lock@intel.com 

minimize size of SW update bundle

[Robert Berger]

Hi,

Imagine you have only a limited bandwidth to your devices available for 
SW update. So you would like to minimize what's uploaded (over the air) 
to your devices.

How would you do this with yocto?

[1] and [2] might be able to do incremental updates, but maybe there are 
more ideas out there.

Like how would you get a diff between two release builds...

[1] https://wiki.yoctoproject.org/wiki/Meta-swupd
[2] https://github.com/advancedtelematic/meta-updater

Regards,

Robert

Re: minimize size of SW update bundle

[Alexander Kanavin]

On 10/17/2017 04:12 PM, Robert Berger wrote:
> Imagine you have only a limited bandwidth to your devices available for 
> SW update. So you would like to minimize what's uploaded (over the air) 
> to your devices.
> 
> How would you do this with yocto?
> 
> [1] and [2] might be able to do incremental updates, but maybe there are 
> more ideas out there.
> 
> Like how would you get a diff between two release builds...
> 
> [1] https://wiki.yoctoproject.org/wiki/Meta-swupd
> [2] https://github.com/advancedtelematic/meta-updater

If you want to stick with desktop-style rpm package feeds, then there's 
deltarpm, but YP doesn't currently support it, and I don't know if it's 
still used by anyone. From what I remember, it did cut down on the 
downloads time, but installation became much slower.

Alex

Re: minimize size of SW update bundle

[Marc Ferland]

On Tue, Oct 17, 2017 at 9:12 AM, Robert Berger
<yocto.user.mailinglist@gmail.com> wrote:
> Hi,
>
> Imagine you have only a limited bandwidth to your devices available for SW
> update. So you would like to minimize what's uploaded (over the air) to your
> devices.
>
> How would you do this with yocto?
>
> [1] and [2] might be able to do incremental updates, but maybe there are
> more ideas out there.
>
> Like how would you get a diff between two release builds...
>

Some ideas:

- SquashFS with agressive compression settings can be quite
impressive. I was able to reduce my image size from 650MB
(uncompressed) down to maybe 120MB. This was for a complete system
update though.
- The new zstd algo from facebook looks promising (haven't tried it
yet) and might be worth investigating, I think it is supported as a
squashfs decompressor.
- bsdiff can generate diffs from binary files (be carefull if diffing
compressed files, i.e.: gzip --rsyncable).
- there's also casync (never tried it): https://github.com/systemd/casync/

Enjoy!

Marc

Re: minimize size of SW update bundle

[Paul Eggleton]

On Wednesday, 18 October 2017 3:11:59 AM NZDT Marc Ferland wrote:
> On Tue, Oct 17, 2017 at 9:12 AM, Robert Berger
> <yocto.user.mailinglist@gmail.com> wrote:
> > Hi,
> >
> > Imagine you have only a limited bandwidth to your devices available for SW
> > update. So you would like to minimize what's uploaded (over the air) to 
your
> > devices.
> >
> > How would you do this with yocto?
> >
> > [1] and [2] might be able to do incremental updates, but maybe there are
> > more ideas out there.
> >
> > Like how would you get a diff between two release builds...
> >
> 
> Some ideas:
> 
> - SquashFS with agressive compression settings can be quite
> impressive. I was able to reduce my image size from 650MB
> (uncompressed) down to maybe 120MB. This was for a complete system
> update though.
> - The new zstd algo from facebook looks promising (haven't tried it
> yet) and might be worth investigating, I think it is supported as a
> squashfs decompressor.
> - bsdiff can generate diffs from binary files (be carefull if diffing
> compressed files, i.e.: gzip --rsyncable).
> - there's also casync (never tried it): https://github.com/systemd/casync/

FWIW I have preliminary recipes for casync and zstd here, but I'm having 
trouble with some native dependencies not being populated properly when 
casync-native is pulled in by other recipes (or -c addto_recipe_sysroot), so 
it still needs work:

http://cgit.openembedded.org/meta-openembedded-contrib/log/?h=paule/casync

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: minimize size of SW update bundle

[Patrick Ohly]

On Tue, 2017-10-17 at 16:12 +0300, Robert Berger wrote:
> Hi,
> 
> Imagine you have only a limited bandwidth to your devices available
> for 
> SW update. So you would like to minimize what's uploaded (over the
> air) 
> to your devices.
> 
> How would you do this with yocto?
> 
> [1] and [2] might be able to do incremental updates,

Both can indeed do it.

> Like how would you get a diff between two release builds...
> 
> [1] https://wiki.yoctoproject.org/wiki/Meta-swupd

With meta-swupd, set SWUPD_DELTAPACK_VERSIONS to generate binary diffs
against certain older releases. In addition, a single archive gets
created with just the right content for updating from the older
releases to the current one (= better compression, less GET requests).

> [2] https://github.com/advancedtelematic/meta-updater

That's basically OSTree. See https://ostree.readthedocs.io/en/latest/ma
nual/repository-management/#derived-data-static-deltas-and-the-summary-
file for instructions on generating deltas.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.