[meta-swupd][PATCH] bsdiff: update to latest version

[meta-swupd][PATCH] bsdiff: update to latest version

[André Draszik]

From: André Draszik <adraszik@tycoint.com>

This allows us to completely remove the build time
depenency on libcheck when not needed, reducing
overall build time, and in addition tests can be
converted into a PACKAGECONFIG to enable them if
needed.

Signed-off-by: André Draszik <adraszik@tycoint.com>
---
 recipes-devtools/bsdiff/bsdiff_git.bb | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/recipes-devtools/bsdiff/bsdiff_git.bb b/recipes-devtools/bsdiff/bsdiff_git.bb
index 8eb7271..ca0c4c8 100644
--- a/recipes-devtools/bsdiff/bsdiff_git.bb
+++ b/recipes-devtools/bsdiff/bsdiff_git.bb
@@ -6,14 +6,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=0dbe7a50f028269750631fcbded3846a"
 SRC_URI = "git://github.com/clearlinux/bsdiff.git;protocol=https"
 
 PV = "1.0.1+git${SRCPV}"
-SRCREV = "8c0a87b7c9eb5b22ed4e03a4eb42b32bd390df14"
+SRCREV = "fb5ced7c2cd6aeab0231f0cc3dee6bef72ddfb1e"
 
 S = "${WORKDIR}/git"
 
-DEPENDS = "xz bzip2 zlib libcheck"
+DEPENDS = "xz bzip2 zlib"
 
 inherit pkgconfig autotools
 
-EXTRA_OECONF = "--disable-tests"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[tests] = "--enable-tests, --disable-tests, libcheck"
 
 BBCLASSEXTEND = "native"
-- 
2.10.2

Re: [meta-swupd][PATCH] bsdiff: update to latest version

[Patrick Ohly]

On Mon, 2016-11-21 at 12:03 +0000, André Draszik wrote:
> This allows us to completely remove the build time
> depenency on libcheck when not needed, reducing
> overall build time, and in addition tests can be
> converted into a PACKAGECONFIG to enable them if
> needed.

+1

Sorry for the delay, I had to check with Joshua first who's going to
merge your patches. I'm currently working on a major update of
meta-swupd (see https://github.com/ostroproject/ostro-os/pull/198) and
if there's enough interest for using it as part of Yocto, might continue
maintaining it.

I'm going to stage your patches in that branch and after one round of
testing, will also push them to meta-swupd master.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: [meta-swupd][PATCH] bsdiff: update to latest version

[André Draszik]

On Wed, 2016-11-30 at 12:04 +0100, Patrick Ohly wrote:
> On Mon, 2016-11-21 at 12:03 +0000, André Draszik wrote:
> > This allows us to completely remove the build time
> > depenency on libcheck when not needed, reducing
> > overall build time, and in addition tests can be
> > converted into a PACKAGECONFIG to enable them if
> > needed.
> 
> +1
> 
> Sorry for the delay, I had to check with Joshua first who's going to
> merge your patches. I'm currently working on a major update of
> meta-swupd (see https://github.com/ostroproject/ostro-os/pull/198) and
> if there's enough interest for using it as part of Yocto, might continue
> maintaining it.

Thanks Patrick. Are you saying you would otherwise abandon meta-swupd
completely, or have it be a part of ostro-os only?
swupd via meta-swupd seems to be a very good addition to yocto.

Thanks,
Andre'

update mechanisms (was: Re: [meta-swupd][PATCH] bsdiff: update to latest version)

[Patrick Ohly]

On Wed, 2016-11-30 at 14:31 +0000, André Draszik wrote:
> On Wed, 2016-11-30 at 12:04 +0100, Patrick Ohly wrote:
> > On Mon, 2016-11-21 at 12:03 +0000, André Draszik wrote:
> > > This allows us to completely remove the build time
> > > depenency on libcheck when not needed, reducing
> > > overall build time, and in addition tests can be
> > > converted into a PACKAGECONFIG to enable them if
> > > needed.
> > 
> > +1
> > 
> > Sorry for the delay, I had to check with Joshua first who's going to
> > merge your patches. I'm currently working on a major update of
> > meta-swupd (see https://github.com/ostroproject/ostro-os/pull/198) and
> > if there's enough interest for using it as part of Yocto, might continue
> > maintaining it.
> 
> Thanks Patrick. Are you saying you would otherwise abandon meta-swupd
> completely, or have it be a part of ostro-os only?

That's undecided. We are currently trying to figure out which update
mechanism is a good fit for Yocto. Depending on the outcome of that and
available resources, we may or may not have the time to support
something.

I've started a Wiki page
https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
moment, but might as well be mentioned already now.

Perhaps you can say a few words about your usage of swupd and what you
liked or didn't like about it? Note that performance will become better
with the upcoming update.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms (was: Re: [meta-swupd][PATCH] bsdiff: update to latest version)

[André Draszik]

On Wed, 2016-11-30 at 15:59 +0100, Patrick Ohly wrote:
> On Wed, 2016-11-30 at 14:31 +0000, André Draszik wrote:
> > On Wed, 2016-11-30 at 12:04 +0100, Patrick Ohly wrote:
> > > On Mon, 2016-11-21 at 12:03 +0000, André Draszik wrote:
> > > > This allows us to completely remove the build time
> > > > depenency on libcheck when not needed, reducing
> > > > overall build time, and in addition tests can be
> > > > converted into a PACKAGECONFIG to enable them if
> > > > needed.
> > > 
> > > +1
> > > 
> > > Sorry for the delay, I had to check with Joshua first who's going to
> > > merge your patches. I'm currently working on a major update of
> > > meta-swupd (see https://github.com/ostroproject/ostro-os/pull/198) and
> > > if there's enough interest for using it as part of Yocto, might
> > > continue
> > > maintaining it.
> > 
> > Thanks Patrick. Are you saying you would otherwise abandon meta-swupd
> > completely, or have it be a part of ostro-os only?
> 
> That's undecided. We are currently trying to figure out which update
> mechanism is a good fit for Yocto. Depending on the outcome of that and
> available resources, we may or may not have the time to support
> something.
> 
> I've started a Wiki page
> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> moment, but might as well be mentioned already now.
> 
> Perhaps you can say a few words about your usage of swupd and what you
> liked or didn't like about it? Note that performance will become better
> with the upcoming update.

In the wiki or here? I'll be out for the next week, but when I come back I
will write some more.

BTW, you probably know this: 
https://lists.linuxfoundation.org/pipermail/automotive-discussions/2016-May/002061.html

I liked swupd for its ability to be used both for initial provisioning as
well as for incremental updates. The latter being important when you have
*loads* of devices, where it doesn't seem to make sense to download a full
image for a tiny change to each device (think cellular!), the former
streamlining the update mechanism, so that not two different independent
things have to be maintained.


Cheers,
Andre'

Re: update mechanisms (was: Re: [meta-swupd][PATCH] bsdiff: update to latest version)

[Patrick Ohly]

On Wed, 2016-11-30 at 17:19 +0000, André Draszik wrote:
> On Wed, 2016-11-30 at 15:59 +0100, Patrick Ohly wrote:
> > I've started a Wiki page
> > https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> > moment, but might as well be mentioned already now.
> > 
> > Perhaps you can say a few words about your usage of swupd and what you
> > liked or didn't like about it? Note that performance will become better
> > with the upcoming update.
> 
> In the wiki or here? I'll be out for the next week, but when I come back I
> will write some more.

Let's discuss here and if necessary, summarize in the wiki.

> BTW, you probably know this: 
> https://lists.linuxfoundation.org/pipermail/automotive-discussions/2016-May/002061.html

I might have seen it at some point, but I hadn't paid that much
attention then and it's a good, timely reminder. I'll link to it from
the Wiki page.

> I liked swupd for its ability to be used both for initial provisioning

You mean installing from the update repository? That's something that
Clear Linux OS can do with their installer, but nothing like that has
been tried with a Yocto-based build. That doesn't mean that it can't be
done, it's just work.

>  as
> well as for incremental updates. The latter being important when you have
> *loads* of devices, where it doesn't seem to make sense to download a full
> image for a tiny change to each device (think cellular!),

That's indeed one of the strengths of swupd. OSTree comes close in terms
of some key characteristics (file-based, persistent /etc and /var). It
would be interesting to know how efficient updating via OSTree is.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms (was: Re: [meta-swupd][PATCH] bsdiff: update to latest version)

[André Draszik]

On Thu, 2016-12-01 at 08:42 +0100, Patrick Ohly wrote:
> On Wed, 2016-11-30 at 17:19 +0000, André Draszik wrote:
> > I liked swupd for its ability to be used both for initial provisioning
> 
> You mean installing from the update repository? That's something that
> Clear Linux OS can do with their installer, but nothing like that has
> been tried with a Yocto-based build. That doesn't mean that it can't be
> done, it's just work.

Yes. In our case we can only provision the NOR flash in the factory (which
is too small for the real file system), so I have the swupd-client inside a
small initramfs in NOR flash, and from there I can provision NAND flash
using swupd verify -i

All yocto based, but in its current state not in a generally useful shape
for a wider audience.

> >  as
> > well as for incremental updates. The latter being important when you
> > have
> > *loads* of devices, where it doesn't seem to make sense to download a
> > full
> > image for a tiny change to each device (think cellular!),
> 
> That's indeed one of the strengths of swupd. OSTree comes close in terms
> of some key characteristics (file-based, persistent /etc and /var). It
> would be interesting to know how efficient updating via OSTree is.

True.


A.

Re: update mechanisms (was: Re: [meta-swupd][PATCH] bsdiff: update to latest version)

[Patrick Ohly]

On Thu, 2016-12-01 at 10:26 +0000, André Draszik wrote:
> On Thu, 2016-12-01 at 08:42 +0100, Patrick Ohly wrote:
> > On Wed, 2016-11-30 at 17:19 +0000, André Draszik wrote:
> > > I liked swupd for its ability to be used both for initial provisioning
> > 
> > You mean installing from the update repository? That's something that
> > Clear Linux OS can do with their installer, but nothing like that has
> > been tried with a Yocto-based build. That doesn't mean that it can't be
> > done, it's just work.
> 
> Yes. In our case we can only provision the NOR flash in the factory (which
> is too small for the real file system), so I have the swupd-client inside a
> small initramfs in NOR flash, and from there I can provision NAND flash
> using swupd verify -i

Interesting, I hadn't thought of using it like that. When doing this,
does it download the "from-0" pack files?

"swupd bundle-add" uses those; I'm less sure about verify. It would have
to detect that it misses all files from the os-core bundle and then as a
special optimization get the pack file instead of individual files.

Speaking of bundles, is that concept something that you find useful for
your purposes? It's not strictly needed for a pure system update
mechanism.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Stefano Babic]

Hi Patrick,

On 30/11/2016 15:59, Patrick Ohly wrote:
> On Wed, 2016-11-30 at 14:31 +0000, André Draszik wrote:
>> On Wed, 2016-11-30 at 12:04 +0100, Patrick Ohly wrote:
>>> On Mon, 2016-11-21 at 12:03 +0000, André Draszik wrote:
>>>> This allows us to completely remove the build time
>>>> depenency on libcheck when not needed, reducing
>>>> overall build time, and in addition tests can be
>>>> converted into a PACKAGECONFIG to enable them if
>>>> needed.
>>>
>>> +1
>>>
>>> Sorry for the delay, I had to check with Joshua first who's going to
>>> merge your patches. I'm currently working on a major update of
>>> meta-swupd (see https://github.com/ostroproject/ostro-os/pull/198) and
>>> if there's enough interest for using it as part of Yocto, might continue
>>> maintaining it.
>>
>> Thanks Patrick. Are you saying you would otherwise abandon meta-swupd
>> completely, or have it be a part of ostro-os only?
> 
> That's undecided. We are currently trying to figure out which update
> mechanism is a good fit for Yocto. Depending on the outcome of that and
> available resources, we may or may not have the time to support
> something.
> 
> I've started a Wiki page
> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> moment, but might as well be mentioned already now.

I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
like to edit for better explanation on some parts. Should I try to edit
directly the page or is it better to discuss it here ?

Regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: update mechanisms

[Patrick Ohly]

On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
> Hi Patrick,
> 
> On 30/11/2016 15:59, Patrick Ohly wrote:
> > I've started a Wiki page
> > https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> > moment, but might as well be mentioned already now.
> 
> I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
> like to edit for better explanation on some parts. Should I try to edit
> directly the page or is it better to discuss it here ?

Use your own judgment. If its uncontroversial, the feel free to edit the
page directly, otherwise let's discuss it here.

If feel that putting information directly into the table is too limiting
(it should be brief), then feel free to start a complete section about
SWUpdate. 

I'll do the same for swupd. Editing the sections should be possible
without conflicts, we just have to be more careful about editing the
table concurrently.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Lopez, Mariano]

On 12/6/2016 3:45 AM, Patrick Ohly wrote:
> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
>> Hi Patrick,
>>
>> On 30/11/2016 15:59, Patrick Ohly wrote:
>>> I've started a Wiki page
>>> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
>>> moment, but might as well be mentioned already now.
>> I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
>> like to edit for better explanation on some parts. Should I try to edit
>> directly the page or is it better to discuss it here ?

It will be interesting to have your input directly, I tried to fill the 
table with my experience with SWUpdate but I think you could give a 
better insight of this. Thanks for joining the effort.

> Use your own judgment. If its uncontroversial, the feel free to edit the
> page directly, otherwise let's discuss it here.
>
> If feel that putting information directly into the table is too limiting
> (it should be brief), then feel free to start a complete section about
> SWUpdate.
>
> I'll do the same for swupd. Editing the sections should be possible
> without conflicts, we just have to be more careful about editing the
> table concurrently.
>

I agree with you, that we need to have a section per project, the table 
is too limiting.

Mariano

Re: update mechanisms

[Philip Balister]

On 12/06/2016 06:11 AM, Lopez, Mariano wrote:
> 
> 
> On 12/6/2016 3:45 AM, Patrick Ohly wrote:
>> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
>>> Hi Patrick,
>>>
>>> On 30/11/2016 15:59, Patrick Ohly wrote:
>>>> I've started a Wiki page
>>>> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
>>>> moment, but might as well be mentioned already now.
>>> I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
>>> like to edit for better explanation on some parts. Should I try to edit
>>> directly the page or is it better to discuss it here ?
> 
> It will be interesting to have your input directly, I tried to fill the
> table with my experience with SWUpdate but I think you could give a
> better insight of this. Thanks for joining the effort.
> 
>> Use your own judgment. If its uncontroversial, the feel free to edit the
>> page directly, otherwise let's discuss it here.
>>
>> If feel that putting information directly into the table is too limiting
>> (it should be brief), then feel free to start a complete section about
>> SWUpdate.
>>
>> I'll do the same for swupd. Editing the sections should be possible
>> without conflicts, we just have to be more careful about editing the
>> table concurrently.
>>
> 
> I agree with you, that we need to have a section per project, the table
> is too limiting.

I just skimmed the page and the table format isn't working well with the
length of the blocks of text.

I'd also suggest referencing Matt Porter's talk from ELCE 2016 about
software update technology:

http://elinux.org/images/3/31/Comparison_of_Linux_Software_Update_Technologies.pdf

https://youtu.be/pdHV9H9nZks?list=PLbzoR-pLrL6pRFP6SOywVJWdEHlmQE51q

Philip


> 
> Mariano

Re: update mechanisms

[Stefano Babic]

Hi Philip,

On 06/12/2016 19:45, Philip Balister wrote:
> On 12/06/2016 06:11 AM, Lopez, Mariano wrote:
>>
>>
>> On 12/6/2016 3:45 AM, Patrick Ohly wrote:
>>> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
>>>> Hi Patrick,
>>>>
>>>> On 30/11/2016 15:59, Patrick Ohly wrote:
>>>>> I've started a Wiki page
>>>>> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
>>>>> moment, but might as well be mentioned already now.
>>>> I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
>>>> like to edit for better explanation on some parts. Should I try to edit
>>>> directly the page or is it better to discuss it here ?
>>
>> It will be interesting to have your input directly, I tried to fill the
>> table with my experience with SWUpdate but I think you could give a
>> better insight of this. Thanks for joining the effort.
>>
>>> Use your own judgment. If its uncontroversial, the feel free to edit the
>>> page directly, otherwise let's discuss it here.
>>>
>>> If feel that putting information directly into the table is too limiting
>>> (it should be brief), then feel free to start a complete section about
>>> SWUpdate.
>>>
>>> I'll do the same for swupd. Editing the sections should be possible
>>> without conflicts, we just have to be more careful about editing the
>>> table concurrently.
>>>
>>
>> I agree with you, that we need to have a section per project, the table
>> is too limiting.
> 
> I just skimmed the page and the table format isn't working well with the
> length of the blocks of text.
> 

Agree - I think it is better to have a section per project.

> I'd also suggest referencing Matt Porter's talk from ELCE 2016 about
> software update technology:
> 
> http://elinux.org/images/3/31/Comparison_of_Linux_Software_Update_Technologies.pdf
> 
> https://youtu.be/pdHV9H9nZks?list=PLbzoR-pLrL6pRFP6SOywVJWdEHlmQE51q
> 

I have added references to the talks at last ELCE from Matt, Chris and
Silvano.

Stefano


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: update mechanisms

[Kristian Amlie]

On 06/12/16 23:38, Stefano Babic wrote:
> On 06/12/2016 19:45, Philip Balister wrote:
>> I just skimmed the page and the table format isn't working well with the
>> length of the blocks of text.
>>
> 
> Agree - I think it is better to have a section per project.

And now I added yet another one, Mender, to
https://wiki.yoctoproject.org/wiki/System_Update.

-- 
Kristian

Re: update mechanisms

[Patrick Ohly]

Hello everyone!

Thanks for contributing directly to the page. It's great to see this
done collaboratively.

On Tue, 2016-12-06 at 23:38 +0100, Stefano Babic wrote:
> Hi Philip,
> 
> On 06/12/2016 19:45, Philip Balister wrote:
> > On 12/06/2016 06:11 AM, Lopez, Mariano wrote:
> >> I agree with you, that we need to have a section per project, the table
> >> is too limiting.
> > 
> > I just skimmed the page and the table format isn't working well with the
> > length of the blocks of text.
> > 
> 
> Agree - I think it is better to have a section per project.

That always has been the goal, and I now updated the page accordingly. I
tried to summarize the key aspects of each mechanism in the table
itself. That's something that I haven't seen elsewhere and something
that the page can  I tried to be as fair and objective as possible,
please shout if I messed something up or you don't agree with my
summary.

In particular the "complexity" column is a bit subjective. Stefano, I
hope you don't mind that I did not quite buy the "easy to use"
characterization of swupdate ;-) For a system that is as flexible as
swupdate, I'd expect a more difficult learning curve and some need to
customize the system before using it, so I added a "(but requires
customization!?)" comment to that statement.

Kristian, I added a comment that Mender requires U-Boot. That is
correct, isn't it? I understand that this restriction allows you to
implement things like automatic rollback, but it's worth mentioning that
this comes with that limitation.


-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Stefano Babic]

Hi Patrick,

On 09/12/2016 16:13, Patrick Ohly wrote:
> Hello everyone!
> 
> Thanks for contributing directly to the page. It's great to see this
> done collaboratively.
> 
> On Tue, 2016-12-06 at 23:38 +0100, Stefano Babic wrote:
>> Hi Philip,
>>
>> On 06/12/2016 19:45, Philip Balister wrote:
>>> On 12/06/2016 06:11 AM, Lopez, Mariano wrote:
>>>> I agree with you, that we need to have a section per project, the table
>>>> is too limiting.
>>>
>>> I just skimmed the page and the table format isn't working well with the
>>> length of the blocks of text.
>>>
>>
>> Agree - I think it is better to have a section per project.
> 
> That always has been the goal, and I now updated the page accordingly.

Page is really better now :-)

> I
> tried to summarize the key aspects of each mechanism in the table
> itself. That's something that I haven't seen elsewhere and something
> that the page can  I tried to be as fair and objective as possible,
> please shout if I messed something up or you don't agree with my
> summary.
> 
> In particular the "complexity" column is a bit subjective. Stefano, I
> hope you don't mind that I did not quite buy the "easy to use"
> characterization of swupdate ;-)

No worry...and I have not written myself. It was inserted by Mariano, so
it looks like that swupdate at least for Mariano is "easy to use".
I think it is correct to point out that customization is required.

> For a system that is as flexible as
> swupdate, I'd expect a more difficult learning curve and some need to
> customize the system before using it, so I added a "(but requires
> customization!?)" comment to that statement.

Agree.

> 
> Kristian, I added a comment that Mender requires U-Boot. That is
> correct, isn't it? I understand that this restriction allows you to
> implement things like automatic rollback, but it's worth mentioning that
> this comes with that limitation.

Best regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: update mechanisms

[Mariano Lopez]

>> I
>> tried to summarize the key aspects of each mechanism in the table
>> itself. That's something that I haven't seen elsewhere and something
>> that the page can  I tried to be as fair and objective as possible,
>> please shout if I messed something up or you don't agree with my
>> summary.
>>
>> In particular the "complexity" column is a bit subjective. Stefano, I
>> hope you don't mind that I did not quite buy the "easy to use"
>> characterization of swupdate ;-)
> No worry...and I have not written myself. It was inserted by Mariano, so
> it looks like that swupdate at least for Mariano is "easy to use".
> I think it is correct to point out that customization is required.

Compared to other update mechanism that I tested it was the easier to
implement.

Re: update mechanisms

[Patrick Ohly]

On Mon, 2016-12-12 at 08:59 -0600, Mariano Lopez wrote:
> >> In particular the "complexity" column is a bit subjective. Stefano, I
> >> hope you don't mind that I did not quite buy the "easy to use"
> >> characterization of swupdate ;-)
> > No worry...and I have not written myself. It was inserted by Mariano, so
> > it looks like that swupdate at least for Mariano is "easy to use".
> > I think it is correct to point out that customization is required.
> 
> Compared to other update mechanism that I tested it was the easier to
> implement.

Which "getting started" document or presentation were you using? The
documentation for mender (https://docs.mender.io/) is very
straight-forward (partly of course because it doesn't need to cover many
variations), while for swupdate
(http://sbabic.github.io/swupdate/swupdate.html) I found it less clear
how to begin.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Mariano Lopez]

On 12/12/16 09:41, Patrick Ohly wrote:
> On Mon, 2016-12-12 at 08:59 -0600, Mariano Lopez wrote:
>>>> In particular the "complexity" column is a bit subjective. Stefano, I
>>>> hope you don't mind that I did not quite buy the "easy to use"
>>>> characterization of swupdate ;-)
>>> No worry...and I have not written myself. It was inserted by Mariano, so
>>> it looks like that swupdate at least for Mariano is "easy to use".
>>> I think it is correct to point out that customization is required.
>> Compared to other update mechanism that I tested it was the easier to
>> implement.
> Which "getting started" document or presentation were you using? The
> documentation for mender (https://docs.mender.io/) is very
> straight-forward (partly of course because it doesn't need to cover many
> variations), while for swupdate
> (http://sbabic.github.io/swupdate/swupdate.html) I found it less clear
> how to begin.
>

When I did a research in update mechanism, mender wasn't yet available,
and indeed it seems very straight forward (need to test it before final
veredict). But if you compare SWUpdate, swupd, and OSTree; SWUpdate is
by far less complex than the other two

Re: update mechanisms

[Patrick Ohly]

On Mon, 2016-12-12 at 09:49 -0600, Mariano Lopez wrote:
> 
> On 12/12/16 09:41, Patrick Ohly wrote:
> > On Mon, 2016-12-12 at 08:59 -0600, Mariano Lopez wrote:
> >>>> In particular the "complexity" column is a bit subjective. Stefano, I
> >>>> hope you don't mind that I did not quite buy the "easy to use"
> >>>> characterization of swupdate ;-)
> >>> No worry...and I have not written myself. It was inserted by Mariano, so
> >>> it looks like that swupdate at least for Mariano is "easy to use".
> >>> I think it is correct to point out that customization is required.
> >> Compared to other update mechanism that I tested it was the easier to
> >> implement.
> > Which "getting started" document or presentation were you using? The
> > documentation for mender (https://docs.mender.io/) is very
> > straight-forward (partly of course because it doesn't need to cover many
> > variations), while for swupdate
> > (http://sbabic.github.io/swupdate/swupdate.html) I found it less clear
> > how to begin.
> >
> 
> When I did a research in update mechanism, mender wasn't yet available,
> and indeed it seems very straight forward (need to test it before final
> veredict). But if you compare SWUpdate, swupd, and OSTree; SWUpdate is
> by far less complex than the other two

Ease-of-use is not necessarily determined by the complexity. Good
integration and documentation can go a long way towards making a complex
solution easy to use - when sticking to the pre-defined usage patterns.

The opposite is also true: a simple solution may be hard to use if all
one gets is the source code and one first has to reverse-engineer the
usage model.

I agree that the complexity is roughly swupdate < ostree < swupd and
there's also no doubt that the latter two aren't easy to use (mostly due
to lacking documentation and integration), but I'm still not sure what
documentation the "easy to use" verdict for swupdate is based on.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Lopez, Mariano]

On 12/12/2016 1:02 PM, Patrick Ohly wrote:
> On Mon, 2016-12-12 at 09:49 -0600, Mariano Lopez wrote:
>> On 12/12/16 09:41, Patrick Ohly wrote:
>>> On Mon, 2016-12-12 at 08:59 -0600, Mariano Lopez wrote:
>>>>>> In particular the "complexity" column is a bit subjective. Stefano, I
>>>>>> hope you don't mind that I did not quite buy the "easy to use"
>>>>>> characterization of swupdate ;-)
>>>>> No worry...and I have not written myself. It was inserted by Mariano, so
>>>>> it looks like that swupdate at least for Mariano is "easy to use".
>>>>> I think it is correct to point out that customization is required.
>>>> Compared to other update mechanism that I tested it was the easier to
>>>> implement.
>>> Which "getting started" document or presentation were you using? The
>>> documentation for mender (https://docs.mender.io/) is very
>>> straight-forward (partly of course because it doesn't need to cover many
>>> variations), while for swupdate
>>> (http://sbabic.github.io/swupdate/swupdate.html) I found it less clear
>>> how to begin.
>>>
>> When I did a research in update mechanism, mender wasn't yet available,
>> and indeed it seems very straight forward (need to test it before final
>> veredict). But if you compare SWUpdate, swupd, and OSTree; SWUpdate is
>> by far less complex than the other two
> Ease-of-use is not necessarily determined by the complexity. Good
> integration and documentation can go a long way towards making a complex
> solution easy to use - when sticking to the pre-defined usage patterns.
>
> The opposite is also true: a simple solution may be hard to use if all
> one gets is the source code and one first has to reverse-engineer the
> usage model.

Completely agree with you Patrick, it could be a simple solution very 
hard to use because of the lack of documentation

>
> I agree that the complexity is roughly swupdate < ostree < swupd and
> there's also no doubt that the latter two aren't easy to use (mostly due
> to lacking documentation and integration), but I'm still not sure what
> documentation the "easy to use" verdict for swupdate is based on.
>

I based my verdict using this source:
https://sbabic.github.io/swupdate/
You can follow this and create your update image few hours.

Cheers
Mariano

Re: update mechanisms

[Kristian Amlie]

On 09/12/16 16:13, Patrick Ohly wrote:
> Kristian, I added a comment that Mender requires U-Boot. That is
> correct, isn't it? I understand that this restriction allows you to
> implement things like automatic rollback, but it's worth mentioning that
> this comes with that limitation.

Yes, that is correct, I should have added it the first time around.

-- 
Kristian

Re: update mechanisms

[Eystein Måløy Stenberg]

On Tue, 2016-12-06 at 10:45 +0100, Patrick Ohly wrote:
> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
> > Hi Patrick,
> >
> > On 30/11/2016 15:59, Patrick Ohly wrote:
> > > I've started a Wiki page
> > > https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> > > moment, but might as well be mentioned already now.
> >
> > I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
> > like to edit for better explanation on some parts. Should I try to edit
> > directly the page or is it better to discuss it here ?
>
> Use your own judgment. If its uncontroversial, the feel free to edit the
> page directly, otherwise let's discuss it here.
>
> If feel that putting information directly into the table is too limiting
> (it should be brief), then feel free to start a complete section about
> SWUpdate.
>
> I'll do the same for swupd. Editing the sections should be possible
> without conflicts, we just have to be more careful about editing the
> table concurrently.

I updated the Mender part of the wiki now that the stable version Mender 
1.0 is released. These changes should not be controversial, but let me 
know if you disagree. We are planning to keep the Mender section 
up-to-date as we release new versions, as I think this is what you expect.

Are there any plans for next steps or is the wiki the "final state" in 
terms of integrating OTA updates in Yocto/OE?


-- 

Eystein

Re: update mechanisms

[Patrick Ohly]

On Wed, 2017-03-01 at 16:35 -0800, Eystein Måløy Stenberg wrote:
> On Tue, 2016-12-06 at 10:45 +0100, Patrick Ohly wrote:
> > On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
> > > Hi Patrick,
> > >
> > > On 30/11/2016 15:59, Patrick Ohly wrote:
> > > > I've started a Wiki page
> > > > https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> > > > moment, but might as well be mentioned already now.
> > >
> > > I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
> > > like to edit for better explanation on some parts. Should I try to edit
> > > directly the page or is it better to discuss it here ?
> >
> > Use your own judgment. If its uncontroversial, the feel free to edit the
> > page directly, otherwise let's discuss it here.
> >
> > If feel that putting information directly into the table is too limiting
> > (it should be brief), then feel free to start a complete section about
> > SWUpdate.
> >
> > I'll do the same for swupd. Editing the sections should be possible
> > without conflicts, we just have to be more careful about editing the
> > table concurrently.
> 
> I updated the Mender part of the wiki now that the stable version Mender 
> 1.0 is released. These changes should not be controversial, but let me 
> know if you disagree. We are planning to keep the Mender section 
> up-to-date as we release new versions, as I think this is what you expect.

Yes, that's useful.

> Are there any plans for next steps or is the wiki the "final state" in 
> terms of integrating OTA updates in Yocto/OE?

My own conclusion is that it is impossible to integrate a specific OTA
update into Yocto/OE (because there's no single solution that fits all
requirements) and/or it would be unfair to those solutions that don't
get such special testing. In that sense the Wiki page is the final
result of the investigation. Anyone interested in picking a solution can
go there, consider the pros and cons, and then make a choice.

However, I see room for some collaborative work that then can happen in
Yocto/OE:
      * carrying local system configuration changes across system
        updates: I find it promising to investigate the "stateless"
        concept and have started some exploratory work, see
        https://wiki.yoctoproject.org/wiki/Stateless#Status_and_goals_for_.22stateless.22_in_Yocto (more on that soon)
      * supporting UEFI-based machines

In contrast to Yocto/OE, the IoT Reference OS Kit can (and in a way, has
to) make some choices because it needs a functional solution out of the
box. My current thinking there is to support one file-based (OSTree?)
and one block-based (Mender.io?) solution, the block-based one perhaps
combined with dm-verity.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Kristian Amlie]

On 10/03/17 14:02, Patrick Ohly wrote:
> On Wed, 2017-03-01 at 16:35 -0800, Eystein Måløy Stenberg wrote:
>> On Tue, 2016-12-06 at 10:45 +0100, Patrick Ohly wrote:
>>> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
>>>> Hi Patrick,
>>>>
>>>> On 30/11/2016 15:59, Patrick Ohly wrote:
>>>>> I've started a Wiki page
>>>>> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
>>>>> moment, but might as well be mentioned already now.
>>>>
>>>> I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
>>>> like to edit for better explanation on some parts. Should I try to edit
>>>> directly the page or is it better to discuss it here ?
>>>
>>> Use your own judgment. If its uncontroversial, the feel free to edit the
>>> page directly, otherwise let's discuss it here.
>>>
>>> If feel that putting information directly into the table is too limiting
>>> (it should be brief), then feel free to start a complete section about
>>> SWUpdate.
>>>
>>> I'll do the same for swupd. Editing the sections should be possible
>>> without conflicts, we just have to be more careful about editing the
>>> table concurrently.
>>
>> I updated the Mender part of the wiki now that the stable version Mender 
>> 1.0 is released. These changes should not be controversial, but let me 
>> know if you disagree. We are planning to keep the Mender section 
>> up-to-date as we release new versions, as I think this is what you expect.
> 
> Yes, that's useful.
> 
>> Are there any plans for next steps or is the wiki the "final state" in 
>> terms of integrating OTA updates in Yocto/OE?
> 
> My own conclusion is that it is impossible to integrate a specific OTA
> update into Yocto/OE (because there's no single solution that fits all
> requirements) and/or it would be unfair to those solutions that don't
> get such special testing. In that sense the Wiki page is the final
> result of the investigation. Anyone interested in picking a solution can
> go there, consider the pros and cons, and then make a choice.

Makes sense. We can always revisit this at a later point, if one method
starts to emerge as the preferred option for most people.

> However, I see room for some collaborative work that then can happen in
> Yocto/OE:
>       * carrying local system configuration changes across system
>         updates: I find it promising to investigate the "stateless"
>         concept and have started some exploratory work, see
>         https://wiki.yoctoproject.org/wiki/Stateless#Status_and_goals_for_.22stateless.22_in_Yocto (more on that soon)

What's the relation (if any) between this and a read-only rootfs?

Also this may be only vaguely related, but I have it in my queue to
finish the built-in partitioning of rootfs images [1], which will help
divide the filesystem into a stateful and a stateless part. The wic part
is done [2], but ext4 images are not covered yet.

[1] It works by choosing which sub directories of the complete rootfs
you want on each partition.

[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=10712

>       * supporting UEFI-based machines

Great, this is something we are interested in as well!

-- 
Kristian

Re: update mechanisms

[Patrick Ohly]

On Fri, 2017-03-10 at 14:35 +0100, Kristian Amlie wrote:
> On 10/03/17 14:02, Patrick Ohly wrote:
> > On Wed, 2017-03-01 at 16:35 -0800, Eystein Måløy Stenberg wrote:
> >> On Tue, 2016-12-06 at 10:45 +0100, Patrick Ohly wrote:
> >>> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
> >>>> Hi Patrick,
> >>>>
> >>>> On 30/11/2016 15:59, Patrick Ohly wrote:
> >>>>> I've started a Wiki page
> >>>>> https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> >>>>> moment, but might as well be mentioned already now.
> >>>>
> >>>> I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
> >>>> like to edit for better explanation on some parts. Should I try to edit
> >>>> directly the page or is it better to discuss it here ?
> >>>
> >>> Use your own judgment. If its uncontroversial, the feel free to edit the
> >>> page directly, otherwise let's discuss it here.
> >>>
> >>> If feel that putting information directly into the table is too limiting
> >>> (it should be brief), then feel free to start a complete section about
> >>> SWUpdate.
> >>>
> >>> I'll do the same for swupd. Editing the sections should be possible
> >>> without conflicts, we just have to be more careful about editing the
> >>> table concurrently.
> >>
> >> I updated the Mender part of the wiki now that the stable version Mender 
> >> 1.0 is released. These changes should not be controversial, but let me 
> >> know if you disagree. We are planning to keep the Mender section 
> >> up-to-date as we release new versions, as I think this is what you expect.
> > 
> > Yes, that's useful.
> > 
> >> Are there any plans for next steps or is the wiki the "final state" in 
> >> terms of integrating OTA updates in Yocto/OE?
> > 
> > My own conclusion is that it is impossible to integrate a specific OTA
> > update into Yocto/OE (because there's no single solution that fits all
> > requirements) and/or it would be unfair to those solutions that don't
> > get such special testing. In that sense the Wiki page is the final
> > result of the investigation. Anyone interested in picking a solution can
> > go there, consider the pros and cons, and then make a choice.
> 
> Makes sense. We can always revisit this at a later point, if one method
> starts to emerge as the preferred option for most people.
> 
> > However, I see room for some collaborative work that then can happen in
> > Yocto/OE:
> >       * carrying local system configuration changes across system
> >         updates: I find it promising to investigate the "stateless"
> >         concept and have started some exploratory work, see
> >         https://wiki.yoctoproject.org/wiki/Stateless#Status_and_goals_for_.22stateless.22_in_Yocto (more on that soon)
> 
> What's the relation (if any) between this and a read-only rootfs?

/etc needs to be on the read/write partition. What "stateless" adds is
that there's no need to pre-populate that read/write part. That is done
during first boot, potentially even during each boot (no persistent
state at all).

In addition, factory reset can be done by simply wiping out /etc.
When /etc contains a mix of pre-populated files which are essential for
booting and local modifications which must be wiped out, that part
becomes harder.

In an ideal world, the OS would work entirely without files in /etc. We
don't live in that world (yet).

> Also this may be only vaguely related, but I have it in my queue to
> finish the built-in partitioning of rootfs images [1], which will help
> divide the filesystem into a stateful and a stateless part. The wic part
> is done [2], but ext4 images are not covered yet.

This is still useful, for example to pre-populate content than then
never gets touched anymore by the OS. I'm thinking of pre-installed apps
here, where app update/add/remove is done normally done independently of
the OS.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[Mike Looijmans]

On 09-12-16 16:13, Patrick Ohly wrote:
> Hello everyone!
>
> Thanks for contributing directly to the page. It's great to see this
> done collaboratively.

Nice informative page.

Only thing lacking is simple, OE-built in upgrade procedures like: "Just run 
'opkg update && opkg upgrade'"



Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Re: update mechanisms

[Patrick Ohly]

On Tue, 2016-12-13 at 09:51 +0100, Mike Looijmans wrote:
> On 09-12-16 16:13, Patrick Ohly wrote:
> > Hello everyone!
> >
> > Thanks for contributing directly to the page. It's great to see this
> > done collaboratively.
> 
> Nice informative page.
> 
> Only thing lacking is simple, OE-built in upgrade procedures like: "Just run 
> 'opkg update && opkg upgrade'"

Want to add it? :-)

Seriously, it would be a good addition.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Re: update mechanisms

[André Draszik]

Hi,

On Tue, 2016-12-06 at 10:45 +0100, Patrick Ohly wrote:
> On Tue, 2016-12-06 at 10:01 +0100, Stefano Babic wrote:
> > Hi Patrick,
> > 
> > On 30/11/2016 15:59, Patrick Ohly wrote:
> > > I've started a Wiki page
> > > https://wiki.yoctoproject.org/wiki/System_Update - rudimentary at the
> > > moment, but might as well be mentioned already now.
> > 
> > I have seen Mariano added an entry for SWUpdate, too, thanks  - I would
> > like to edit for better explanation on some parts. Should I try to edit
> > directly the page or is it better to discuss it here ?
> 
> Use your own judgment. If its uncontroversial, the feel free to edit the
> page directly, otherwise let's discuss it here.
> 
> If feel that putting information directly into the table is too limiting
> (it should be brief), then feel free to start a complete section about
> SWUpdate. 
> 
> I'll do the same for swupd. Editing the sections should be possible
> without conflicts, we just have to be more careful about editing the
> table concurrently.

It looks as if some highlights about swupdate can equally be said about
swupd:

- dual copy is supported
- my minimal swupd-based rescue initramfs is around 4MB


Cheers,
Andre'

Re: update mechanisms

[Patrick Ohly]

On Mon, 2016-12-12 at 15:13 +0000, André Draszik wrote:
> Hi,
> 
> On Tue, 2016-12-06 at 10:45 +0100, Patrick Ohly wrote: 
> > I'll do the same for swupd. Editing the sections should be possible
> > without conflicts, we just have to be more careful about editing the
> > table concurrently.
> 
> It looks as if some highlights about swupdate can equally be said about
> swupd:
> 
> - dual copy is supported
> - my minimal swupd-based rescue initramfs is around 4MB

swupdate has support for a "dual copy
strategy" (http://sbabic.github.io/swupdate/swupdate.html#software-collections) while out-of-the-box (i.e. with what is currently available) meta-swupd and swupd itself don't. So I think it is correct to say that swupdate has some (implementation) advantage here.

The "could be extended to do updates without that risk" in the
"swupd/Failure resilience" section was meant to include a dual-copy
approach. Should that be rephrased to be more explicit? I was thinking
of several possible scenarios:
      * single partition: stage files, stop services, update, restart
        services or reboot
      * dual partition: update inactive partition, swap partitions,
        reboot

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.