Deploying Yocto build images

Deploying Yocto build images

[Stefano Babic]

Hi everybody,

in the last ELCE, David point out in his presentation that we should
improve how to deploy Yocto images on the target.

I did some work this year to provide a reliable way for some customers
of us to install Yocto's images in field, and I have published last week
the sources. Here the link of the announcement:

	http://comments.gmane.org/gmane.comp.embedded.eldk/2397

Mainly, it is a tool that can be stored in the rootfs or will be put in
a separate initrd image and whose goal is only to update the system. I
have tried to describe pros and cons of several different solutions
(updating via bootloader ? single copy against dual copy ?) - you can
find details in the doc directory in the swupdate repository.

Maybe someone of you can find this helpful, and I will be happy if I
could get some feedback.

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: Deploying Yocto build images

[Diego Sueiro]

[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]

Stefano,

This is a really great tool. I'm always developing a new software update
tool for each new project, since there are different requirements.
As I can see you are dealing with different scenarios, and this is really
amazing.

I'm already cloning meta-swupdate to test it.
Looking at recipes I realized that swupdate is initialized in an sysVinit
environment.
Did you plan to use systemd too?
Maybe I can contribute to this.

Thanks for sharing.

Regards,

--
*dS
Diego Sueiro

/*long live rock 'n roll*/


2013/11/21 Stefano Babic <sbabic@denx.de>

> Hi everybody,
>
> in the last ELCE, David point out in his presentation that we should
> improve how to deploy Yocto images on the target.
>
> I did some work this year to provide a reliable way for some customers
> of us to install Yocto's images in field, and I have published last week
> the sources. Here the link of the announcement:
>
>         http://comments.gmane.org/gmane.comp.embedded.eldk/2397
>
> Mainly, it is a tool that can be stored in the rootfs or will be put in
> a separate initrd image and whose goal is only to update the system. I
> have tried to describe pros and cons of several different solutions
> (updating via bootloader ? single copy against dual copy ?) - you can
> find details in the doc directory in the swupdate repository.
>
> Maybe someone of you can find this helpful, and I will be happy if I
> could get some feedback.
>
> Best regards,
> Stefano Babic
>
> --
> =====================================================================
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
> =====================================================================
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

[-- Attachment #2: Type: text/html, Size: 3083 bytes --]

Re: Deploying Yocto build images

[Stefano Babic]

Hi Diego,

On 21/11/2013 13:34, Diego Sueiro wrote:
> Stefano,
> 
> This is a really great tool. I'm always developing a new software update
> tool for each new project, since there are different requirements.

That is right. I did the same in the past, until I recognize that I
waste a lot of effort.

> As I can see you are dealing with different scenarios, and this is
> really amazing.
> 

Sure. I hope that the tool remains general enough to allow to include
scenarios that I was not able to foresee.

> I'm already cloning meta-swupdate to test it.
> Looking at recipes I realized that swupdate is initialized in an
> sysVinit environment.
> Did you plan to use systemd too?
> Maybe I can contribute to this.

Let's say that how the tool is started is open and can be easy
customized. Updating to systemd is surely nice.

One use of case I have is to not use neither sysVinit not systemd at
all. The tool is put into a initrd and it is the only task running, that
is I modified rcS to start only the tool and to speed up the booting
process. But as I said, this is only a use case.

Best regards,
Stefano



-- 
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: Deploying Yocto build images

[Bryan Evenson]

Stefano,

Looks interesting.  I have a question that I didn't see covered in the documentation (yet).  I can see the benefit to having a single image firmware upgrade, but how does swupdate handle configuration differences?  I can see cases in which the majority of the software will be the same but there will be configuration file differences from device to device.  I like the idea of a single image upgrade, but not at the cost of wiping the device-specific configuration.

Thanks,
Bryan E.

> -----Original Message-----
> From: yocto-bounces@yoctoproject.org [mailto:yocto-
> bounces@yoctoproject.org] On Behalf Of Stefano Babic
> Sent: Thursday, November 21, 2013 7:11 AM
> To: yocto@yoctoproject.org
> Subject: [yocto] Deploying Yocto build images
> 
> Hi everybody,
> 
> in the last ELCE, David point out in his presentation that we should
> improve how to deploy Yocto images on the target.
> 
> I did some work this year to provide a reliable way for some customers
> of us to install Yocto's images in field, and I have published last
> week the sources. Here the link of the announcement:
> 
> 	http://comments.gmane.org/gmane.comp.embedded.eldk/2397
> 
> Mainly, it is a tool that can be stored in the rootfs or will be put in
> a separate initrd image and whose goal is only to update the system. I
> have tried to describe pros and cons of several different solutions
> (updating via bootloader ? single copy against dual copy ?) - you can
> find details in the doc directory in the swupdate repository.
> 
> Maybe someone of you can find this helpful, and I will be happy if I
> could get some feedback.
> 
> Best regards,
> Stefano Babic
> 
> --
> =====================================================================
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
> =====================================================================
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: Deploying Yocto build images

[Stefano Babic]

Hi Bryan,

On 21/11/2013 16:48, Bryan Evenson wrote:
> Stefano,
> 
> Looks interesting.  I have a question that I didn't see covered in
> the documentation (yet).

I know, this is my fault. Writing documentation is a task that is always
postponed ;-)

> I can see the benefit to having a single image firmware upgrade, but
> how does swupdate handle configuration differences?  I can see cases
> in which the majority of the software will be the same but there will
> be configuration file differences from device to device.  I like the
> idea of a single image upgrade, but not at the cost of wiping the
> device-specific configuration.

I explain how it was done in a real project. The scope was to have a
single image, containing update for multiple devices. Let's say this is
an update of version Y for a whole family of products, but each product
has its own parts (mostly kernel differs, but not only).

In that case, an external parser was used. The sw-description is written
as XML, with a syntax like this:

<?xml version="1.0" encoding="UTF-8"?>
<software version="1.0">
  <name>Update Image</name>
  <version>1.0.0</version>
   <description>Firmware for XXXXX Project</description>

   <images>
   <image device="firstdevice" version="0.9">
     <stream name="dev1-uImage" type="ubivol" volume="kernel" />
     <stream name="dev1.dtb" type="ubivol" volume="dtb" />
     <stream name="dev1-rootfs.ubifs" type="ubivol" volume="rootfs"/>
     <stream name="dev1-uboot-env" type="uboot" />
     <stream name="raw_vfat" type="raw" dest="/dev/mmcblk0p4" />
     <stream name="sdcard.lua" type="lua" />
   </image>

   <image device="seconddevice" version="0.9">
     <stream name="dev2.dtb" rev="0.9" type="dtb" />
     <stream name="dev2-uImage" rev="0.9" type="kernel" />
......

That is: there is a single description, that describes the components
for all devices. Each device must recognized where it is running - this
was done with a configuration file or reading directly from hardware
(GPIOs, SPI EPROM).

When the image is downloaded, the description is parsed and only the
significant parts are extracted - the rest is discarded.
Of course, the image can be significantly bigger - it contains the
software for many devices. In my real project, this was not an issue -
the scope was really to have a single image for a product family.

Note: this is only an example how the tool was used. But this shows also
that the syntax of the sw-description is open. I could also add the LUA
script/XML parser to the repository, it is at the moment not stored - I
have thought at it as a custom specific requirement.

Best regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: Deploying Yocto build images

[Michael_E_Brown]

[-- Attachment #1: Type: text/plain, Size: 3295 bytes --]

Thanks for posting this. It's very timely for a project I am in the initial stages of designing. I have looked over the docs and some of the code, and am interested in using this.

I would make a couple suggestions:

1)      As you mention, documentation is important. Unless this is very well documented, it's difficult for other people to take up and use effectively, or to advocate for its use in a group

2)      Progress indications: it's going to be fairly important to build in some way to notify other process about the state of the update. A flexible framework for sharing progress would be very much appreciated. There are a couple scenarios I have, but the main one is a network notification where one device is being updated by another. The updated device should be able to signal update progress to the updater device in a flexible fashion. I haven't looked at the code hard enough to see how difficult this would be to add.
--
Michael

From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Diego Sueiro
Sent: Thursday, November 21, 2013 6:35 AM
To: Stefano Babic
Cc: yocto@yoctoproject.org
Subject: Re: [yocto] Deploying Yocto build images

Stefano,

This is a really great tool. I'm always developing a new software update tool for each new project, since there are different requirements.
As I can see you are dealing with different scenarios, and this is really amazing.

I'm already cloning meta-swupdate to test it.
Looking at recipes I realized that swupdate is initialized in an sysVinit environment.
Did you plan to use systemd too?
Maybe I can contribute to this.

Thanks for sharing.

Regards,

--
*dS
Diego Sueiro

/*long live rock 'n roll*/

2013/11/21 Stefano Babic <sbabic@denx.de<mailto:sbabic@denx.de>>
Hi everybody,

in the last ELCE, David point out in his presentation that we should
improve how to deploy Yocto images on the target.

I did some work this year to provide a reliable way for some customers
of us to install Yocto's images in field, and I have published last week
the sources. Here the link of the announcement:

        http://comments.gmane.org/gmane.comp.embedded.eldk/2397

Mainly, it is a tool that can be stored in the rootfs or will be put in
a separate initrd image and whose goal is only to update the system. I
have tried to describe pros and cons of several different solutions
(updating via bootloader ? single copy against dual copy ?) - you can
find details in the doc directory in the swupdate repository.

Maybe someone of you can find this helpful, and I will be happy if I
could get some feedback.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53<tel:%2B49-8142-66989-53> Fax: +49-8142-66989-80<tel:%2B49-8142-66989-80> Email: sbabic@denx.de<mailto:sbabic@denx.de>
=====================================================================
_______________________________________________
yocto mailing list
yocto@yoctoproject.org<mailto:yocto@yoctoproject.org>
https://lists.yoctoproject.org/listinfo/yocto


[-- Attachment #2: Type: text/html, Size: 10616 bytes --]

Re: Deploying Yocto build images

[Stefano Babic]

Hi Michael,

On 21/11/2013 19:20, Michael_E_Brown@Dell.com wrote:
> Thanks for posting this. It’s very timely for a project I am in the
> initial stages of designing. I have looked over the docs and some of the
> code, and am interested in using this.
> 
>  
> 
> I would make a couple suggestions:
> 

Thanks !

> 1)      As you mention, documentation is important. Unless this is very
> well documented, it’s difficult for other people to take up and use
> effectively, or to advocate for its use in a group

I know, and I am understand there are a lot of things that I have not
explained. Documentation is first priority.

> 2)      Progress indications: it’s going to be fairly important to build
> in some way to notify other process about the state of the update.
> A
> flexible framework for sharing progress would be very much appreciated.

There is an interface using Unix Domain Socket to communicate with the
installer. There is only a couple of implemented messages (start
installer - get status). It is quite rudimentary, but it can be extended.

This interface is also used by the AJAX interface (see in ./www) to
report the current status to the operator's browser when a downloaded is
started from network. Another use case is to output the status on a LCD:
a separate process can ask the installer and render the output for the
operator.

> There are a couple scenarios I have, but the main one is a network
> notification where one device is being updated by another. The updated
> device should be able to signal update progress to the updater device in
> a flexible fashion. I haven’t looked at the code hard enough to see how
> difficult this would be to add.

I think it is should be not very difficult. As I said, there is already
an interface that can be easy extended.

Regards,
Stefano


-- 
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================