Embedded Linux Flag Version

Embedded Linux Flag Version

[Tim Bird]

Embedded Flag Version


Recently, two embedded Linux summit meetings were held, one in Tokyo,
Japan and one in Cambridge, UK. The purpose of these meetings was to
discuss among industry and community leaders things that might help
improve the value or decrease the cost of embedded Linux. One initiative
which came out of these meetings, was the notion of declaring a "flag
version" of Linux, for embedded use. This would be a specific version of
the Linux kernel, chosen as a rallying point for embedded Linux
software, add-ons and products.


In hearing from several different consumer electronics product companies
and semi-conductor producers at the summits, it became apparent that a
significant problem in the embedded space is variation in the kernel
version. This could be called version fragmentation. Companies,
especially those at the end of the software supply chain, often get
stuck using a particular kernel version for a product due to the cost to
integrate software coming from different entities.


Of course the optimal solution to this problem would be if all required
software were pre-integrated into the latest stable release, at all
times. However, software can not be mainlined instantaneously. Not
everyone is using the latest version of the Linux kernel (2.6.36, as of
this writing). In point of fact, no one in the CE industry does (or
really can), use the absolute latest kernel for a product that is
shipping today. It takes time to assemble the software and perform the
testing required to ship a product. However, it is desirable for many
reasons to stay as close to the current version of Linux as possible.


Selecting a "flag version" is intended to help with this problem. First,
it should be explained what having a flag version means. It means that
suppliers and vendors throughout the embedded industry will be
encouraged to use a particular version of the kernel for software
development, integration and testing. Also, industry and community
developers agree to work together to maintain a long-term stable branch
of the flag version of the kernel (until the next flag version is
declared), in an effort to share costs and improve stability and quality.


It may seem counter-intuitive that selecting a version of the kernel for
long-term stable maintenance would help companies keep up with the
mainline version. But there are a number of reasons why it can. First
and foremost, a lot of effort is spent integrating software components
to make a product. This includes software components from IP block
vendors, semi-conductor vendors, parts suppliers, Linux vendors, and
in-house software teams. By decreasing the effort required to integrate
this software, it becomes possible to spend more time working on other
features, and working with upstream, and increases the frequency that a
company can afford to switch kernel versions. Also, by leveraging the
testing of multiple organizations and developers, including those
outside one's own company, a group can more quickly ensure that a kernel
is suitable for production use. This also decreases the effort required
to use the kernel, and increases the possibility of quicker version changes.


It was noted at the summit that several CE companies and embedded
projects will be using (or are already using) 2.6.35 for upcoming
products or releases. This includes Sony, Google, Meego, and Linaro. On
behalf of the CE Linux Forum and a number of consumer electronics
companies, projects and community developers, we therefore declare
2.6.35 as a flag version of the kernel for embedded use. Several
companies will be investing in development, integration and testing of
this version. Entities wanting to do business with those companies would
therefore be well-advised to make sure their hardware, drivers and
enhancements work well with this version of the kernel.


Tim Bird
Architecture Group Chair
CE Linux Forum
November 4, 2010

Re: Embedded Linux Flag Version

[Josh Boyer]

On Thu, Nov 4, 2010 at 6:07 PM, Tim Bird <tim.bird@am.sony.com> wrote:
> It was noted at the summit that several CE companies and embedded
> projects will be using (or are already using) 2.6.35 for upcoming
> products or releases. This includes Sony, Google, Meego, and Linaro. On
> behalf of the CE Linux Forum and a number of consumer electronics
> companies, projects and community developers, we therefore declare
> 2.6.35 as a flag version of the kernel for embedded use. Several
> companies will be investing in development, integration and testing of
> this version. Entities wanting to do business with those companies would
> therefore be well-advised to make sure their hardware, drivers and
> enhancements work well with this version of the kernel.

What is the expected lifetime for a flag version?  A new version every
year, 2 years, etc?  While the reasoning behind this makes sense to
me, I'm curious how this will impact migration to the next flag
version, and how the lifetime of a flag version will play into future
migration.

josh

Re: Embedded Linux Flag Version

[Mike Frysinger]

On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote:
> It was noted at the summit that several CE companies and embedded
> projects will be using (or are already using) 2.6.35 for upcoming
> products or releases. This includes Sony, Google, Meego, and Linaro. On
> behalf of the CE Linux Forum and a number of consumer electronics
> companies, projects and community developers, we therefore declare
> 2.6.35 as a flag version of the kernel for embedded use. Several
> companies will be investing in development, integration and testing of
> this version. Entities wanting to do business with those companies would
> therefore be well-advised to make sure their hardware, drivers and
> enhancements work well with this version of the kernel.

wouldnt it make more sense to piggy back the extensive work going into
the "stable" tree ?  many of the points you raise after all are the
entire founding point of it.  plus, all the main distros form around
that, are spending time working on that, is marked as supported for 2
or 3 years, and there is already infrastructure/framework/process in
place (stable@kernel.org).

so instead of picking arbitrary versions (like 2.6.35) and needlessly
replicating the huge work load, simply declare these stable trees as
the "flag" versions.  that means today it'd be 2.6.32.y.
-mike

Re: Embedded Linux Flag Version

[Tim Bird]

On 11/09/2010 03:19 AM, Mike Frysinger wrote:
> On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote:
>> It was noted at the summit that several CE companies and embedded
>> projects will be using (or are already using) 2.6.35 for upcoming
>> products or releases. This includes Sony, Google, Meego, and Linaro. On
>> behalf of the CE Linux Forum and a number of consumer electronics
>> companies, projects and community developers, we therefore declare
>> 2.6.35 as a flag version of the kernel for embedded use. Several
>> companies will be investing in development, integration and testing of
>> this version. Entities wanting to do business with those companies would
>> therefore be well-advised to make sure their hardware, drivers and
>> enhancements work well with this version of the kernel.
> 
> wouldnt it make more sense to piggy back the extensive work going into
> the "stable" tree ?  many of the points you raise after all are the
> entire founding point of it.  plus, all the main distros form around
> that, are spending time working on that, is marked as supported for 2
> or 3 years, and there is already infrastructure/framework/process in
> place (stable@kernel.org).
> 
> so instead of picking arbitrary versions (like 2.6.35) and needlessly
> replicating the huge work load, simply declare these stable trees as
> the "flag" versions.  that means today it'd be 2.6.32.y.

The fact that this tree is already a year old, and not likely to be
brought forward for at least another 2 years is the reason we didn't
choose it this time.  Most of the high-profile, active embedded projects
are already on 2.6.35.  For companies looking to adopt a new base kernel
in the next 12 months, I don't want to have them start with a year-old
kernel.  We did consider the utility of synchronizing with the enterprise
stable tree, but the timing just didn't work too well this time around.

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Network Entertainment
=============================

Re: Embedded Linux Flag Version

[Nicholas Mc Guire]

On Tue, 09 Nov 2010, Tim Bird wrote:

> On 11/09/2010 03:19 AM, Mike Frysinger wrote:
> > On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote:
> >> It was noted at the summit that several CE companies and embedded
> >> projects will be using (or are already using) 2.6.35 for upcoming
> >> products or releases. This includes Sony, Google, Meego, and Linaro. On
> >> behalf of the CE Linux Forum and a number of consumer electronics
> >> companies, projects and community developers, we therefore declare
> >> 2.6.35 as a flag version of the kernel for embedded use. Several
> >> companies will be investing in development, integration and testing of
> >> this version. Entities wanting to do business with those companies would
> >> therefore be well-advised to make sure their hardware, drivers and
> >> enhancements work well with this version of the kernel.
> > 
> > wouldnt it make more sense to piggy back the extensive work going into
> > the "stable" tree ?  many of the points you raise after all are the
> > entire founding point of it.  plus, all the main distros form around
> > that, are spending time working on that, is marked as supported for 2
> > or 3 years, and there is already infrastructure/framework/process in
> > place (stable@kernel.org).
> > 
> > so instead of picking arbitrary versions (like 2.6.35) and needlessly
> > replicating the huge work load, simply declare these stable trees as
> > the "flag" versions.  that means today it'd be 2.6.32.y.
> 
> The fact that this tree is already a year old, and not likely to be
> brought forward for at least another 2 years is the reason we didn't
> choose it this time.  Most of the high-profile, active embedded projects
> are already on 2.6.35.  For companies looking to adopt a new base kernel
> in the next 12 months, I don't want to have them start with a year-old
> kernel.  We did consider the utility of synchronizing with the enterprise
> stable tree, but the timing just didn't work too well this time around.
>
I guess one of the key issues is that it would need to be defined beforehand
what version will be used as the next "flag" version so vendors could make
sure that there drivers are available. Further if such a selection were to
be made then there would need to be consesus on maintaining the respective
version for a sufficiently long time to allow for the next "flag" version
to evolve. Finally for those working on products even if you would now define
the current version as the "flag" version it would not help much as they
would hardly be able to shift to a different kernel short term - so again the
key is defining what version would be atleast planed for the next "flag" 
version - something like now defining it to be 2.6.38 - and then testing 
appropriately in the runup to 2.6.38 to deliver, most notably for the 
embedded architectures.

thx!
hofrat 

Re: Embedded Linux Flag Version

[Tim Bird]

On 11/09/2010 09:25 AM, Nicholas Mc Guire wrote:
> I guess one of the key issues is that it would need to be defined beforehand
> what version will be used as the next "flag" version so vendors could make
> sure that there drivers are available.

Yeah.  People keep asking about that.  Unfortunately, this was
not discussed or settled on - so I can't really comment on that
as an industry representative.  However, I will say that we'd like
to work to synchronize our next flag version with the next
enterprise stable release.  (But again, it will depend on timing).

Personally (not speaking on behalf of anyone else here), I think
there would be a certain poetry to selecting 2.6.42 (which may
just also be the same as or one before kernel version 3.1)

;-)

 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Network Entertainment
=============================

Re: Embedded Linux Flag Version

[Nicolas Pitre]

On Tue, 9 Nov 2010, Nicholas Mc Guire wrote:

> On Tue, 09 Nov 2010, Tim Bird wrote:
> 
> > On 11/09/2010 03:19 AM, Mike Frysinger wrote:
> > > On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote:
> > >> It was noted at the summit that several CE companies and embedded
> > >> projects will be using (or are already using) 2.6.35 for upcoming
> > >> products or releases. This includes Sony, Google, Meego, and Linaro. On
> > >> behalf of the CE Linux Forum and a number of consumer electronics
> > >> companies, projects and community developers, we therefore declare
> > >> 2.6.35 as a flag version of the kernel for embedded use. Several
> > >> companies will be investing in development, integration and testing of
> > >> this version. Entities wanting to do business with those companies would
> > >> therefore be well-advised to make sure their hardware, drivers and
> > >> enhancements work well with this version of the kernel.
> > > 
> > > wouldnt it make more sense to piggy back the extensive work going into
> > > the "stable" tree ?  many of the points you raise after all are the
> > > entire founding point of it.  plus, all the main distros form around
> > > that, are spending time working on that, is marked as supported for 2
> > > or 3 years, and there is already infrastructure/framework/process in
> > > place (stable@kernel.org).
> > > 
> > > so instead of picking arbitrary versions (like 2.6.35) and needlessly
> > > replicating the huge work load, simply declare these stable trees as
> > > the "flag" versions.  that means today it'd be 2.6.32.y.
> > 
> > The fact that this tree is already a year old, and not likely to be
> > brought forward for at least another 2 years is the reason we didn't
> > choose it this time.  Most of the high-profile, active embedded projects
> > are already on 2.6.35.  For companies looking to adopt a new base kernel
> > in the next 12 months, I don't want to have them start with a year-old
> > kernel.  We did consider the utility of synchronizing with the enterprise
> > stable tree, but the timing just didn't work too well this time around.
> >
> I guess one of the key issues is that it would need to be defined beforehand
> what version will be used as the next "flag" version so vendors could make
> sure that there drivers are available. Further if such a selection were to
> be made then there would need to be consesus on maintaining the respective
> version for a sufficiently long time to allow for the next "flag" version
> to evolve. Finally for those working on products even if you would now define
> the current version as the "flag" version it would not help much as they
> would hardly be able to shift to a different kernel short term - so again the
> key is defining what version would be atleast planed for the next "flag" 
> version - something like now defining it to be 2.6.38 - and then testing 
> appropriately in the runup to 2.6.38 to deliver, most notably for the 
> embedded architectures.

FWIW, the imminent Linaro release is using 2.6.35.  The next release 
scheduled for May 2011 will most probably use 2.6.38.


Nicolas

Re: Embedded Linux Flag Version

[Mike Frysinger]

On Tue, Nov 9, 2010 at 12:17, Tim Bird wrote:
> On 11/09/2010 03:19 AM, Mike Frysinger wrote:
>> On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote:
>>> It was noted at the summit that several CE companies and embedded
>>> projects will be using (or are already using) 2.6.35 for upcoming
>>> products or releases. This includes Sony, Google, Meego, and Linaro. On
>>> behalf of the CE Linux Forum and a number of consumer electronics
>>> companies, projects and community developers, we therefore declare
>>> 2.6.35 as a flag version of the kernel for embedded use. Several
>>> companies will be investing in development, integration and testing of
>>> this version. Entities wanting to do business with those companies would
>>> therefore be well-advised to make sure their hardware, drivers and
>>> enhancements work well with this version of the kernel.
>>
>> wouldnt it make more sense to piggy back the extensive work going into
>> the "stable" tree ?  many of the points you raise after all are the
>> entire founding point of it.  plus, all the main distros form around
>> that, are spending time working on that, is marked as supported for 2
>> or 3 years, and there is already infrastructure/framework/process in
>> place (stable@kernel.org).
>>
>> so instead of picking arbitrary versions (like 2.6.35) and needlessly
>> replicating the huge work load, simply declare these stable trees as
>> the "flag" versions.  that means today it'd be 2.6.32.y.
>
> The fact that this tree is already a year old, and not likely to be
> brought forward for at least another 2 years is the reason we didn't
> choose it this time.  Most of the high-profile, active embedded projects
> are already on 2.6.35.  For companies looking to adopt a new base kernel
> in the next 12 months, I don't want to have them start with a year-old
> kernel.  We did consider the utility of synchronizing with the enterprise
> stable tree, but the timing just didn't work too well this time around.

so you're suggesting this is a one-off choice.  in the future, the
"flag" versions will simply piggy the existing stable trees ?
-mike

Re: Embedded Linux Flag Version

[Tim Bird]

On 11/10/2010 2:16 AM, Mike Frysinger wrote:
> On Tue, Nov 9, 2010 at 12:17, Tim Bird wrote:
>> On 11/09/2010 03:19 AM, Mike Frysinger wrote:
>>> On Thu, Nov 4, 2010 at 18:07, Tim Bird wrote:
>>>> It was noted at the summit that several CE companies and embedded
>>>> projects will be using (or are already using) 2.6.35 for upcoming
>>>> products or releases. This includes Sony, Google, Meego, and Linaro. On
>>>> behalf of the CE Linux Forum and a number of consumer electronics
>>>> companies, projects and community developers, we therefore declare
>>>> 2.6.35 as a flag version of the kernel for embedded use. Several
>>>> companies will be investing in development, integration and testing of
>>>> this version. Entities wanting to do business with those companies would
>>>> therefore be well-advised to make sure their hardware, drivers and
>>>> enhancements work well with this version of the kernel.
>>> wouldnt it make more sense to piggy back the extensive work going into
>>> the "stable" tree ?  many of the points you raise after all are the
>>> entire founding point of it.  plus, all the main distros form around
>>> that, are spending time working on that, is marked as supported for 2
>>> or 3 years, and there is already infrastructure/framework/process in
>>> place (stable@kernel.org).
>>>
>>> so instead of picking arbitrary versions (like 2.6.35) and needlessly
>>> replicating the huge work load, simply declare these stable trees as
>>> the "flag" versions.  that means today it'd be 2.6.32.y.
>> The fact that this tree is already a year old, and not likely to be
>> brought forward for at least another 2 years is the reason we didn't
>> choose it this time.  Most of the high-profile, active embedded projects
>> are already on 2.6.35.  For companies looking to adopt a new base kernel
>> in the next 12 months, I don't want to have them start with a year-old
>> kernel.  We did consider the utility of synchronizing with the enterprise
>> stable tree, but the timing just didn't work too well this time around.
> so you're suggesting this is a one-off choice.  in the future, the
> "flag" versions will simply piggy the existing stable trees ?
Yes, if the timing works then.
I can't promise anything, but I think it would be a very strong goal.
  -- Tim