Uses of Linux backports in the industry

Uses of Linux backports in the industry

[Luis R. Rodriguez]

Me and Julia are working on a paper which evaluates use of Coccinelle
on backports, a preliminary draft of such paper can be found on github
[0]. We are making some tweaks to this, one of which is covering the
uses of Linux backports [1] in the industry, for this we'd like to try
to get feedback as to where and how folks are using backports. Please
let me and Julia know -- or if the information is not sensitive please
feel free to just reply to this thread and share with others. All
feedback is greatly appreciated. If you are OK in having us list or
generalize your usage please indicate so.

Thanks,

[0] https://github.com/mcgrof/paper-backports/blob/master/paper.pdf
[1] https://backports.wiki.kernel.org

 Luis

Re: Uses of Linux backports in the industry

[Johannes Berg]

-lkml, that list is overfull anyway ...

On Thu, 2015-05-28 at 19:54 -0700, Luis R. Rodriguez wrote:
> Me and Julia are working on a paper which evaluates use of Coccinelle
> on backports, a preliminary draft of such paper can be found on github
> [0]. We are making some tweaks to this, one of which is covering the
> uses of Linux backports [1] in the industry, for this we'd like to try
> to get feedback as to where and how folks are using backports. Please
> let me and Julia know -- or if the information is not sensitive please
> feel free to just reply to this thread and share with others. All
> feedback is greatly appreciated. If you are OK in having us list or
> generalize your usage please indicate so.

We've just published one of our internal git trees that is actually
based on backports:
https://git.kernel.org/cgit/linux/kernel/git/iwlwifi/backport-iwlwifi.git/

johannes

Re: Uses of Linux backports in the industry

[Arend van Spriel]

On 05/29/15 09:00, Johannes Berg wrote:
> -lkml, that list is overfull anyway ...
>
> On Thu, 2015-05-28 at 19:54 -0700, Luis R. Rodriguez wrote:
>> Me and Julia are working on a paper which evaluates use of Coccinelle
>> on backports, a preliminary draft of such paper can be found on github
>> [0]. We are making some tweaks to this, one of which is covering the
>> uses of Linux backports [1] in the industry, for this we'd like to try
>> to get feedback as to where and how folks are using backports. Please
>> let me and Julia know -- or if the information is not sensitive please
>> feel free to just reply to this thread and share with others. All
>> feedback is greatly appreciated.

We use the backports project solely for testing. We build and test 
nightly for a number of lab machines. Some are running recent 
wireless-testing kernel, but a number of those run a relatively old FC19 
kernel (3.11.1) for which we build backport brcm80211 drivers. We do 
that for our internal tree and for the wireless-testing tree.

>> If you are OK in having us list or
>> generalize your usage please indicate so.

Go ahead and make your list ;-)

Regards,
Arend

> We've just published one of our internal git trees that is actually
> based on backports:
> https://git.kernel.org/cgit/linux/kernel/git/iwlwifi/backport-iwlwifi.git/
>
> johannes
>
> --
> To unsubscribe from this list: send the line "unsubscribe backports" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Uses of Linux backports in the industry

[Julia Lawall]

On Fri, 29 May 2015, Arend van Spriel wrote:

> On 05/29/15 09:00, Johannes Berg wrote:
> > -lkml, that list is overfull anyway ...
> >
> > On Thu, 2015-05-28 at 19:54 -0700, Luis R. Rodriguez wrote:
> > > Me and Julia are working on a paper which evaluates use of Coccinelle
> > > on backports, a preliminary draft of such paper can be found on github
> > > [0]. We are making some tweaks to this, one of which is covering the
> > > uses of Linux backports [1] in the industry, for this we'd like to try
> > > to get feedback as to where and how folks are using backports. Please
> > > let me and Julia know -- or if the information is not sensitive please
> > > feel free to just reply to this thread and share with others. All
> > > feedback is greatly appreciated.
>
> We use the backports project solely for testing. We build and test nightly for
> a number of lab machines. Some are running recent wireless-testing kernel, but
> a number of those run a relatively old FC19 kernel (3.11.1) for which we build
> backport brcm80211 drivers. We do that for our internal tree and for the
> wireless-testing tree.
>
> > > If you are OK in having us list or
> > > generalize your usage please indicate so.
>
> Go ahead and make your list ;-)

Thanks for the feedback!

julia

Re: Uses of Linux backports in the industry

[Arend van Spriel]

On 05/29/15 10:08, Arend van Spriel wrote:
> On 05/29/15 09:00, Johannes Berg wrote:
>> -lkml, that list is overfull anyway ...
>>
>> On Thu, 2015-05-28 at 19:54 -0700, Luis R. Rodriguez wrote:
>>> Me and Julia are working on a paper which evaluates use of Coccinelle
>>> on backports, a preliminary draft of such paper can be found on github
>>> [0]. We are making some tweaks to this, one of which is covering the
>>> uses of Linux backports [1] in the industry, for this we'd like to try
>>> to get feedback as to where and how folks are using backports. Please
>>> let me and Julia know -- or if the information is not sensitive please
>>> feel free to just reply to this thread and share with others. All
>>> feedback is greatly appreciated.
>
> We use the backports project solely for testing. We build and test
> nightly for a number of lab machines. Some are running recent
> wireless-testing kernel, but a number of those run a relatively old FC19
> kernel (3.11.1) for which we build backport brcm80211 drivers. We do
> that for our internal tree and for the wireless-testing tree.

Guess I don't need to mention OpenWrt is using it, right?

Gr. AvS

>>> If you are OK in having us list or
>>> generalize your usage please indicate so.
>
> Go ahead and make your list ;-)
>
> Regards,
> Arend
>
>> We've just published one of our internal git trees that is actually
>> based on backports:
>> https://git.kernel.org/cgit/linux/kernel/git/iwlwifi/backport-iwlwifi.git/
>>
>>
>> johannes
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe backports" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe backports" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Uses of Linux backports in the industry

[Luis R. Rodriguez]

On Fri, May 29, 2015 at 12:00 AM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> -lkml, that list is overfull anyway ...
>
> On Thu, 2015-05-28 at 19:54 -0700, Luis R. Rodriguez wrote:
>> Me and Julia are working on a paper which evaluates use of Coccinelle
>> on backports, a preliminary draft of such paper can be found on github
>> [0]. We are making some tweaks to this, one of which is covering the
>> uses of Linux backports [1] in the industry, for this we'd like to try
>> to get feedback as to where and how folks are using backports. Please
>> let me and Julia know -- or if the information is not sensitive please
>> feel free to just reply to this thread and share with others. All
>> feedback is greatly appreciated. If you are OK in having us list or
>> generalize your usage please indicate so.
>
> We've just published one of our internal git trees that is actually
> based on backports:
> https://git.kernel.org/cgit/linux/kernel/git/iwlwifi/backport-iwlwifi.git/

Great thanks! Any chance you can elaborate a bit on how its used? Some
verbiage we can use perhaps to explain, in the context from silicon
manufacturing / testing / production, etc.

 Luis

Re: Uses of Linux backports in the industry

[Johannes Berg]

On Fri, 2015-05-29 at 09:33 -0700, Luis R. Rodriguez wrote:

> > We've just published one of our internal git trees that is actually
> > based on backports:
> > https://git.kernel.org/cgit/linux/kernel/git/iwlwifi/backport-iwlwifi.git/
> 
> Great thanks! Any chance you can elaborate a bit on how its used? Some
> verbiage we can use perhaps to explain, in the context from silicon
> manufacturing / testing / production, etc.

Err, it's just a driver. Everything you need a driver for, you use this.

johannes

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:

Hi,

<snip>

let me add some personal views for backports issue in general - or,
more precisely, the opposite - forward porting:

Silicon vendors often just provide drivers for very old kernels, even
worse: proprietary drivers, which need ugly kernel hacks (eg. Freescale
is one of the worst). So I'll have to forward-port drivers to newer
kernels - but I never had the practical need to backport one.

Actually, I really wonder why folks are sticking to ancient kernels on
newer hardware.


--mtx

--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.

Re: Uses of Linux backports in the industry

[Richard Weinberger]

On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
<weigelt@melag.de> wrote:
> Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
> Actually, I really wonder why folks are sticking to ancient kernels on
> newer hardware.

Enterprise distribution kernels. Or "special" kernels like PREEMPT_RT.
Sometimes the vendor BSP is that horrid that a customer cannot afford
to forward port it
but wants recent stuff. So you need to backport...

-- 
Thanks,
//richard

Re: Uses of Linux backports in the industry

[Theodore Ts'o]

On Fri, May 29, 2015 at 05:01:00PM +0200, Richard Weinberger wrote:
> On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
> <weigelt@melag.de> wrote:
> > Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
> > Actually, I really wonder why folks are sticking to ancient kernels on
> > newer hardware.
> 
> Enterprise distribution kernels. Or "special" kernels like PREEMPT_RT.
> Sometimes the vendor BSP is that horrid that a customer cannot afford
> to forward port it
> but wants recent stuff. So you need to backport...

Yep.  The technique I used for the backporting ext4 encryption into
the 3.10 android-common git tree in AOSP was to drop in the 3.18
versions of fs/ext4 and fs/jbd2 into the 3.10 tree (along with the
associaed include files in include/linux and include/trace/events, of
course), and then fix things up until they built correctly (using
cherry-picks and in some cases, reverting some changes in the 3.18
version of fs/ext4).  After I was sure the transplant of the 3.18
version of ext4 had "taken" correctly, with no test regressions, only
then did I cherry-pick all of the ext4 encryption changes on top of
3.10.

The backport of ext4 encryption to the 3.18 version of android-common
should be much easier.  :-)   Unfortunately, I also have to do a
backport to the 3.14 android-common branch as well.   <sigh>

Yes, it's ugly, but there still are some SOC and drivers that aren't
available on newer kernels.  Basically, the handset vendors need to
lean a lot harder on the SOC and other peripheral (cell radios, GPS,
etc., etc.).  :-(

    	  	     	   	   	      - Ted

Re: Uses of Linux backports in the industry

[Luis R. Rodriguez]

On Fri, May 29, 2015 at 10:36 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> On Fri, May 29, 2015 at 05:01:00PM +0200, Richard Weinberger wrote:
>> On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
>> <weigelt@melag.de> wrote:
>> > Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:

Just to clarify the original goal was to hear how folks use the Linux
backports project out there in the industry for purposes of the paper
being edited, hearing how folks might do their own backports is
certainly useful but I am also in hopes they might consider using
Linux backports for future work to help, well automatically backport
Linux.

>> > Actually, I really wonder why folks are sticking to ancient kernels on
>> > newer hardware.
>>
>> Enterprise distribution kernels. Or "special" kernels like PREEMPT_RT.
>> Sometimes the vendor BSP is that horrid that a customer cannot afford
>> to forward port it
>> but wants recent stuff. So you need to backport...
>
> Yep.  The technique I used for the backporting ext4 encryption into
> the 3.10 android-common git tree in AOSP was to drop in the 3.18
> versions of fs/ext4 and fs/jbd2 into the 3.10 tree (along with the
> associaed include files in include/linux and include/trace/events, of
> course), and then fix things up until they built correctly (using
> cherry-picks and in some cases, reverting some changes in the 3.18
> version of fs/ext4).  After I was sure the transplant of the 3.18
> version of ext4 had "taken" correctly, with no test regressions, only
> then did I cherry-pick all of the ext4 encryption changes on top of
> 3.10.

FWIW, if the Linux backports project started carrying ext4 into it,
you'd likely have less work to do and we'd enable use of ext4 to be
used on the range of supported kernels.

> The backport of ext4 encryption to the 3.18 version of android-common
> should be much easier.  :-)   Unfortunately, I also have to do a
> backport to the 3.14 android-common branch as well.   <sigh>
>
> Yes, it's ugly, but there still are some SOC and drivers that aren't
> available on newer kernels.  Basically, the handset vendors need to
> lean a lot harder on the SOC and other peripheral (cell radios, GPS,
> etc., etc.).  :-(

Yeah we dealt with a lot of those dependencies when backporting the
regulator subsystem for Media drivers, it was hard but hey we did
quite a bit of the work. Now that we have in-kernel integration
support the options on what to backport should be easier [1].

[1] https://backports.wiki.kernel.org/index.php/Documentation/integration

 Luis

Re: Uses of Linux backports in the industry

[Pavel Machek]

On Fri 2015-05-29 13:36:09, Theodore Ts'o wrote:
> On Fri, May 29, 2015 at 05:01:00PM +0200, Richard Weinberger wrote:
> > On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
> > <weigelt@melag.de> wrote:
> > > Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
> > > Actually, I really wonder why folks are sticking to ancient kernels on
> > > newer hardware.
> > 
> > Enterprise distribution kernels. Or "special" kernels like PREEMPT_RT.
> > Sometimes the vendor BSP is that horrid that a customer cannot afford
> > to forward port it
> > but wants recent stuff. So you need to backport...
> 
> Yep.  The technique I used for the backporting ext4 encryption into
> the 3.10 android-common git tree in AOSP was to drop in the 3.18
> versions of fs/ext4 and fs/jbd2 into the 3.10 tree (along with the
> associaed include files in include/linux and include/trace/events, of
> course), and then fix things up until they built correctly (using
> cherry-picks and in some cases, reverting some changes in the 3.18
> version of fs/ext4).  After I was sure the transplant of the 3.18
> version of ext4 had "taken" correctly, with no test regressions, only
> then did I cherry-pick all of the ext4 encryption changes on top of
> 3.10.
> 
> The backport of ext4 encryption to the 3.18 version of android-common
> should be much easier.  :-)   Unfortunately, I also have to do a
> backport to the 3.14 android-common branch as well.   <sigh>
> 
> Yes, it's ugly, but there still are some SOC and drivers that aren't
> available on newer kernels.  Basically, the handset vendors need to
> lean a lot harder on the SOC and other peripheral (cell radios, GPS,
> etc., etc.).  :-(

Actually, what we really want is chip vendors to clean up the
interfaces, and merge their changes upstream... Perhaps we'll be able
to install normal distros on cellphones one day...

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Pavel Machek]

On Fri 2015-05-29 13:36:09, Theodore Ts'o wrote:
> On Fri, May 29, 2015 at 05:01:00PM +0200, Richard Weinberger wrote:
> > On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
> > <weigelt@melag.de> wrote:
> > > Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
> > > Actually, I really wonder why folks are sticking to ancient kernels on
> > > newer hardware.
> > 
> > Enterprise distribution kernels. Or "special" kernels like PREEMPT_RT.
> > Sometimes the vendor BSP is that horrid that a customer cannot afford
> > to forward port it
> > but wants recent stuff. So you need to backport...
> 
> Yep.  The technique I used for the backporting ext4 encryption into
> the 3.10 android-common git tree in AOSP was to drop in the 3.18
> versions of fs/ext4 and fs/jbd2 into the 3.10 tree (along with the
> associaed include files in include/linux and include/trace/events, of
> course), and then fix things up until they built correctly (using
> cherry-picks and in some cases, reverting some changes in the 3.18
> version of fs/ext4).  After I was sure the transplant of the 3.18
> version of ext4 had "taken" correctly, with no test regressions, only
> then did I cherry-pick all of the ext4 encryption changes on top of
> 3.10.
> 
> The backport of ext4 encryption to the 3.18 version of android-common
> should be much easier.  :-)   Unfortunately, I also have to do a
> backport to the 3.14 android-common branch as well.   <sigh>
> 
> Yes, it's ugly, but there still are some SOC and drivers that aren't
> available on newer kernels.  Basically, the handset vendors need to
> lean a lot harder on the SOC and other peripheral (cell radios, GPS,
> etc., etc.).  :-(

Actually, what we really want is chip vendors to clean up the
interfaces, and merge their changes upstream... Perhaps we'll be able
to install normal distros on cellphones one day...

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 29.05.2015 um 19:36 schrieb Theodore Ts'o:

 > Yes, it's ugly, but there still are some SOC and drivers that aren't
> available on newer kernels.

Why so, exactly ?


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 29.05.2015 um 19:36 schrieb Theodore Ts'o:

 > Yes, it's ugly, but there still are some SOC and drivers that aren't
> available on newer kernels.

Why so, exactly ?


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 29.05.2015 um 17:01 schrieb Richard Weinberger:

Hi,

> On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
> <weigelt@melag.de> wrote:
>> Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
>> Actually, I really wonder why folks are sticking to ancient kernels on
>> newer hardware.
>
> Enterprise distribution kernels.

hmm, by "enterprise" you mean distros like RHEL, which even can't get a
dist-upgrade right ? ;-p

In that case, it's the duty of the dist vendor, to port their (often
horrible) vendor patches. I wouldn't run those distros bare-metal
anyways, so the need for new kernel features (eg. drivers) wouldn't
that huge.

 > Or "special" kernels like PREEMPT_RT.

PREEMPT_RT is pretty close to upstream.
There're at 4.0.5 right now, and 4.1 is still very fresh.

If I'd have the need for it (actually was already considering it for our
project), I'd rather port it to 4.1. (as our BSP already is at 4.1)

> Sometimes the vendor BSP is that horrid that a customer cannot afford
> to forward port it but wants recent stuff. So you need to backport...

By "vendor BSP", you perhaps mean certain soc or board manufacturer
stuff ? Just dont use it, it's usually horrible crap anyways. These
usually are fire-and-forget showcases, not suited for production use.
Waste of resources.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 29.05.2015 um 17:01 schrieb Richard Weinberger:

Hi,

> On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
> <weigelt@melag.de> wrote:
>> Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
>> Actually, I really wonder why folks are sticking to ancient kernels on
>> newer hardware.
>
> Enterprise distribution kernels.

hmm, by "enterprise" you mean distros like RHEL, which even can't get a
dist-upgrade right ? ;-p

In that case, it's the duty of the dist vendor, to port their (often
horrible) vendor patches. I wouldn't run those distros bare-metal
anyways, so the need for new kernel features (eg. drivers) wouldn't
that huge.

 > Or "special" kernels like PREEMPT_RT.

PREEMPT_RT is pretty close to upstream.
There're at 4.0.5 right now, and 4.1 is still very fresh.

If I'd have the need for it (actually was already considering it for our
project), I'd rather port it to 4.1. (as our BSP already is at 4.1)

> Sometimes the vendor BSP is that horrid that a customer cannot afford
> to forward port it but wants recent stuff. So you need to backport...

By "vendor BSP", you perhaps mean certain soc or board manufacturer
stuff ? Just dont use it, it's usually horrible crap anyways. These
usually are fire-and-forget showcases, not suited for production use.
Waste of resources.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.

Re: Uses of Linux backports in the industry

[Richard Weinberger]

Am 24.06.2015 um 11:09 schrieb Enrico Weigelt, metux IT consult:
> Am 29.05.2015 um 17:01 schrieb Richard Weinberger:
> 
> Hi,
> 
>> On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
>> <weigelt@melag.de> wrote:
>>> Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
>>> Actually, I really wonder why folks are sticking to ancient kernels on
>>> newer hardware.
>>
>> Enterprise distribution kernels.
> 
> hmm, by "enterprise" you mean distros like RHEL, which even can't get a
> dist-upgrade right ? ;-p

Please send such prepubescent flames to /dev/null.

> In that case, it's the duty of the dist vendor, to port their (often
> horrible) vendor patches. I wouldn't run those distros bare-metal
> anyways, so the need for new kernel features (eg. drivers) wouldn't
> that huge.
> 
>> Or "special" kernels like PREEMPT_RT.
> 
> PREEMPT_RT is pretty close to upstream.
> There're at 4.0.5 right now, and 4.1 is still very fresh.
> 
> If I'd have the need for it (actually was already considering it for our
> project), I'd rather port it to 4.1. (as our BSP already is at 4.1)

Porting PREEMPT_RT is not that easy.
Did you ever?

>> Sometimes the vendor BSP is that horrid that a customer cannot afford
>> to forward port it but wants recent stuff. So you need to backport...
> 
> By "vendor BSP", you perhaps mean certain soc or board manufacturer
> stuff ? Just dont use it, it's usually horrible crap anyways. These
> usually are fire-and-forget showcases, not suited for production use.
> Waste of resources.

So, you rewrite all drivers and the board support from scratch?
Interesting. I'd love to meet your customers they seem to have
a lot of money and time. ;-)

Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Richard Weinberger]

Am 24.06.2015 um 11:09 schrieb Enrico Weigelt, metux IT consult:
> Am 29.05.2015 um 17:01 schrieb Richard Weinberger:
> 
> Hi,
> 
>> On Fri, May 29, 2015 at 4:53 PM, Enrico Weigelt, metux IT consult
>> <weigelt@melag.de> wrote:
>>> Am 29.05.2015 um 04:54 schrieb Luis R. Rodriguez:
>>> Actually, I really wonder why folks are sticking to ancient kernels on
>>> newer hardware.
>>
>> Enterprise distribution kernels.
> 
> hmm, by "enterprise" you mean distros like RHEL, which even can't get a
> dist-upgrade right ? ;-p

Please send such prepubescent flames to /dev/null.

> In that case, it's the duty of the dist vendor, to port their (often
> horrible) vendor patches. I wouldn't run those distros bare-metal
> anyways, so the need for new kernel features (eg. drivers) wouldn't
> that huge.
> 
>> Or "special" kernels like PREEMPT_RT.
> 
> PREEMPT_RT is pretty close to upstream.
> There're at 4.0.5 right now, and 4.1 is still very fresh.
> 
> If I'd have the need for it (actually was already considering it for our
> project), I'd rather port it to 4.1. (as our BSP already is at 4.1)

Porting PREEMPT_RT is not that easy.
Did you ever?

>> Sometimes the vendor BSP is that horrid that a customer cannot afford
>> to forward port it but wants recent stuff. So you need to backport...
> 
> By "vendor BSP", you perhaps mean certain soc or board manufacturer
> stuff ? Just dont use it, it's usually horrible crap anyways. These
> usually are fire-and-forget showcases, not suited for production use.
> Waste of resources.

So, you rewrite all drivers and the board support from scratch?
Interesting. I'd love to meet your customers they seem to have
a lot of money and time. ;-)

Thanks,
//richard

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 24.06.2015 um 11:19 schrieb Richard Weinberger:

Hi,

 > Porting PREEMPT_RT is not that easy.
> Did you ever?

I know.

OTOH, is backporting drivers to ancient kernels (where internal APIs
often are _completely_ different) really easier ? Perhaps it might look
so, if it's just one individual driver - but often it doesn't keep this
way, sooner or later other things pop up.

> So, you rewrite all drivers and the board support from scratch?

Sometimes, if I have to. Because - on my own experience - what SoC
vendors provide usually is pretty unusable, just a quick showcase.

Right now, I'm working on a project w/ some imx53-based board.
What freescale provides here is practically unusable. Really ancient
(last time I checked, it was an old 2.6.x), unsable and insecure
(anybody had a closer look at their "kgsl" patch or their gst-plugin ?)

We'll have to drop the whole idea of using the GPUs, due to lack of
support - the existing driver/libgl is known to be broken and insecure,
no support from fsl whatsoever, we're lacking resources for a full
reverse engineering, and moving to another SoC is out of question
(at least for the forseeable future). So, it ends up in having no
GPU, therefore no GL/GSL, therefore no QtQuick/QML.


Pavel already mentioned the correct way to go: chip vendors should
provide proper (mainline'able) patches, or at least full specs.

And I'll add: those who dont, should simply be boycotted.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 24.06.2015 um 11:19 schrieb Richard Weinberger:

Hi,

 > Porting PREEMPT_RT is not that easy.
> Did you ever?

I know.

OTOH, is backporting drivers to ancient kernels (where internal APIs
often are _completely_ different) really easier ? Perhaps it might look
so, if it's just one individual driver - but often it doesn't keep this
way, sooner or later other things pop up.

> So, you rewrite all drivers and the board support from scratch?

Sometimes, if I have to. Because - on my own experience - what SoC
vendors provide usually is pretty unusable, just a quick showcase.

Right now, I'm working on a project w/ some imx53-based board.
What freescale provides here is practically unusable. Really ancient
(last time I checked, it was an old 2.6.x), unsable and insecure
(anybody had a closer look at their "kgsl" patch or their gst-plugin ?)

We'll have to drop the whole idea of using the GPUs, due to lack of
support - the existing driver/libgl is known to be broken and insecure,
no support from fsl whatsoever, we're lacking resources for a full
reverse engineering, and moving to another SoC is out of question
(at least for the forseeable future). So, it ends up in having no
GPU, therefore no GL/GSL, therefore no QtQuick/QML.


Pavel already mentioned the correct way to go: chip vendors should
provide proper (mainline'able) patches, or at least full specs.

And I'll add: those who dont, should simply be boycotted.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.

Re: Uses of Linux backports in the industry

[Richard Weinberger]

Am 24.06.2015 um 11:55 schrieb Enrico Weigelt, metux IT consult:
> Am 24.06.2015 um 11:19 schrieb Richard Weinberger:
> 
> Hi,
> 
>> Porting PREEMPT_RT is not that easy.
>> Did you ever?
> 
> I know.
> 
> OTOH, is backporting drivers to ancient kernels (where internal APIs
> often are _completely_ different) really easier ? Perhaps it might look
> so, if it's just one individual driver - but often it doesn't keep this
> way, sooner or later other things pop up.

At the end of the day customers will do what is less costly.
Sometimes backporting a driver is much less effort.
That's why we have the excellent backports project.

>> So, you rewrite all drivers and the board support from scratch?
> 
> Sometimes, if I have to. Because - on my own experience - what SoC
> vendors provide usually is pretty unusable, just a quick showcase.
> 
> Right now, I'm working on a project w/ some imx53-based board.
> What freescale provides here is practically unusable. Really ancient
> (last time I checked, it was an old 2.6.x), unsable and insecure
> (anybody had a closer look at their "kgsl" patch or their gst-plugin ?)
> 
> We'll have to drop the whole idea of using the GPUs, due to lack of
> support - the existing driver/libgl is known to be broken and insecure,
> no support from fsl whatsoever, we're lacking resources for a full
> reverse engineering, and moving to another SoC is out of question
> (at least for the forseeable future). So, it ends up in having no
> GPU, therefore no GL/GSL, therefore no QtQuick/QML.
> 
> 
> Pavel already mentioned the correct way to go: chip vendors should
> provide proper (mainline'able) patches, or at least full specs.

Sure, in a perfect world. But as of now we have to deal with that.

Thanks,
//richard
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Richard Weinberger]

Am 24.06.2015 um 11:55 schrieb Enrico Weigelt, metux IT consult:
> Am 24.06.2015 um 11:19 schrieb Richard Weinberger:
> 
> Hi,
> 
>> Porting PREEMPT_RT is not that easy.
>> Did you ever?
> 
> I know.
> 
> OTOH, is backporting drivers to ancient kernels (where internal APIs
> often are _completely_ different) really easier ? Perhaps it might look
> so, if it's just one individual driver - but often it doesn't keep this
> way, sooner or later other things pop up.

At the end of the day customers will do what is less costly.
Sometimes backporting a driver is much less effort.
That's why we have the excellent backports project.

>> So, you rewrite all drivers and the board support from scratch?
> 
> Sometimes, if I have to. Because - on my own experience - what SoC
> vendors provide usually is pretty unusable, just a quick showcase.
> 
> Right now, I'm working on a project w/ some imx53-based board.
> What freescale provides here is practically unusable. Really ancient
> (last time I checked, it was an old 2.6.x), unsable and insecure
> (anybody had a closer look at their "kgsl" patch or their gst-plugin ?)
> 
> We'll have to drop the whole idea of using the GPUs, due to lack of
> support - the existing driver/libgl is known to be broken and insecure,
> no support from fsl whatsoever, we're lacking resources for a full
> reverse engineering, and moving to another SoC is out of question
> (at least for the forseeable future). So, it ends up in having no
> GPU, therefore no GL/GSL, therefore no QtQuick/QML.
> 
> 
> Pavel already mentioned the correct way to go: chip vendors should
> provide proper (mainline'able) patches, or at least full specs.

Sure, in a perfect world. But as of now we have to deal with that.

Thanks,
//richard

Re: Uses of Linux backports in the industry

[Felix Fietkau]

On 2015-05-29 04:54, Luis R. Rodriguez wrote:
> Me and Julia are working on a paper which evaluates use of Coccinelle
> on backports, a preliminary draft of such paper can be found on github
> [0]. We are making some tweaks to this, one of which is covering the
> uses of Linux backports [1] in the industry, for this we'd like to try
> to get feedback as to where and how folks are using backports. Please
> let me and Julia know -- or if the information is not sensitive please
> feel free to just reply to this thread and share with others. All
> feedback is greatly appreciated. If you are OK in having us list or
> generalize your usage please indicate so.
In OpenWrt, we use backports to stay up to date with current wireless
drivers without being forced into frequently updating the kernels as
well. We support many different platforms, and sometimes it takes a
while to update the kernel on them.
Using backports significantly reduces the amount of effort that we need
to put into maintaining the wireless drivers.
When making changes to wireless drivers or mac80211, which I submit
upstream, I also develop them in our most recent backports snapshot
first (typically generated from wireless-testing). When they are done, I
port them to a proper git tree and submit them from there.

In OpenWrt, we typically update the backports snapshot outside of the
normal kernel release cycle (always to latest wireless-testing) and
stabilize that by cherry-picking individual patches on top of it.

- Felix

Re: Uses of Linux backports in the industry

[Julia Lawall]

On Mon, 1 Jun 2015, Felix Fietkau wrote:

> On 2015-05-29 04:54, Luis R. Rodriguez wrote:
> > Me and Julia are working on a paper which evaluates use of Coccinelle
> > on backports, a preliminary draft of such paper can be found on github
> > [0]. We are making some tweaks to this, one of which is covering the
> > uses of Linux backports [1] in the industry, for this we'd like to try
> > to get feedback as to where and how folks are using backports. Please
> > let me and Julia know -- or if the information is not sensitive please
> > feel free to just reply to this thread and share with others. All
> > feedback is greatly appreciated. If you are OK in having us list or
> > generalize your usage please indicate so.
> In OpenWrt, we use backports to stay up to date with current wireless
> drivers without being forced into frequently updating the kernels as
> well. We support many different platforms, and sometimes it takes a
> while to update the kernel on them.
> Using backports significantly reduces the amount of effort that we need
> to put into maintaining the wireless drivers.
> When making changes to wireless drivers or mac80211, which I submit
> upstream, I also develop them in our most recent backports snapshot
> first (typically generated from wireless-testing). When they are done, I
> port them to a proper git tree and submit them from there.
> 
> In OpenWrt, we typically update the backports snapshot outside of the
> normal kernel release cycle (always to latest wireless-testing) and
> stabilize that by cherry-picking individual patches on top of it.

Thanks for the feedback!

julia

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 01.06.2015 um 20:50 schrieb Felix Fietkau:

> We support many different platforms, and sometimes it takes a
> while to update the kernel on them.

Just curious: what makes these platforms so different from each other,
so it takes so much time to port to new kernel ? Are there things so
special, that they can't go into mainline ?

In a few weeks, I'll (hopefully) have a few weeks off and plan to play
around w/ some spare DSL routers. Maybe we can have a talk on how to
get more things mainlined, if you wish :)

> Using backports significantly reduces the amount of effort that we need
> to put into maintaining the wireless drivers.
> When making changes to wireless drivers or mac80211, which I submit
> upstream, I also develop them in our most recent backports snapshot
> first (typically generated from wireless-testing). When they are done, I
> port them to a proper git tree and submit them from there.

How much has changed from your old baseline compared to current
mainline ? I could imagine, in your area it might not be as much
as in others (eg. graphics or v4l subsystem).

> In OpenWrt, we typically update the backports snapshot outside of the
> normal kernel release cycle (always to latest wireless-testing) and
> stabilize that by cherry-picking individual patches on top of it.

Can you estimate the required workforce ?
Some statistics on that would be really nice.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Uses of Linux backports in the industry

[Enrico Weigelt, metux IT consult]

Am 01.06.2015 um 20:50 schrieb Felix Fietkau:

> We support many different platforms, and sometimes it takes a
> while to update the kernel on them.

Just curious: what makes these platforms so different from each other,
so it takes so much time to port to new kernel ? Are there things so
special, that they can't go into mainline ?

In a few weeks, I'll (hopefully) have a few weeks off and plan to play
around w/ some spare DSL routers. Maybe we can have a talk on how to
get more things mainlined, if you wish :)

> Using backports significantly reduces the amount of effort that we need
> to put into maintaining the wireless drivers.
> When making changes to wireless drivers or mac80211, which I submit
> upstream, I also develop them in our most recent backports snapshot
> first (typically generated from wireless-testing). When they are done, I
> port them to a proper git tree and submit them from there.

How much has changed from your old baseline compared to current
mainline ? I could imagine, in your area it might not be as much
as in others (eg. graphics or v4l subsystem).

> In OpenWrt, we typically update the backports snapshot outside of the
> normal kernel release cycle (always to latest wireless-testing) and
> stabilize that by cherry-picking individual patches on top of it.

Can you estimate the required workforce ?
Some statistics on that would be really nice.


cu
--
Enrico Weigelt, metux IT consult
+49-151-27565287
MELAG Medizintechnik oHG Sitz Berlin Registergericht AG Charlottenburg HRA 21333 B

Wichtiger Hinweis: Diese Nachricht kann vertrauliche oder nur für einen begrenzten Personenkreis bestimmte Informationen enthalten. Sie ist ausschließlich für denjenigen bestimmt, an den sie gerichtet worden ist. Wenn Sie nicht der Adressat dieser E-Mail sind, dürfen Sie diese nicht kopieren, weiterleiten, weitergeben oder sie ganz oder teilweise in irgendeiner Weise nutzen. Sollten Sie diese E-Mail irrtümlich erhalten haben, so benachrichtigen Sie bitte den Absender, indem Sie auf diese Nachricht antworten. Bitte löschen Sie in diesem Fall diese Nachricht und alle Anhänge, ohne eine Kopie zu behalten.
Important Notice: This message may contain confidential or privileged information. It is intended only for the person it was addressed to. If you are not the intended recipient of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you received this email in error please notify the sender by replying and delete this message and any attachments without retaining a copy.

Re: Uses of Linux backports in the industry

[Szymon Janc]

Hi,

On Thursday 28 of May 2015 19:54:57 Luis R. Rodriguez wrote:
> Me and Julia are working on a paper which evaluates use of Coccinelle
> on backports, a preliminary draft of such paper can be found on github
> [0]. We are making some tweaks to this, one of which is covering the
> uses of Linux backports [1] in the industry, for this we'd like to try
> to get feedback as to where and how folks are using backports. Please
> let me and Julia know -- or if the information is not sensitive please
> feel free to just reply to this thread and share with others. All
> feedback is greatly appreciated. If you are OK in having us list or
> generalize your usage please indicate so.

We use backports in 'BlueZ for Android' project [0]. Our reference tree [1] 
based on AOSP for Nexus devices is shipped with 3.4 kernel which is way too 
old for any Bluetooth Low Energy. Thanks to backports we can easily update BT 
subsystem while interesting feature enters bluetooth-next (via linux-next).
If there were no backports our project would be much harder if not impossible 
to achieve!

Since we use Android kernel we do have to fix some build issues from time to 
time but in overall this is quite easy and really straight forward.

[0] https://01.org/bluez-android
[1] https://code.google.com/p/aosp-bluez/

-- 
BR
Szymon Janc