Kernel specific branches question

Kernel specific branches question

[Dan Kephart]

SGksDQogDQpBdCBMYWlyZCwgd2UgdXNlIGJhY2twb3J0cyBmb3Igc3VwcG9ydGluZyBvdXIgd2ly
ZWxlc3MgbW9kdWxlcywgc3BlY2lmaWNhbGx5IHVzaW5nIGFuIHVubW9kaWZpZWQgYnJjbWZtYWMs
IGEgZmFpcmx5IG1vZGlmaWVkIGF0aDZrbCwgYW5kIGFuIG91dCBvZiB0cmVlIGN1c3RvbSB2ZXJz
aW9uIG9mIHRoZSBub24tdXBzdHJlYW1lZCBtd2x3aWZpLCBhbmQgYWxzbyB1c2luZyB1bm1vZGlm
aWVkIEJsdWV0b290aC4gIEF0IHRoZSBtb21lbnQsIHdlIGFyZSBiYWNrcG9ydGluZyBvdXIgZm9y
ayBvZiB0aGUgNC45IGtlcm5lbCBmb3IgdGhvc2UgZHJpdmVycy4gIEkgcHVsbGVkIGEgY29weSBv
ZiB0aGUgZ2l0IHJlcG8gYSBmZXcgbW9udGhzIGJhY2sgYW5kIHdlIGhhdmUgd29ya2VkIGZyb20g
dGhhdC4gIEnigJltIHdvbmRlcmluZyBob3cgd2UgY2FuIGhlbHAgcmVzdXJyZWN0IHRoZSBwcm9j
ZXNzIG9mIGhhdmluZyBicmFuY2hlcyBmb3Igc3BlY2lmaWMga2VybmVsIHJlbGVhc2VzIGFuZCB0
aGUgdGVzdGluZyB0aGF0IHdhcyBkb25lIGZvciB0aG9zZSBhcyBzZWVuIGluIHRoaXMgY29tbWl0
Og0KaHR0cHM6Ly9naXQua2VybmVsLm9yZy9wdWIvc2NtL2xpbnV4L2tlcm5lbC9naXQvYmFja3Bv
cnRzL2JhY2twb3J0cy5naXQvY29tbWl0Lz9oPWxpbnV4LTQuNC55JmlkPWJlYzQwMzdhNTRkZDBl
MWExYWVlZmU3Mzg1ZmFmODJiYWM0NGQ2YTANCg0KVGhhbmtzDQpEYW4NCg0K
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Kernel specific branches question

[Johannes Berg]

Hi Dan,

> At Laird, we use backports for supporting our wireless modules,
> specifically using an unmodified brcmfmac, a fairly modified ath6kl,
> and an out of tree custom version of the non-upstreamed mwlwifi, and
> also using unmodified Bluetooth.  

:)

> At the moment, we are backporting our fork of the 4.9 kernel for
> those drivers.  I pulled a copy of the git repo a few months back and
> we have worked from that.  I’m wondering how we can help resurrect
> the process of having branches for specific kernel releases and the
> testing that was done for those as seen in this commit:
> https://git.kernel.org/pub/scm/linux/kernel/git/backports/backports.g
> it/commit/?h=linux-4.4.y&id=bec4037a54dd0e1a1aeefe7385faf82bac44d6a0

So the testing thing there is just the output from the "ckmake" script
that's there in the repository. It needs a bunch of disk space and lots
of CPU time, but otherwise shouldn't be that hard to do. We still have
the compat build box, and I still have a plan to put some kind of
automation in place, but haven't really had time for it yet.

However, this is pretty different from how most other people seem to
work?

Since kernel releases are sort of ephemeral, and drivers are
continually developed along with the kernel, we typically see usage
more of "latest backport + latest kernel" to get to "latest driver for
use on any kernel".

Additionally, backports for a fixed kernel version like 4.9 really
should be pretty stable, so once you have it, it shouldn't really need
to change? So it seems to me like a tag on master would be sufficient.

Now, we're not actually doing a good job of following any of upstream,
linux-next or wireless-testing with backports right now - we should
probably pick one of those, or multiple of those, and create branches
in the backports.git for each one of those, rather than having a
"master" branch.

To do that, I think we should actually set up a build bot to do a daily
tests of applying backports on those trees, and then compiling the
result on all (supported) kernels. This can then email the list, and
somebody can fix the issues.

Sounds like a plan?

Note that I'm going to go no sabbatical for July/August, so I won't be
working on this then either though.

johannes
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Kernel specific branches question

[Dan Kephart]

Hi Johannes,
=A0  =20
>Hi Dan,

>> At Laird, we use backports for supporting our wireless modules,
>> specifically using an unmodified brcmfmac, a fairly modified ath6kl,
>> and an out of tree custom version of the non-upstreamed mwlwifi, and
>> also using unmodified Bluetooth.=A0=A0

>:)

>> At the moment, we are backporting our fork of the 4.9 kernel for
>> those drivers.=A0=A0I pulled a copy of the git repo a few months back an=
d
>> we have worked from that.=A0=A0I=92m wondering how we can help resurrect
>> the process of having branches for specific kernel releases and the
>> testing that was done for those as seen in this commit:
>>  https://git.kernel.org/pub/scm/linux/kernel/git/backports/backports.g
>> it/commit/?h=3Dlinux-4.4.y&id=3Dbec4037a54dd0e1a1aeefe7385faf82bac44d6a0

>So the testing thing there is just the output from the "ckmake" script
>that's there in the repository. It needs a bunch of disk space and lots
>of CPU time, but otherwise shouldn't be that hard to do. We still have
>the compat build box, and I still have a plan to put some kind of
>automation in place, but haven't really had time for it yet.


This is great information, we will probably attempt to set this up
internally for our QA testing.

>However, this is pretty different from how most other people seem to
>work?

>Since kernel releases are sort of ephemeral, and drivers are
>continually developed along with the kernel, we typically see usage
>more of "latest backport + latest kernel" to get to "latest driver for
>use on any kernel".

I think only wireless module vendors would have our use case.  Just to=20
elaborate a bit more.  We sell several system on modules
with 802.11 and bluetooth combo modules. We also sell the wireless
combo modules separately in a variety of form factors for integrating=20
into other systems. We jump the kernels on those SOMs to the latest LTS=20
kernel when released.  We do all our wireless module bring up, debugging,=20
baseline, and QA testing of those wireless modules using LTS kernels
using our SOMs.  To support software releases for those wireless only modul=
es
we use backports on those SOMs kernel to release a driver that builds for=20
many different kernels and rely on the QA testing that pertains to=20
the wireless module on the SOM.  This backports tarball is released with
the firmware and the wpa_supplicant that was used during this
QA process.  If a specific major release works well for the customer, they=
=20
may stay on this release and only want a bug fix or two. =20
We continually move forward with new releases but maintain those older
releases for customers if required.

>Additionally, backports for a fixed kernel version like 4.9 really
>should be pretty stable, so once you have it, it shouldn't really need
>to change? So it seems to me like a tag on master would be sufficient.

I agree this really is all we would need.

>Now, we're not actually doing a good job of following any of upstream,
>linux-next or wireless-testing with backports right now - we should
>probably pick one of those, or multiple of those, and create branches
>in the backports.git for each one of those, rather than having a
>"master" branch.

My preference would be linux-next, but it would be interesting to have both=
.

>To do that, I think we should actually set up a build bot to do a daily
>tests of applying backports on those trees, and then compiling the
>result on all (supported) kernels. This can then email the list, and
>somebody can fix the issues.

>Sounds like a plan?

This sounds fantastic and would enable us to help keep the project up
to date :).

>Note that I'm going to go no sabbatical for July/August, so I won't be
>working on this then either though.

Gives us time to see if we can get the cmake script set up ourselves as wel=
l.

Thanks for the reply.  Looking forward to helping out!

Dan

    =
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: Kernel specific branches question

[Johannes Berg]

Hi Dan,

> > However, this is pretty different from how most other people seem
> > to work?
> > Since kernel releases are sort of ephemeral, and drivers are
> > continually developed along with the kernel, we typically see usage
> > more of "latest backport + latest kernel" to get to "latest driver
> > for use on any kernel".
> 
> I think only wireless module vendors would have our use case.  Just
> to elaborate a bit more.  We sell several system on modules
> with 802.11 and bluetooth combo modules. We also sell the wireless
> combo modules separately in a variety of form factors for
> integrating into other systems. We jump the kernels on those SOMs to
> the latest LTS kernel when released.  We do all our wireless module
> bring up, debugging, baseline, and QA testing of those wireless
> modules using LTS kernels using our SOMs.  To support software
> releases for those wireless only modules we use backports on those
> SOMs kernel to release a driver that builds for many different
> kernels and rely on the QA testing that pertains to the wireless
> module on the SOM.  This backports tarball is released with
> the firmware and the wpa_supplicant that was used during this
> QA process.  If a specific major release works well for the customer,
> they may stay on this release and only want a bug fix or two.  
> We continually move forward with new releases but maintain those
> older releases for customers if required.

Well, I work for Intel, so we're a wireless (module) vendor too :-)

But yeah - we do things differently - we don't even really support the
upstream LTS releases or kernels per se, for product work we *only*
support our backports-based "core release" tree, which is also where we
do all of our development. This branches off from wherever we're at
wrt. upstream and/or our driver development, and then we stabilize
that. It's always (intended to be) stable enough so that this
stabilization doesn't take all that much time.

Anyway, I see what you're doing, and it seems like a reasonable thing
to do.

> > Additionally, backports for a fixed kernel version like 4.9 really
> > should be pretty stable, so once you have it, it shouldn't really
> > need to change? So it seems to me like a tag on master would be
> > sufficient.
> 
> I agree this really is all we would need.

We'd still have to get to a point where we can churn those out - I
think automation could help here, to maintain the "linus" branch of the
backports tree we can "gentree" it every night or so, send a warning if
that breaks, or run ckmake to make sure it still builds against all the
other trees.
This would be the generic "make sure it applies/builds" part.

In addition, everyone else who has skin in the game could contribute a
cron job that every night does the same steps, but also runs some tests
when  If this gets tagged in the repo or so everyone could even easily
pull it down and run some tests on it for their specific driver/etc.

> > Now, we're not actually doing a good job of following any of
> > upstream,
> > linux-next or wireless-testing with backports right now - we should
> > probably pick one of those, or multiple of those, and create
> > branches
> > in the backports.git for each one of those, rather than having a
> > "master" branch.
> 
> My preference would be linux-next, but it would be interesting to
> have both.

My priority order would be somewhere along the lines of

 * Linus's tree
 * linux-next
 * wireless-testing

though most users probably actually want exactly the other way around
;-) But this is what helps us most for our development.

Once we have the setup, we can probably deal with having three branches
- linux-next would have to get the fixes first, then they'd percolate
to wireless-testing, and eventually to the "linus" branch.


I already have a task ticket for me to look at all of this when I
return, but I'm happy if you want to do anything along these lines in
the meantime. I don't think it should be very difficult.

johannes
--
To unsubscribe from this list: send the line "unsubscribe backports" in