* [RFC] Rebuild external kernel modules on kernel change
@ 2009-04-04 13:31 Koen Kooi
2009-04-04 15:51 ` Frans Meulenbroeks
` (4 more replies)
0 siblings, 5 replies; 21+ messages in thread
From: Koen Kooi @ 2009-04-04 13:31 UTC (permalink / raw)
To: openembedded-devel
Hi,
For beagleboard I have a few things I need to rebuild everytime the
kernel changes:
* powervr kerneldrivers
* sdma kernel module
* dmai kernel module
* codec-engine
And I have roughly two kinds of kernel changes:
1) version upgrade (e.g. 2.6.29 -> 2.6.29)
2) config changes (e.g. enable ethernet bridging)
The first type of change could be solved by putting KERNEL_VERSION in PV
or PR, but that needs a non-trivial amount of python since the
information isn't available at parsing time (exactly like debian.bbclass).
The second kind of change is a lot harder to detect, unless we start
storing md5sums for kernel defconfigs.
I have a lowtech proposal for this:
-----
conf/bitbake.conf:
# Define a PR for kernels that machines can override so things like
# modules get rebuilt
MACHINE_KERNEL_PR ?= "r0"
conf/machine/beagleboard.conf:
# Increase this everytime the kernel changes
MACHINE_KERNEL_PR = "r39"
classes/kernel.bbclass:
# A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force
# rebuilds for kernel and external modules
PR = "${MACHINE_KERNEL_PR}"
class/module-base.bbclass:
# A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force
# rebuilds for kernel and external modules
PR = "${MACHINE_KERNEL_PR}"
-----
I don't really like this method, but I'm having a hard time coming up
with a decent solution that:
a) works
b) requires less or equal manual work
c) keeps PR in sync between different buildhosts
regards,
Koen
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 13:31 [RFC] Rebuild external kernel modules on kernel change Koen Kooi @ 2009-04-04 15:51 ` Frans Meulenbroeks 2009-04-04 16:44 ` Koen Kooi 2009-04-04 17:46 ` Otavio Salvador ` (3 subsequent siblings) 4 siblings, 1 reply; 21+ messages in thread From: Frans Meulenbroeks @ 2009-04-04 15:51 UTC (permalink / raw) To: openembedded-devel Hm, I don't like that one needs to touch conf/machine/beagleboard.conf each time the kernel is updated. This is likely to be forgotten at some time. Can't we use something in the linux-omap-*.bb file ? Btw, I always bump PR if I change something in the config. Isn't it possible to add rebuilding those modules to e.g.. virtual/kernel? Frans 2009/4/4 Koen Kooi <k.kooi@student.utwente.nl>: > Hi, > > For beagleboard I have a few things I need to rebuild everytime the kernel > changes: > > * powervr kerneldrivers > * sdma kernel module > * dmai kernel module > * codec-engine > > And I have roughly two kinds of kernel changes: > > 1) version upgrade (e.g. 2.6.29 -> 2.6.29) > 2) config changes (e.g. enable ethernet bridging) > > The first type of change could be solved by putting KERNEL_VERSION in PV or > PR, but that needs a non-trivial amount of python since the information > isn't available at parsing time (exactly like debian.bbclass). > The second kind of change is a lot harder to detect, unless we start storing > md5sums for kernel defconfigs. > > I have a lowtech proposal for this: > > ----- > > conf/bitbake.conf: > # Define a PR for kernels that machines can override so things like > # modules get rebuilt > MACHINE_KERNEL_PR ?= "r0" > > conf/machine/beagleboard.conf: > # Increase this everytime the kernel changes > MACHINE_KERNEL_PR = "r39" > > classes/kernel.bbclass: > # A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force > # rebuilds for kernel and external modules > PR = "${MACHINE_KERNEL_PR}" > > class/module-base.bbclass: > # A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force > # rebuilds for kernel and external modules > PR = "${MACHINE_KERNEL_PR}" > > ----- > > I don't really like this method, but I'm having a hard time coming up with a > decent solution that: > > a) works > b) requires less or equal manual work > c) keeps PR in sync between different buildhosts > > regards, > > Koen > > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 15:51 ` Frans Meulenbroeks @ 2009-04-04 16:44 ` Koen Kooi 2009-04-04 20:18 ` Frans Meulenbroeks 0 siblings, 1 reply; 21+ messages in thread From: Koen Kooi @ 2009-04-04 16:44 UTC (permalink / raw) To: openembedded-devel On 04-04-09 17:51, Frans Meulenbroeks wrote: > Hm, > > I don't like that one needs to touch conf/machine/beagleboard.conf > each time the kernel is updated. > This is likely to be forgotten at some time. > Can't we use something in the linux-omap-*.bb file ? You can't set global vars in recipes. regards, Koen > Btw, I always bump PR if I change something in the config. > > Isn't it possible to add rebuilding those modules to e.g.. virtual/kernel? > > Frans > > 2009/4/4 Koen Kooi<k.kooi@student.utwente.nl>: >> Hi, >> >> For beagleboard I have a few things I need to rebuild everytime the kernel >> changes: >> >> * powervr kerneldrivers >> * sdma kernel module >> * dmai kernel module >> * codec-engine >> >> And I have roughly two kinds of kernel changes: >> >> 1) version upgrade (e.g. 2.6.29 -> 2.6.29) >> 2) config changes (e.g. enable ethernet bridging) >> >> The first type of change could be solved by putting KERNEL_VERSION in PV or >> PR, but that needs a non-trivial amount of python since the information >> isn't available at parsing time (exactly like debian.bbclass). >> The second kind of change is a lot harder to detect, unless we start storing >> md5sums for kernel defconfigs. >> >> I have a lowtech proposal for this: >> >> ----- >> >> conf/bitbake.conf: >> # Define a PR for kernels that machines can override so things like >> # modules get rebuilt >> MACHINE_KERNEL_PR ?= "r0" >> >> conf/machine/beagleboard.conf: >> # Increase this everytime the kernel changes >> MACHINE_KERNEL_PR = "r39" >> >> classes/kernel.bbclass: >> # A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force >> # rebuilds for kernel and external modules >> PR = "${MACHINE_KERNEL_PR}" >> >> class/module-base.bbclass: >> # A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force >> # rebuilds for kernel and external modules >> PR = "${MACHINE_KERNEL_PR}" >> >> ----- >> >> I don't really like this method, but I'm having a hard time coming up with a >> decent solution that: >> >> a) works >> b) requires less or equal manual work >> c) keeps PR in sync between different buildhosts >> >> regards, >> >> Koen >> >> >> _______________________________________________ >> Openembedded-devel mailing list >> Openembedded-devel@lists.openembedded.org >> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel >> ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 16:44 ` Koen Kooi @ 2009-04-04 20:18 ` Frans Meulenbroeks 2009-04-04 21:14 ` Koen Kooi 0 siblings, 1 reply; 21+ messages in thread From: Frans Meulenbroeks @ 2009-04-04 20:18 UTC (permalink / raw) To: openembedded-devel 2009/4/4 Koen Kooi <k.kooi@student.utwente.nl>: > On 04-04-09 17:51, Frans Meulenbroeks wrote: >> >> Hm, >> >> I don't like that one needs to touch conf/machine/beagleboard.conf >> each time the kernel is updated. >> This is likely to be forgotten at some time. >> Can't we use something in the linux-omap-*.bb file ? > > You can't set global vars in recipes. Thinking of it the real issue is that we do have DEPENDS that takes care that if A needs B, B is build before A (and I guess that after updating B a bitbake of A will result in a recompile, even if A's PR is not changed. Maybe a solution could be to be able to rebuild a package plus all its dependencies (the information is there in the DEPENDS lines). Frans. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 20:18 ` Frans Meulenbroeks @ 2009-04-04 21:14 ` Koen Kooi 0 siblings, 0 replies; 21+ messages in thread From: Koen Kooi @ 2009-04-04 21:14 UTC (permalink / raw) To: openembedded-devel On 04-04-09 22:18, Frans Meulenbroeks wrote: > 2009/4/4 Koen Kooi<k.kooi@student.utwente.nl>: >> On 04-04-09 17:51, Frans Meulenbroeks wrote: >>> >>> Hm, >>> >>> I don't like that one needs to touch conf/machine/beagleboard.conf >>> each time the kernel is updated. >>> This is likely to be forgotten at some time. >>> Can't we use something in the linux-omap-*.bb file ? >> >> You can't set global vars in recipes. > > Thinking of it the real issue is that we do have DEPENDS that takes > care that if A needs B, B is build before A (and I guess that after > updating B a bitbake of A will result in a recompile, even if A's PR > is not changed. > > Maybe a solution could be to be able to rebuild a package plus all its > dependencies (the information is there in the DEPENDS lines). We can already do that with BB_STAMP_POLICY, the problem present is that it will generate packages with the same PV and PR, so it won't show up as upgrade, so users get a broken installation after the package manager upgrades the kernel, but not the modules since PV and PR didn't change for those. regards, Koen ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 13:31 [RFC] Rebuild external kernel modules on kernel change Koen Kooi 2009-04-04 15:51 ` Frans Meulenbroeks @ 2009-04-04 17:46 ` Otavio Salvador 2009-04-05 16:43 ` Koen Kooi 2009-04-04 20:46 ` Jeremy Lainé ` (2 subsequent siblings) 4 siblings, 1 reply; 21+ messages in thread From: Otavio Salvador @ 2009-04-04 17:46 UTC (permalink / raw) To: openembedded-devel; +Cc: openembedded-devel On Sat, Apr 4, 2009 at 10:31 AM, Koen Kooi <k.kooi@student.utwente.nl> wrote: > Hi, [...] > I don't really like this method, but I'm having a hard time coming up with a > decent solution that: [...] I agree that is not beauty but it works. I also belive it could be used in other recipes too as initramfs-images. This way forcing the rebuild of it if the kernel is changed and machine revision is bumped. Fully support it. +1 -- Otavio Salvador O.S. Systems E-mail: otavio@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 17:46 ` Otavio Salvador @ 2009-04-05 16:43 ` Koen Kooi 2009-04-05 17:07 ` Koen Kooi 0 siblings, 1 reply; 21+ messages in thread From: Koen Kooi @ 2009-04-05 16:43 UTC (permalink / raw) To: openembedded-devel On 04-04-09 19:46, Otavio Salvador wrote: > On Sat, Apr 4, 2009 at 10:31 AM, Koen Kooi<k.kooi@student.utwente.nl> wrote: >> Hi, > [...] >> I don't really like this method, but I'm having a hard time coming up with a >> decent solution that: > [...] > > I agree that is not beauty but it works. > > I also belive it could be used in other recipes too as initramfs-images. This > way forcing the rebuild of it if the kernel is changed and machine revision > is bumped. > > Fully support it. > > +1 Thanks, I'll try cooking up a patch for beagleboard tomorrow to show a real life example. regards, Koen ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-05 16:43 ` Koen Kooi @ 2009-04-05 17:07 ` Koen Kooi 0 siblings, 0 replies; 21+ messages in thread From: Koen Kooi @ 2009-04-05 17:07 UTC (permalink / raw) To: openembedded-devel On 05-04-09 18:43, Koen Kooi wrote: > On 04-04-09 19:46, Otavio Salvador wrote: >> On Sat, Apr 4, 2009 at 10:31 AM, Koen Kooi<k.kooi@student.utwente.nl> >> wrote: >>> Hi, >> [...] >>> I don't really like this method, but I'm having a hard time coming up >>> with a >>> decent solution that: >> [...] >> >> I agree that is not beauty but it works. >> >> I also belive it could be used in other recipes too as >> initramfs-images. This >> way forcing the rebuild of it if the kernel is changed and machine >> revision >> is bumped. >> >> Fully support it. >> >> +1 > > Thanks, I'll try cooking up a patch for beagleboard tomorrow to show a > real life example. Here it is, this will be done in 2 commits: diff --git a/classes/kernel.bbclass b/classes/kernel.bbclass index 17e8941..954c407 100644 --- a/classes/kernel.bbclass +++ b/classes/kernel.bbclass @@ -64,6 +64,10 @@ export CMDLINE_CONSOLE = "console=${@bb.data.getVar("KERNEL_CONSOLE",d,1) or "tt KERNEL_VERSION = "${@get_kernelversion('${S}')}" KERNEL_MAJOR_VERSION = "${@get_kernelmajorversion('${KERNEL_VERSION}')}" +# A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force +# rebuilds for kernel and external modules +PR = "${MACHINE_KERNEL_PR}" + KERNEL_LOCALVERSION ?= "" diff --git a/classes/module-base.bbclass b/classes/module-base.bbclass index c98bace..bc53e1b 100644 --- a/classes/module-base.bbclass +++ b/classes/module-base.bbclass @@ -5,6 +5,10 @@ inherit kernel-arch export OS = "${TARGET_OS}" export CROSS_COMPILE = "${TARGET_PREFIX}" +# A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force +# rebuilds for kernel and external modules +PR = "${MACHINE_KERNEL_PR}" + diff --git a/conf/bitbake.conf b/conf/bitbake.conf index a0d652a..136f01c 100644 --- a/conf/bitbake.conf +++ b/conf/bitbake.conf @@ -162,6 +162,10 @@ EXTENDPEVER = "${@['','${PE\x7d:'][bb.data.getVar('PE',d,1) > 0]}" EXTENDPV = "${EXTENDPEVER}${PV}-${PR}${DISTRO_PR}" P = "${PN}-${PV}" +# Define a PR for kernels that machines can override so things like +# modules get rebuilt +MACHINE_KERNEL_PR ?= "r0" + diff --git a/conf/machine/beagleboard.conf b/conf/machine/beagleboard.conf index 78c6301..3d55ebc 100644 --- a/conf/machine/beagleboard.conf +++ b/conf/machine/beagleboard.conf @@ -25,6 +25,7 @@ EXTRA_IMAGECMD_jffs2 = "-lnp " SERIAL_CONSOLE = "115200 ttyS2" PREFERRED_PROVIDER_virtual/kernel = "linux-omap" +MACHINE_KERNEL_PR = "r20" KERNEL_IMAGETYPE = "uImage" diff --git a/recipes/dsplink/gstreamer-ti_svn.bb b/recipes/dsplink/gstreamer-ti_svn.bb index ab705f3..6944ad5 100644 --- a/recipes/dsplink/gstreamer-ti_svn.bb +++ b/recipes/dsplink/gstreamer-ti_svn.bb @@ -4,7 +4,7 @@ SRC_URI = "svn://gforge.ti.com/svn/gstreamer_ti/trunk;module=gstreamer_ti;proto= " SRCREV = "160" -PR = "r7" +PR = "${MACHINE_KERNEL_PR}" diff --git a/recipes/dsplink/ti-codec-engine_2.21.bb b/recipes/dsplink/ti-codec-engine_2.21.bb index 5b1d5e0..f0b0d7e 100644 --- a/recipes/dsplink/ti-codec-engine_2.21.bb +++ b/recipes/dsplink/ti-codec-engine_2.21.bb @@ -8,7 +8,6 @@ RDEPENDS = "update-modules" inherit module # tconf from xdctools dislikes '.' in pwd :/ -PR = "r19" PV = "221" diff --git a/recipes/dsplink/ti-codec-engine_2.23.bb b/recipes/dsplink/ti-codec-engine_2.23.bb index a42c33b..bbf54d4 100644 --- a/recipes/dsplink/ti-codec-engine_2.23.bb +++ b/recipes/dsplink/ti-codec-engine_2.23.bb @@ -9,7 +9,6 @@ inherit module DEFAULT_PREFERENCE = "-1" # tconf from xdctools dislikes '.' in pwd :/ -PR = "r0" PV = "223" diff --git a/recipes/dsplink/ti-dmai_svn.bb b/recipes/dsplink/ti-dmai_svn.bb index bcbaea8..76fa74a 100644 --- a/recipes/dsplink/ti-dmai_svn.bb +++ b/recipes/dsplink/ti-dmai_svn.bb @@ -15,7 +15,7 @@ SRCREV = "36" S = "${WORKDIR}/BRIJESH_GIT_022309/davinci_multimedia_application_interface/dmai" # Yes, the xdc stuff still breaks with a '.' in PWD PV = "120+svnr${SRCREV}" -PR = "r16" +PR = "${MACHINE_KERNEL_PR}" diff --git a/recipes/linux/linux-omap_2.6.28.bb b/recipes/linux/linux-omap_2.6.28.bb index bc085c7..92de11e 100644 --- a/recipes/linux/linux-omap_2.6.28.bb +++ b/recipes/linux/linux-omap_2.6.28.bb @@ -12,7 +12,6 @@ DEFAULT_PREFERENCE_omap5912osk = "1" SRCREV = "79d042a081d3e467c735bb0d9569ed6296f85a3c" PV = "2.6.28" -PR = "r19" diff --git a/recipes/linux/linux-omap_2.6.29.bb b/recipes/linux/linux-omap_2.6.29.bb index a2391ee..43be953 100644 --- a/recipes/linux/linux-omap_2.6.29.bb +++ b/recipes/linux/linux-omap_2.6.29.bb @@ -10,7 +10,7 @@ DEFAULT_PREFERENCE_overo = "1" SRCREV = "58cf2f1425abfd3a449f9fe985e48be2d2555022" -PR = "r7+gitr${SRCREV}" +PR_append = "+gitr${SRCREV}" diff --git a/recipes/powervr-drivers/omap3-sgx-modules_1.3.13.1397.bb b/recipes/powervr-drivers/omap3-sgx-modules_1.3.13.1397.bb index 985a0c0..fc9f02f 100644 --- a/recipes/powervr-drivers/omap3-sgx-modules_1.3.13.1397.bb +++ b/recipes/powervr-drivers/omap3-sgx-modules_1.3.13.1397.bb @@ -1,8 +1,6 @@ DESCRIPTION = "Kernel drivers for the PowerVR SGX chipset found in the omap3 SoCs" LICENSE = "GPLv2" -PR = "r6" - inherit module ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 13:31 [RFC] Rebuild external kernel modules on kernel change Koen Kooi 2009-04-04 15:51 ` Frans Meulenbroeks 2009-04-04 17:46 ` Otavio Salvador @ 2009-04-04 20:46 ` Jeremy Lainé 2009-04-05 22:43 ` Richard Purdie 2009-06-01 16:58 ` Tom Rini 4 siblings, 0 replies; 21+ messages in thread From: Jeremy Lainé @ 2009-04-04 20:46 UTC (permalink / raw) To: openembedded-devel; +Cc: openembedded-devel H > conf/machine/beagleboard.conf: > # Increase this everytime the kernel changes > MACHINE_KERNEL_PR = "r39" > > classes/kernel.bbclass: > # A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force > # rebuilds for kernel and external modules > PR = "${MACHINE_KERNEL_PR}" > > class/module-base.bbclass: > # A machine.conf or local.conf can increase MACHINE_KERNEL_PR to force > # rebuilds for kernel and external modules > PR = "${MACHINE_KERNEL_PR}" > > ----- > > I don't really like this method, but I'm having a hard time coming up > with a decent solution that: > > a) works > b) requires less or equal manual work > c) keeps PR in sync between different buildhosts > > regards, > > Koen > > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel -- Jeremy LAINE Bolloré telecom | 11bis, rue Scribe | F-75009 Paris ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 13:31 [RFC] Rebuild external kernel modules on kernel change Koen Kooi ` (2 preceding siblings ...) 2009-04-04 20:46 ` Jeremy Lainé @ 2009-04-05 22:43 ` Richard Purdie 2009-04-06 7:16 ` Koen Kooi 2009-06-01 16:58 ` Tom Rini 4 siblings, 1 reply; 21+ messages in thread From: Richard Purdie @ 2009-04-05 22:43 UTC (permalink / raw) To: openembedded-devel On Sat, 2009-04-04 at 15:31 +0200, Koen Kooi wrote: > For beagleboard I have a few things I need to rebuild everytime the > kernel changes: > > * powervr kerneldrivers > * sdma kernel module > * dmai kernel module > * codec-engine > > And I have roughly two kinds of kernel changes: > > 1) version upgrade (e.g. 2.6.29 -> 2.6.29) > 2) config changes (e.g. enable ethernet bridging) > > The first type of change could be solved by putting KERNEL_VERSION in PV > or PR, but that needs a non-trivial amount of python since the > information isn't available at parsing time (exactly like debian.bbclass). > The second kind of change is a lot harder to detect, unless we start > storing md5sums for kernel defconfigs. > > I have a lowtech proposal for this: The thing is if we start doing this for kernels, we'll probably end up having to do it for other things and the end result will be a mess. A better fix for this is BB_STAMP_POLICY = "whitelist" which will at least cause the right things to rebuild. No, this doesn't bump the PR when packages change but I'd actually like to see some new code to fix this problem and several others once and for all. The idea would be to "compare" the current build with any previous ones and error if bad differences are detected. In this case if PR = X had already built that would be an "error" which the system may work around by using PR = "X-Z1" in the output packages. You could teach that code interesting (even distribution specific) logic e.g. to ignore changes in system libraries like libc. This would imply this comparison data would have to come from a shared source for multiple build machines for a distribution like Angstrom. No, this solution isn't simple but hacking around the problem just builds a house on top of quicksand ;-). Cheers, Richard ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-05 22:43 ` Richard Purdie @ 2009-04-06 7:16 ` Koen Kooi 0 siblings, 0 replies; 21+ messages in thread From: Koen Kooi @ 2009-04-06 7:16 UTC (permalink / raw) To: openembedded-devel On 06-04-09 00:43, Richard Purdie wrote: > On Sat, 2009-04-04 at 15:31 +0200, Koen Kooi wrote: >> For beagleboard I have a few things I need to rebuild everytime the >> kernel changes: >> >> * powervr kerneldrivers >> * sdma kernel module >> * dmai kernel module >> * codec-engine >> >> And I have roughly two kinds of kernel changes: >> >> 1) version upgrade (e.g. 2.6.29 -> 2.6.29) >> 2) config changes (e.g. enable ethernet bridging) >> >> The first type of change could be solved by putting KERNEL_VERSION in PV >> or PR, but that needs a non-trivial amount of python since the >> information isn't available at parsing time (exactly like debian.bbclass). >> The second kind of change is a lot harder to detect, unless we start >> storing md5sums for kernel defconfigs. >> >> I have a lowtech proposal for this: > > The thing is if we start doing this for kernels, we'll probably end up > having to do it for other things and the end result will be a mess. > > A better fix for this is BB_STAMP_POLICY = "whitelist" which will at > least cause the right things to rebuild. > > No, this doesn't bump the PR when packages change but I'd actually like > to see some new code to fix this problem and several others once and for > all. The idea would be to "compare" the current build with any previous > ones and error if bad differences are detected. In this case if PR = X > had already built that would be an "error" which the system may work > around by using PR = "X-Z1" in the output packages. You could teach that > code interesting (even distribution specific) logic e.g. to ignore > changes in system libraries like libc. > > This would imply this comparison data would have to come from a shared > source for multiple build machines for a distribution like Angstrom. > > No, this solution isn't simple but hacking around the problem just > builds a house on top of quicksand ;-). The problems with a default OE/angstrom setup are: 1) No automatic rebuild of dependant packages - Already solved with BB_STAMP_POLICY = "whitelist" 2) No automatic way to increase PR of dependant packages 3) No distributed way to record increase of PR of dependant packages 4) No way to inject kernelversion and PR into external modules - Can be solved with code similar to debian.bbclass if someone can tackle 4) we're further along with making things saner. regards, Koen ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-04-04 13:31 [RFC] Rebuild external kernel modules on kernel change Koen Kooi ` (3 preceding siblings ...) 2009-04-05 22:43 ` Richard Purdie @ 2009-06-01 16:58 ` Tom Rini 2009-06-01 17:25 ` Koen Kooi 4 siblings, 1 reply; 21+ messages in thread From: Tom Rini @ 2009-06-01 16:58 UTC (permalink / raw) To: openembedded-devel On Sat, Apr 04, 2009 at 03:31:25PM +0200, Koen Kooi wrote: > Hi, > > For beagleboard I have a few things I need to rebuild everytime the > kernel changes: > > * powervr kerneldrivers > * sdma kernel module > * dmai kernel module > * codec-engine > > And I have roughly two kinds of kernel changes: > > 1) version upgrade (e.g. 2.6.29 -> 2.6.29) Based on the rest of the email, I assume you mean 2.6.28 -> 2.6.29 > 2) config changes (e.g. enable ethernet bridging) So, by coincidence I had this thread around still. For (1), you already have kernel-abiversion, but maybe we need some helpers to make this more useful? For (2) that's not true unless you're changing one of the variables that actually changes the running kernel abi (which isn't caught in kernel-abiversion file). This shouldn't be a frequent operation. There is, I suppose a 3rd problem which would be within version API changes, for new and in-progress features. This I'm not sure we need a complex mechanism around for. Use/publish WIP stuff, make sure users understand it's WIP and bump PRs as needed, manually. -- Tom Rini ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 16:58 ` Tom Rini @ 2009-06-01 17:25 ` Koen Kooi 2009-06-01 18:17 ` Phil Blundell ` (2 more replies) 0 siblings, 3 replies; 21+ messages in thread From: Koen Kooi @ 2009-06-01 17:25 UTC (permalink / raw) To: openembedded-devel On 01-06-09 18:58, Tom Rini wrote: > On Sat, Apr 04, 2009 at 03:31:25PM +0200, Koen Kooi wrote: > >> Hi, >> >> For beagleboard I have a few things I need to rebuild everytime the >> kernel changes: >> >> * powervr kerneldrivers >> * sdma kernel module >> * dmai kernel module >> * codec-engine >> >> And I have roughly two kinds of kernel changes: >> >> 1) version upgrade (e.g. 2.6.29 -> 2.6.29) > > Based on the rest of the email, I assume you mean 2.6.28 -> 2.6.29 right. >> 2) config changes (e.g. enable ethernet bridging) > > So, by coincidence I had this thread around still. For (1), you already > have kernel-abiversion, but maybe we need some helpers to make this more > useful? For (2) that's not true unless you're changing one of the > variables that actually changes the running kernel abi (which isn't > caught in kernel-abiversion file). This shouldn't be a frequent > operation. 2) is *very* hard to track down, e.g. people don't expect that enabling netfilter as modules breaks external wifi drivers since struct sk_buff will change (ask nokia 770 tablet users). I bet there are a lot more cases where kernel runtime abi changes, so every kernel PR bump will trigger a rebuild *and* (here's comes the most important bit) an automatic PR bump for recipes using module*.bbclass. So far I've only seen suggestions that will trigger a rebuild of recipes using module*bbclass, but no solutions for automagic version changes so that the package manager will pick it up as well. regards, Koen ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 17:25 ` Koen Kooi @ 2009-06-01 18:17 ` Phil Blundell 2009-06-01 18:45 ` Koen Kooi 2009-06-01 20:17 ` Phil Blundell 2009-06-01 20:55 ` Tom Rini 2 siblings, 1 reply; 21+ messages in thread From: Phil Blundell @ 2009-06-01 18:17 UTC (permalink / raw) To: openembedded-devel; +Cc: openembedded-devel On Mon, 2009-06-01 at 19:25 +0200, Koen Kooi wrote: > So far I've only seen suggestions that will trigger a rebuild of recipes > using module*bbclass, but no solutions for automagic version changes so > that the package manager will pick it up as well. I don't quite understand why this is so complicated; can't you just append the kernel's ABI version to either ${PR} or ${PV} (depending on whether or not you want parallel installability) for the modules? p. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 18:17 ` Phil Blundell @ 2009-06-01 18:45 ` Koen Kooi 2009-06-01 19:10 ` Phil Blundell 0 siblings, 1 reply; 21+ messages in thread From: Koen Kooi @ 2009-06-01 18:45 UTC (permalink / raw) To: openembedded-devel On 01-06-09 20:17, Phil Blundell wrote: > On Mon, 2009-06-01 at 19:25 +0200, Koen Kooi wrote: >> So far I've only seen suggestions that will trigger a rebuild of recipes >> using module*bbclass, but no solutions for automagic version changes so >> that the package manager will pick it up as well. > > I don't quite understand why this is so complicated; can't you just > append the kernel's ABI version to either ${PR} or ${PV} (depending on > whether or not you want parallel installability) for the modules? No, since PV and PR need to be known at parsing time when the kernel hasn't been built yet. regards, Koen ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 18:45 ` Koen Kooi @ 2009-06-01 19:10 ` Phil Blundell 0 siblings, 0 replies; 21+ messages in thread From: Phil Blundell @ 2009-06-01 19:10 UTC (permalink / raw) To: openembedded-devel; +Cc: openembedded-devel On Mon, 2009-06-01 at 20:45 +0200, Koen Kooi wrote: > On 01-06-09 20:17, Phil Blundell wrote: > > On Mon, 2009-06-01 at 19:25 +0200, Koen Kooi wrote: > >> So far I've only seen suggestions that will trigger a rebuild of recipes > >> using module*bbclass, but no solutions for automagic version changes so > >> that the package manager will pick it up as well. > > > > I don't quite understand why this is so complicated; can't you just > > append the kernel's ABI version to either ${PR} or ${PV} (depending on > > whether or not you want parallel installability) for the modules? > > No, since PV and PR need to be known at parsing time when the kernel > hasn't been built yet. I can't think offhand of any reason why it's important to have PR available at parse time, but if that's the case then you could splice it into DISTRO_PR (or some equivalent variable) instead. My reference to PV above was spurious; it's no different to PR in this context. If you want parallel installation then it's actually PKG_xx that you need to modify (or, conceptually, PN, but fiddling with that would almost certainly be a bad idea). p. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 17:25 ` Koen Kooi 2009-06-01 18:17 ` Phil Blundell @ 2009-06-01 20:17 ` Phil Blundell 2009-06-01 20:52 ` Koen Kooi 2009-06-01 20:55 ` Tom Rini 2 siblings, 1 reply; 21+ messages in thread From: Phil Blundell @ 2009-06-01 20:17 UTC (permalink / raw) To: openembedded-devel; +Cc: openembedded-devel On Mon, 2009-06-01 at 19:25 +0200, Koen Kooi wrote: > 2) is *very* hard to track down, e.g. people don't expect that enabling > netfilter as modules breaks external wifi drivers since struct sk_buff > will change (ask nokia 770 tablet users). > I bet there are a lot more cases where kernel runtime abi changes, so > every kernel PR bump will trigger a rebuild *and* (here's comes the most > important bit) an automatic PR bump for recipes using module*.bbclass. The networking code is indeed particularly bad in that respect. But in general Tom is right, the vast majority of kernel config changes (enabling drivers, adjusting command line parameters or default settings, that kind of thing) don't have ABI impact and there is no need to rebuild everything as a result. The easiest way to deal with that issue is probably to have a separate KERNEL_PACKAGE_ABIVERSION variable which is set locally in the kernel's own .bb file, which is appended to the regular kernel-abiversion, and which you can manually increment when making a change that causes, or is likely to cause, an ABI change. If you, as a kernel package maintainer, want to take a particularly conservative line with your own kernels then you can simply set KERNEL_PACKAGE_ABIVERSION = "${PR}" and it will act as you describe above; other maintainers who want to avoid unnecessary rebuilds can use their own judgement about when to increment it. For folks who use MODVERSIONS, it'd probably be easy enough to add an extra check to detect when the symbol hashes have changed since the last build and emit an error. In an ideal world, you'd even be able to generate a single ABI footprint by, say, taking a sha256 hash of the catenation of all the symbol versions. Then, your kernel package could Provide: kernel-abi-123456abc, all your module packages could Depend: on the same thing, and (modulo a little bit of package manager tweaking along roughly the same lines as we use for locales) everything would "just work". p. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 20:17 ` Phil Blundell @ 2009-06-01 20:52 ` Koen Kooi 2009-06-01 21:32 ` Tom Rini 2009-06-01 21:32 ` Phil Blundell 0 siblings, 2 replies; 21+ messages in thread From: Koen Kooi @ 2009-06-01 20:52 UTC (permalink / raw) To: openembedded-devel On 01-06-09 22:17, Phil Blundell wrote: Then, your kernel package could > Provide: kernel-abi-123456abc, all your module packages could Depend: on > the same thing, and (modulo a little bit of package manager tweaking > along roughly the same lines as we use for locales) everything would > "just work". only if you use {o,i}pkg, which isn't good enough. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 20:52 ` Koen Kooi @ 2009-06-01 21:32 ` Tom Rini 2009-06-01 21:32 ` Phil Blundell 1 sibling, 0 replies; 21+ messages in thread From: Tom Rini @ 2009-06-01 21:32 UTC (permalink / raw) To: openembedded-devel On Mon, Jun 01, 2009 at 10:52:04PM +0200, Koen Kooi wrote: > On 01-06-09 22:17, Phil Blundell wrote: > Then, your kernel package could >> Provide: kernel-abi-123456abc, all your module packages could Depend: on >> the same thing, and (modulo a little bit of package manager tweaking >> along roughly the same lines as we use for locales) everything would >> "just work". > > only if you use {o,i}pkg, which isn't good enough. Similar stuff should be doable for deb and rpm. -- Tom Rini ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 20:52 ` Koen Kooi 2009-06-01 21:32 ` Tom Rini @ 2009-06-01 21:32 ` Phil Blundell 1 sibling, 0 replies; 21+ messages in thread From: Phil Blundell @ 2009-06-01 21:32 UTC (permalink / raw) To: openembedded-devel; +Cc: openembedded-devel On Mon, 2009-06-01 at 22:52 +0200, Koen Kooi wrote: > On 01-06-09 22:17, Phil Blundell wrote: > > In an ideal world, you'd even be able to generate a single ABI > > footprint by, say, taking a sha256 hash of the catenation of all > > the symbol versions. Then, your kernel package could > > Provide: kernel-abi-123456abc, all your module packages could Depend: on > > the same thing, and (modulo a little bit of package manager tweaking > > along roughly the same lines as we use for locales) everything would > > "just work". > > only if you use {o,i}pkg, which isn't good enough. Indeed, in that respect this isn't an ideal world. The other things that I wrote in that same email are still applicable, though. p. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [RFC] Rebuild external kernel modules on kernel change 2009-06-01 17:25 ` Koen Kooi 2009-06-01 18:17 ` Phil Blundell 2009-06-01 20:17 ` Phil Blundell @ 2009-06-01 20:55 ` Tom Rini 2 siblings, 0 replies; 21+ messages in thread From: Tom Rini @ 2009-06-01 20:55 UTC (permalink / raw) To: openembedded-devel On Mon, Jun 01, 2009 at 07:25:11PM +0200, Koen Kooi wrote: > On 01-06-09 18:58, Tom Rini wrote: >> On Sat, Apr 04, 2009 at 03:31:25PM +0200, Koen Kooi wrote: >> >>> Hi, >>> >>> For beagleboard I have a few things I need to rebuild everytime the >>> kernel changes: >>> >>> * powervr kerneldrivers >>> * sdma kernel module >>> * dmai kernel module >>> * codec-engine >>> >>> And I have roughly two kinds of kernel changes: >>> >>> 1) version upgrade (e.g. 2.6.29 -> 2.6.29) >> >> Based on the rest of the email, I assume you mean 2.6.28 -> 2.6.29 > > right. > >>> 2) config changes (e.g. enable ethernet bridging) >> >> So, by coincidence I had this thread around still. For (1), you already >> have kernel-abiversion, but maybe we need some helpers to make this more >> useful? For (2) that's not true unless you're changing one of the >> variables that actually changes the running kernel abi (which isn't >> caught in kernel-abiversion file). This shouldn't be a frequent >> operation. > > 2) is *very* hard to track down, e.g. people don't expect that enabling > netfilter as modules breaks external wifi drivers since struct sk_buff > will change (ask nokia 770 tablet users). > I bet there are a lot more cases where kernel runtime abi changes, so > every kernel PR bump will trigger a rebuild *and* (here's comes the most > important bit) an automatic PR bump for recipes using module*.bbclass. So we need people to have a better understanding of the kernel, true. I think we've all been there at some point. IIRC, netfilter is the exception, not the rule. > So far I've only seen suggestions that will trigger a rebuild of recipes > using module*bbclass, but no solutions for automagic version changes so > that the package manager will pick it up as well. I think the problem is we're making an automatic hammer for a manual screwdriver task, in 2/3. (1) does need better handling but we have that information around already, in kernel-abiversion, we just need to export that out. -- Tom Rini ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2009-06-01 21:41 UTC | newest] Thread overview: 21+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-04-04 13:31 [RFC] Rebuild external kernel modules on kernel change Koen Kooi 2009-04-04 15:51 ` Frans Meulenbroeks 2009-04-04 16:44 ` Koen Kooi 2009-04-04 20:18 ` Frans Meulenbroeks 2009-04-04 21:14 ` Koen Kooi 2009-04-04 17:46 ` Otavio Salvador 2009-04-05 16:43 ` Koen Kooi 2009-04-05 17:07 ` Koen Kooi 2009-04-04 20:46 ` Jeremy Lainé 2009-04-05 22:43 ` Richard Purdie 2009-04-06 7:16 ` Koen Kooi 2009-06-01 16:58 ` Tom Rini 2009-06-01 17:25 ` Koen Kooi 2009-06-01 18:17 ` Phil Blundell 2009-06-01 18:45 ` Koen Kooi 2009-06-01 19:10 ` Phil Blundell 2009-06-01 20:17 ` Phil Blundell 2009-06-01 20:52 ` Koen Kooi 2009-06-01 21:32 ` Tom Rini 2009-06-01 21:32 ` Phil Blundell 2009-06-01 20:55 ` Tom Rini
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.