Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] RFC: package patching
@ 2011-11-15  8:33 Thomas De Schampheleire
  2011-11-15  8:45 ` Peter Korsgaard
  0 siblings, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-11-15  8:33 UTC (permalink / raw)
  To: buildroot

Hi,

The current automatic patching of packages based on files in
package/foo is not very nice.

01:        if test -d $($(PKG)_DIR_PREFIX)/$(RAWNAME); then \
02:          if test "$(wildcard
$($(PKG)_DIR_PREFIX)/$(RAWNAME)/$(NAMEVER)*.patch*)"; then \
03:            support/scripts/apply-patches.sh $(@D)
$($(PKG)_DIR_PREFIX)/$(RAWNAME) $(NAMEVER)\*.patch
$(NAMEVER)\*.patch.$(ARCH) || exit 1; \
04:          else \
05:            support/scripts/apply-patches.sh $(@D)
$($(PKG)_DIR_PREFIX)/$(RAWNAME) $(RAWNAME)\*.patch
$(RAWNAME)\*.patch.$(ARCH) || exit 1; \
06:            if test -d $($(PKG)_DIR_PREFIX)/$(RAWNAME)/$(NAMEVER); then \
07:              support/scripts/apply-patches.sh $(@D)
$($(PKG)_DIR_PREFIX)/$(RAWNAME)/$(NAMEVER) \*.patch \*.patch.$(ARCH)
|| exit 1; \
08:            fi; \
09:          fi; \
10:        fi; \
11:        )

The current principles are as follows (assume we are patching package
foo with version 1.2.3):

* if there are patches package/foo/foo-1.2.3-something.patch, apply
those, and stop.
* if there are no such files, apply any patch of the form
package/foo/foo-something.patch, and
* check whether there is a directory package/foo/foo-1.2.3, and apply
all patches therein.

Here are some problems with this approach:

1. package/foo/foo-1.2.3.patch cannot co-exist with any other type of
patch, like package/foo/foo-common-change.patch or
package/foo/foo-1.2.3/something.patch. Whenever the first type exists,
none of the others will be applied.

2. it is not possible to hold a patch for a previous version while
there is no patch at all for the current version, unless you move it
to a foo-xyz subdirectory. If you have
package/foo/foo-1.2.2-backport.patch while the current version is
1.2.3 but has no specific patches, then the if on line 02 above will
be false, and line 05 will apply any patch that starts with 'foo' and
ends with '.patch', including foo-1.2.2-backport.patch, which it
shouldn't.


I think we should try and improve this.

A first approach could be to simply remove support for
foo-1.2.3-something.patch, and require version-specific patches to be
in a versioned subdirectory, package/foo/foo-1.2.3. For this, we
should remove line 02-04. In this case, common patches are applied
first, followed by versioned patches in subdirectories.

Alternatively, we can continue to support foo-1.2.3.patch, but we'll
have to make sure that this form is not matched by line 05, to fix
problem 2. Additionally, the mutual exclusivity of line 03 versus
lines 05-07 should be removed.

Comments are welcome.

Best regards,
Thomas

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-15  8:33 [Buildroot] RFC: package patching Thomas De Schampheleire
@ 2011-11-15  8:45 ` Peter Korsgaard
  2011-11-15 19:14   ` Arnout Vandecappelle
  0 siblings, 1 reply; 21+ messages in thread
From: Peter Korsgaard @ 2011-11-15  8:45 UTC (permalink / raw)
  To: buildroot

>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> writes:

Hi,

 Thomas> I think we should try and improve this.

 Thomas> A first approach could be to simply remove support for
 Thomas> foo-1.2.3-something.patch, and require version-specific patches to be
 Thomas> in a versioned subdirectory, package/foo/foo-1.2.3. For this, we
 Thomas> should remove line 02-04. In this case, common patches are applied
 Thomas> first, followed by versioned patches in subdirectories.

Yes, I know - We hit this with E.G. busybox in the past. I would go for
the versioned-patches-in-subdir, as we (luckily) only support multiple
versions for a limited number of packages.

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-15  8:45 ` Peter Korsgaard
@ 2011-11-15 19:14   ` Arnout Vandecappelle
  2011-11-15 21:28     ` Thomas Petazzoni
  0 siblings, 1 reply; 21+ messages in thread
From: Arnout Vandecappelle @ 2011-11-15 19:14 UTC (permalink / raw)
  To: buildroot

On Tuesday 15 November 2011 08:45:11 Peter Korsgaard wrote:
> >>>>> "Thomas" == Thomas De Schampheleire 
<patrickdepinguin+buildroot@gmail.com> writes:
> Hi,
> 
>  Thomas> I think we should try and improve this.
> 
>  Thomas> A first approach could be to simply remove support for
>  Thomas> foo-1.2.3-something.patch, and require version-specific patches to
> be Thomas> in a versioned subdirectory, package/foo/foo-1.2.3. For this,
> we Thomas> should remove line 02-04. In this case, common patches are
> applied Thomas> first, followed by versioned patches in subdirectories.
> 
> Yes, I know - We hit this with E.G. busybox in the past. I would go for
> the versioned-patches-in-subdir, as we (luckily) only support multiple
> versions for a limited number of packages.

 While we're at it, I would also make it policy to not include the version 
number in the patch, except for packages with multiple active versions.  Now, 
when you're upgrading a package, you also have to do a lot of renames of 
patches.

 Note that this fix will require renames of rougly 300 files... 

 Regards,
 Arnout


-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-15 19:14   ` Arnout Vandecappelle
@ 2011-11-15 21:28     ` Thomas Petazzoni
  2011-11-16  6:18       ` Sergey Naumov
  2011-11-16  6:44       ` Thomas De Schampheleire
  0 siblings, 2 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-11-15 21:28 UTC (permalink / raw)
  To: buildroot

Le Tue, 15 Nov 2011 19:14:38 +0000,
Arnout Vandecappelle <arnout@mind.be> a ?crit :

> > Yes, I know - We hit this with E.G. busybox in the past. I would go
> > for the versioned-patches-in-subdir, as we (luckily) only support
> > multiple versions for a limited number of packages.
> 
>  While we're at it, I would also make it policy to not include the
> version number in the patch, except for packages with multiple active
> versions.  Now, when you're upgrading a package, you also have to do
> a lot of renames of patches.

I agree with both Thomas and Arnout. Thomas is right on the fact that
we should clarify the patching logic in the package infrastructure, but
this clarification should also come together with a clarification of
the best practices for storing patches for packages.

As Arnout suggests, I think that most packages should just have patches
whose filename do not contain the version. Only packages that support
multiple versions would have subdirectories.

At the same time, we should also probably generalize the usage of patch
numbering, in order to have a clear order of the patches. I.e,
something such as:

	foobar-0001-something.patch
	foobar-0002-somethingelse.patch

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-15 21:28     ` Thomas Petazzoni
@ 2011-11-16  6:18       ` Sergey Naumov
  2011-11-16  6:50         ` Thomas De Schampheleire
  2011-11-16  6:44       ` Thomas De Schampheleire
  1 sibling, 1 reply; 21+ messages in thread
From: Sergey Naumov @ 2011-11-16  6:18 UTC (permalink / raw)
  To: buildroot

Maybe it is worth to introduce a variable where patches to apply are
mentioned (like in openembedded + bitbake)?
PKGNAME_PATCHES = patch1 patch2 ...  - apply these patches
PKGNAME_PATCHES =                           - do not apply patches
<variable is not specified>                          - default
buildroot behavior

somewhere in GENTARGETS/AUTOTARGETS:
PKGNAME_PATCHES ?= (shell find ...)

but bitbake search pathes in different directories (current, directory
with pkg name, dir with pkgname-version), so this is only a draft.

Sergey Naumov.

2011/11/16, Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
> Le Tue, 15 Nov 2011 19:14:38 +0000,
> Arnout Vandecappelle <arnout@mind.be> a ?crit :
>
>> > Yes, I know - We hit this with E.G. busybox in the past. I would go
>> > for the versioned-patches-in-subdir, as we (luckily) only support
>> > multiple versions for a limited number of packages.
>>
>>  While we're at it, I would also make it policy to not include the
>> version number in the patch, except for packages with multiple active
>> versions.  Now, when you're upgrading a package, you also have to do
>> a lot of renames of patches.
>
> I agree with both Thomas and Arnout. Thomas is right on the fact that
> we should clarify the patching logic in the package infrastructure, but
> this clarification should also come together with a clarification of
> the best practices for storing patches for packages.
>
> As Arnout suggests, I think that most packages should just have patches
> whose filename do not contain the version. Only packages that support
> multiple versions would have subdirectories.
>
> At the same time, we should also probably generalize the usage of patch
> numbering, in order to have a clear order of the patches. I.e,
> something such as:
>
> 	foobar-0001-something.patch
> 	foobar-0002-somethingelse.patch
>
> Regards,
>
> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-15 21:28     ` Thomas Petazzoni
  2011-11-16  6:18       ` Sergey Naumov
@ 2011-11-16  6:44       ` Thomas De Schampheleire
  2011-11-16 18:03         ` Thomas Petazzoni
  1 sibling, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-11-16  6:44 UTC (permalink / raw)
  To: buildroot

On Tue, Nov 15, 2011 at 10:28 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Tue, 15 Nov 2011 19:14:38 +0000,
> Arnout Vandecappelle <arnout@mind.be> a ?crit :
>
>> > Yes, I know - We hit this with E.G. busybox in the past. I would go
>> > for the versioned-patches-in-subdir, as we (luckily) only support
>> > multiple versions for a limited number of packages.
>>
>> ?While we're at it, I would also make it policy to not include the
>> version number in the patch, except for packages with multiple active
>> versions. ?Now, when you're upgrading a package, you also have to do
>> a lot of renames of patches.
>
> I agree with both Thomas and Arnout. Thomas is right on the fact that
> we should clarify the patching logic in the package infrastructure, but
> this clarification should also come together with a clarification of
> the best practices for storing patches for packages.
>
> As Arnout suggests, I think that most packages should just have patches
> whose filename do not contain the version. Only packages that support
> multiple versions would have subdirectories.

I'm not sure if this is so black-and-white. If a patch backports a
change from 1.2.3 to 1.2.2, then IMO the patch should really be named
1.2.2, even if only 1.2.2 is present in Buildroot. If it is named
without version, then it will also be taken along when bumping the
package (and possibly but not necessarily cause a patch conflict).
Only in cases were the change made by the patch is generic and not
intended for a specific version, should we remove the version from the
patch name.

>
> At the same time, we should also probably generalize the usage of patch
> numbering, in order to have a clear order of the patches. I.e,
> something such as:
>
> ? ? ? ?foobar-0001-something.patch
> ? ? ? ?foobar-0002-somethingelse.patch
>

Ok with me.

Best regards,
Thomas

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-16  6:18       ` Sergey Naumov
@ 2011-11-16  6:50         ` Thomas De Schampheleire
  0 siblings, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-11-16  6:50 UTC (permalink / raw)
  To: buildroot

Hi Sergey,

Please don't top-post, see
http://linux.sgms-centre.com/misc/netiquette.php#toppost

On Wed, Nov 16, 2011 at 7:18 AM, Sergey Naumov <sknaumov@gmail.com> wrote:
>
> 2011/11/16, Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
>> Le Tue, 15 Nov 2011 19:14:38 +0000,
>> Arnout Vandecappelle <arnout@mind.be> a ?crit :
>>
>>> > Yes, I know - We hit this with E.G. busybox in the past. I would go
>>> > for the versioned-patches-in-subdir, as we (luckily) only support
>>> > multiple versions for a limited number of packages.
>>>
>>> ?While we're at it, I would also make it policy to not include the
>>> version number in the patch, except for packages with multiple active
>>> versions. ?Now, when you're upgrading a package, you also have to do
>>> a lot of renames of patches.
>>
>> I agree with both Thomas and Arnout. Thomas is right on the fact that
>> we should clarify the patching logic in the package infrastructure, but
>> this clarification should also come together with a clarification of
>> the best practices for storing patches for packages.
>>
>> As Arnout suggests, I think that most packages should just have patches
>> whose filename do not contain the version. Only packages that support
>> multiple versions would have subdirectories.
>>
>> At the same time, we should also probably generalize the usage of patch
>> numbering, in order to have a clear order of the patches. I.e,
>> something such as:
>>
>> ? ? ? foobar-0001-something.patch
>> ? ? ? foobar-0002-somethingelse.patch
>
> Maybe it is worth to introduce a variable where patches to apply are
> mentioned (like in openembedded + bitbake)?
> PKGNAME_PATCHES = patch1 patch2 ...  - apply these patches
> PKGNAME_PATCHES =                           - do not apply patches
> <variable is not specified>                          - default
> buildroot behavior
>
> somewhere in GENTARGETS/AUTOTARGETS:
> PKGNAME_PATCHES ?= (shell find ...)
>
> but bitbake search pathes in different directories (current, directory
> with pkg name, dir with pkgname-version), so this is only a draft.

There already is a PKGNAME_PATCH that can hold an explicit list of
patches to be applied. These patches will be applied before the
patches found in package/foo. I did not copy that snippet in my
original mail though.

Best regards,
Thomas

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-16  6:44       ` Thomas De Schampheleire
@ 2011-11-16 18:03         ` Thomas Petazzoni
  2011-11-17 13:05           ` Thomas De Schampheleire
  0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2011-11-16 18:03 UTC (permalink / raw)
  To: buildroot

Le Wed, 16 Nov 2011 07:44:54 +0100,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> > As Arnout suggests, I think that most packages should just have
> > patches whose filename do not contain the version. Only packages
> > that support multiple versions would have subdirectories.
> 
> I'm not sure if this is so black-and-white. If a patch backports a
> change from 1.2.3 to 1.2.2, then IMO the patch should really be named
> 1.2.2, even if only 1.2.2 is present in Buildroot. If it is named
> without version, then it will also be taken along when bumping the
> package (and possibly but not necessarily cause a patch conflict).
> Only in cases were the change made by the patch is generic and not
> intended for a specific version, should we remove the version from the
> patch name.

A generic patch doesn't exist, a patch is *always* for a specific
version of a source tree. The fact that it might apply on multiple
versions of a given package is just pure luck. Therefore, I don't see
where the distinction between "generic patch" and a "version-specific"
patch is.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-16 18:03         ` Thomas Petazzoni
@ 2011-11-17 13:05           ` Thomas De Schampheleire
  2011-11-17 21:23             ` Arnout Vandecappelle
  2011-11-18 16:41             ` Thomas Petazzoni
  0 siblings, 2 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-11-17 13:05 UTC (permalink / raw)
  To: buildroot

On Wed, Nov 16, 2011 at 7:03 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 16 Nov 2011 07:44:54 +0100,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> > As Arnout suggests, I think that most packages should just have
>> > patches whose filename do not contain the version. Only packages
>> > that support multiple versions would have subdirectories.
>>
>> I'm not sure if this is so black-and-white. If a patch backports a
>> change from 1.2.3 to 1.2.2, then IMO the patch should really be named
>> 1.2.2, even if only 1.2.2 is present in Buildroot. If it is named
>> without version, then it will also be taken along when bumping the
>> package (and possibly but not necessarily cause a patch conflict).
>> Only in cases were the change made by the patch is generic and not
>> intended for a specific version, should we remove the version from the
>> patch name.
>
> A generic patch doesn't exist, a patch is *always* for a specific
> version of a source tree. The fact that it might apply on multiple
> versions of a given package is just pure luck. Therefore, I don't see
> where the distinction between "generic patch" and a "version-specific"
> patch is.

How does a version-bumper currently go about? How is he supposed to
know whether a patch is still valid or not?

Although you are right that there is no such thing as a generic patch,
I still feel there is a conceptual difference between:
* modifications intended for all versions of a package, for example
modifications that we know will never go upstream because they are
buildroot-specific, and
* modifications which are intended for a specific version, for example
a backport of a change that was already fixed in a later version.

To remove our reliance on 'luck', I agree it may be best to add
version numbers to either type of patch. If a package has multiple
versions, the patch may simply be duplicated with separate version
numbers.

But, should we mark the difference between such patches in a different way?

Best regards,
Thomas

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-17 13:05           ` Thomas De Schampheleire
@ 2011-11-17 21:23             ` Arnout Vandecappelle
  2011-11-17 21:42               ` Thomas Petazzoni
  2011-11-18 16:41             ` Thomas Petazzoni
  1 sibling, 1 reply; 21+ messages in thread
From: Arnout Vandecappelle @ 2011-11-17 21:23 UTC (permalink / raw)
  To: buildroot

On Thursday 17 November 2011 13:05:47 Thomas De Schampheleire wrote:
> How does a version-bumper currently go about? How is he supposed to
> know whether a patch is still valid or not?

 I've never bumped a version so I can't be sure, but I expect the procedure
is like this:

1. Bump version number
2. Rename all patches to the new version number
3. make pkg-patch and conflicts or fuzz
4. Remove or modify patches as needed
5. Build and test


> Although you are right that there is no such thing as a generic patch,
> I still feel there is a conceptual difference between:
> * modifications intended for all versions of a package, for example
> modifications that we know will never go upstream because they are
> buildroot-specific, and

 Really buildroot-specific patches shouldn't exist.  It means that either
the package or buildroot is broken (usually the former).  It's more likely
that a patch never goes upstream because upstream just doesn't care
about cross-compilation, or because no buildroot developer pushes it.

> * modifications which are intended for a specific version, for example
> a backport of a change that was already fixed in a later version.

 Only in this case I think it makes sense to include a version number in
the patch.  But what is actually important in that case is the version
number from which it was backported, not the version number to which
it applies...

 With that in mind, I would propose a format like this for backport
patches:

<pkg>-<seqno>-from_<nextversion>-description-of-the-patch

> To remove our reliance on 'luck', I agree it may be best to add
> version numbers to either type of patch. If a package has multiple
> versions, the patch may simply be duplicated with separate version
> numbers .

 I think ThomasP meant that it is best to _remove_ the version numbers.
A version bumper will most likely try to take along all patches anyway, so
putting a version number is just increasing the diffstat.  Without version
numbers, the diffstat will show much better which patches could be
removed, which ones were added and which ones needed to be modified.

 Regards,
 Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-17 21:23             ` Arnout Vandecappelle
@ 2011-11-17 21:42               ` Thomas Petazzoni
  2011-11-18  6:53                 ` Thomas De Schampheleire
  2011-11-18 19:27                 ` Arnout Vandecappelle
  0 siblings, 2 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-11-17 21:42 UTC (permalink / raw)
  To: buildroot

Le Thu, 17 Nov 2011 21:23:05 +0000,
Arnout Vandecappelle <arnout@mind.be> a ?crit :

>  With that in mind, I would propose a format like this for backport
> patches:
> 
> <pkg>-<seqno>-from_<nextversion>-description-of-the-patch

Having a comment inside a patch seems to be enough. Sometimes when the
patch is pushed upstream, it's merged in the Git repo of the upstream
project, but there isn't yet a release with the modification, so it
would be hard to known which "nextversion" the patch will be in.

We already have comments in patches, those comments can carry the
upstream status of the patch, which is also compatible with what we
intend to do with the send-patches.org project.

>  I think ThomasP meant that it is best to _remove_ the version numbers.
> A version bumper will most likely try to take along all patches anyway, so
> putting a version number is just increasing the diffstat.  Without version
> numbers, the diffstat will show much better which patches could be
> removed, which ones were added and which ones needed to be modified.

Agreed.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-17 21:42               ` Thomas Petazzoni
@ 2011-11-18  6:53                 ` Thomas De Schampheleire
  2011-11-18  7:05                   ` Thomas Petazzoni
                                     ` (2 more replies)
  2011-11-18 19:27                 ` Arnout Vandecappelle
  1 sibling, 3 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-11-18  6:53 UTC (permalink / raw)
  To: buildroot

On Thu, Nov 17, 2011 at 10:42 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Thu, 17 Nov 2011 21:23:05 +0000,
> Arnout Vandecappelle <arnout@mind.be> a ?crit :
>
>> ?With that in mind, I would propose a format like this for backport
>> patches:
>>
>> <pkg>-<seqno>-from_<nextversion>-description-of-the-patch
>
> Having a comment inside a patch seems to be enough. Sometimes when the
> patch is pushed upstream, it's merged in the Git repo of the upstream
> project, but there isn't yet a release with the modification, so it
> would be hard to known which "nextversion" the patch will be in.
>
> We already have comments in patches, those comments can carry the
> upstream status of the patch, which is also compatible with what we
> intend to do with the send-patches.org project.
>
>> ?I think ThomasP meant that it is best to _remove_ the version numbers.
>> A version bumper will most likely try to take along all patches anyway, so
>> putting a version number is just increasing the diffstat. ?Without version
>> numbers, the diffstat will show much better which patches could be
>> removed, which ones were added and which ones needed to be modified.
>
> Agreed.

Ok, so an attempt to summarize the discussion so far:

- most patches should live in package/foo and have a filename of the form:
<pkg>-<seqnum>-<description>.patch

- for packages that have multiple versions at once in buildroot,
patches go into package/foo/foo-version, but have the same filename:
<pkg>-<seqnum>-<description>.patch

- support for <pkg>-<version>-<description>.patch is removed, and all
existing such patches are renamed/moved according to the rules above.


Some remaining questions:
* what if a package has multiple versions, and a certain patch applies
to both versions. Should there be one copy of the patch in
package/foo, or should the patch be duplicated in
package/foo/foo-version1 and package/foo/foo-version2 ?

* how many digits should the sequence number have? I now that
git-format-patch uses 4 digits (0001) but really isn't necessary for
buildroot since the number of patches we'll have for each package is
limited. A package with 99 patches would already be extraneous, so I'd
say 01 (2 digits) is enough.
This may seem like a detail, but discussing this should keep things
consistent throughout the future.

Thanks,
Thomas

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-18  6:53                 ` Thomas De Schampheleire
@ 2011-11-18  7:05                   ` Thomas Petazzoni
  2011-11-18  7:34                     ` Peter Korsgaard
  2011-11-18 12:24                   ` Michael S. Zick
  2011-11-18 19:44                   ` Arnout Vandecappelle
  2 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2011-11-18  7:05 UTC (permalink / raw)
  To: buildroot

Le Fri, 18 Nov 2011 07:53:22 +0100,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> Ok, so an attempt to summarize the discussion so far:

Good idea.

> - most patches should live in package/foo and have a filename of the form:
> <pkg>-<seqnum>-<description>.patch

Agreed.

> - for packages that have multiple versions at once in buildroot,
> patches go into package/foo/foo-version, but have the same filename:
> <pkg>-<seqnum>-<description>.patch

I would name the directory package/<foo>/<version> instead of
package/<foo>/<foo>-<version>, because repeating <foo> is useless.

> - support for <pkg>-<version>-<description>.patch is removed, and all
> existing such patches are renamed/moved according to the rules above.

Agreed.

I also would like to see removed:

 * Support for *.patch.$(ARCH). But that requires some work to get rid
   of the current 4 arch-specific patches that we have for liboil, fbv,
   libmad and jamvm.

 * Support for host-*.patch (which are applied only to the host
   variant). We have only one such patch in the tree (for libgtk2) and
   with a bit of effort, we could make it generic enough so that it
   works on both host and target.

> Some remaining questions:
> * what if a package has multiple versions, and a certain patch applies
> to both versions. Should there be one copy of the patch in
> package/foo, or should the patch be duplicated in
> package/foo/foo-version1 and package/foo/foo-version2 ?

Duplicated.

> * how many digits should the sequence number have? I now that
> git-format-patch uses 4 digits (0001) but really isn't necessary for
> buildroot since the number of patches we'll have for each package is
> limited. A package with 99 patches would already be extraneous, so I'd
> say 01 (2 digits) is enough.
> This may seem like a detail, but discussing this should keep things
> consistent throughout the future.

Agreed that 2 digits is enough.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-18  7:05                   ` Thomas Petazzoni
@ 2011-11-18  7:34                     ` Peter Korsgaard
  2011-11-18  9:26                       ` Thomas De Schampheleire
  0 siblings, 1 reply; 21+ messages in thread
From: Peter Korsgaard @ 2011-11-18  7:34 UTC (permalink / raw)
  To: buildroot

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

Hi,

 >> - for packages that have multiple versions at once in buildroot,
 >> patches go into package/foo/foo-version, but have the same filename:
 >> <pkg>-<seqnum>-<description>.patch

 Thomas> I would name the directory package/<foo>/<version> instead of
 Thomas> package/<foo>/<foo>-<version>, because repeating <foo> is useless.

Agreed, but that might break existing users (but the number of people
having a version-specific subdir with custom patches is probably quite
small, so it might be ok).

 Thomas> I also would like to see removed:

 Thomas>  * Support for *.patch.$(ARCH). But that requires some work to get rid
 Thomas>    of the current 4 arch-specific patches that we have for liboil, fbv,
 Thomas>    libmad and jamvm.

 Thomas>  * Support for host-*.patch (which are applied only to the host
 Thomas>    variant). We have only one such patch in the tree (for libgtk2) and
 Thomas>    with a bit of effort, we could make it generic enough so that it
 Thomas>    works on both host and target.

Agreed.

 >> Some remaining questions:
 >> * what if a package has multiple versions, and a certain patch applies
 >> to both versions. Should there be one copy of the patch in
 >> package/foo, or should the patch be duplicated in
 >> package/foo/foo-version1 and package/foo/foo-version2 ?

 Thomas> Duplicated.

Agreed.

 >> * how many digits should the sequence number have? I now that
 >> git-format-patch uses 4 digits (0001) but really isn't necessary for
 >> buildroot since the number of patches we'll have for each package is
 >> limited. A package with 99 patches would already be extraneous, so I'd
 >> say 01 (2 digits) is enough.
 >> This may seem like a detail, but discussing this should keep things
 >> consistent throughout the future.

 Thomas> Agreed that 2 digits is enough.

Me too.

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-18  7:34                     ` Peter Korsgaard
@ 2011-11-18  9:26                       ` Thomas De Schampheleire
  0 siblings, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-11-18  9:26 UTC (permalink / raw)
  To: buildroot

On Fri, Nov 18, 2011 at 8:34 AM, Peter Korsgaard <jacmet@uclibc.org> wrote:
>>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:
>
> Hi,
>
> ?>> - for packages that have multiple versions at once in buildroot,
> ?>> patches go into package/foo/foo-version, but have the same filename:
> ?>> <pkg>-<seqnum>-<description>.patch
>
> ?Thomas> I would name the directory package/<foo>/<version> instead of
> ?Thomas> package/<foo>/<foo>-<version>, because repeating <foo> is useless.
>
> Agreed, but that might break existing users (but the number of people
> having a version-specific subdir with custom patches is probably quite
> small, so it might be ok).

Do we normally have some kind of migration information with each
buildroot release?
If not, this may be a good idea. Changes to the core infrastructure
that requires changes to user-specific packages can be detailed there.
Examples are the patch directory mentioned above, the reduction of the
number of arguments to GENTARGETS, previously the rename of linux26-X
targets to linux-X, etc.

>
> ?Thomas> I also would like to see removed:
>
> ?Thomas> ?* Support for *.patch.$(ARCH). But that requires some work to get rid
> ?Thomas> ? ?of the current 4 arch-specific patches that we have for liboil, fbv,
> ?Thomas> ? ?libmad and jamvm.

liboil has been removed recently, so it leaves us with:
package/java/jamvm/jamvm-1.5.0.patch.avr32
package/multimedia/libmad/libmad-0.15.1b-optimization.patch.avr32
package/fbv/fbv-1.0b-arch.patch.avr32

A quick initial solution could be to add these patches to the
<pkg>_PATCH variable directly. This allows us to remove support for
patch.$(ARCH) from the core immediately, awaiting the final fix to
solve these avr32 patches.

>
> ?Thomas> ?* Support for host-*.patch (which are applied only to the host
> ?Thomas> ? ?variant). We have only one such patch in the tree (for libgtk2) and
> ?Thomas> ? ?with a bit of effort, we could make it generic enough so that it
> ?Thomas> ? ?works on both host and target.
>
> Agreed.

The host-*.patch is not supported in the core infrastructure anyway.
It is only present in the libgtk2.mk file itself. So this can be a
separate effort.

>
> ?>> Some remaining questions:
> ?>> * what if a package has multiple versions, and a certain patch applies
> ?>> to both versions. Should there be one copy of the patch in
> ?>> package/foo, or should the patch be duplicated in
> ?>> package/foo/foo-version1 and package/foo/foo-version2 ?
>
> ?Thomas> Duplicated.
>
> Agreed.
>
> ?>> * how many digits should the sequence number have? I now that
> ?>> git-format-patch uses 4 digits (0001) but really isn't necessary for
> ?>> buildroot since the number of patches we'll have for each package is
> ?>> limited. A package with 99 patches would already be extraneous, so I'd
> ?>> say 01 (2 digits) is enough.
> ?>> This may seem like a detail, but discussing this should keep things
> ?>> consistent throughout the future.
>
> ?Thomas> Agreed that 2 digits is enough.
>
> Me too.

Great! Sounds like we have an (almost complete) plan!


>
> --
> Bye, Peter Korsgaard
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-18  6:53                 ` Thomas De Schampheleire
  2011-11-18  7:05                   ` Thomas Petazzoni
@ 2011-11-18 12:24                   ` Michael S. Zick
  2011-11-18 19:44                   ` Arnout Vandecappelle
  2 siblings, 0 replies; 21+ messages in thread
From: Michael S. Zick @ 2011-11-18 12:24 UTC (permalink / raw)
  To: buildroot

On Fri November 18 2011, Thomas De Schampheleire wrote:
> On Thu, Nov 17, 2011 at 10:42 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Le Thu, 17 Nov 2011 21:23:05 +0000,
> > Arnout Vandecappelle <arnout@mind.be> a ?crit :
> >
> >> ?With that in mind, I would propose a format like this for backport
> >> patches:
> >>
> >> <pkg>-<seqno>-from_<nextversion>-description-of-the-patch
> >
> > Having a comment inside a patch seems to be enough. Sometimes when the
> > patch is pushed upstream, it's merged in the Git repo of the upstream
> > project, but there isn't yet a release with the modification, so it
> > would be hard to known which "nextversion" the patch will be in.
> >
> > We already have comments in patches, those comments can carry the
> > upstream status of the patch, which is also compatible with what we
> > intend to do with the send-patches.org project.
> >
> >> ?I think ThomasP meant that it is best to _remove_ the version numbers.
> >> A version bumper will most likely try to take along all patches anyway, so
> >> putting a version number is just increasing the diffstat. ?Without version
> >> numbers, the diffstat will show much better which patches could be
> >> removed, which ones were added and which ones needed to be modified.
> >
> > Agreed.
> 
> Ok, so an attempt to summarize the discussion so far:
> 
> - most patches should live in package/foo and have a filename of the form:
> <pkg>-<seqnum>-<description>.patch
> 
> - for packages that have multiple versions at once in buildroot,
> patches go into package/foo/foo-version, but have the same filename:
> <pkg>-<seqnum>-<description>.patch
> 
> - support for <pkg>-<version>-<description>.patch is removed, and all
> existing such patches are renamed/moved according to the rules above.
> 
> 
> Some remaining questions:
> * what if a package has multiple versions, and a certain patch applies
> to both versions. Should there be one copy of the patch in
> package/foo, or should the patch be duplicated in
> package/foo/foo-version1 and package/foo/foo-version2 ?
>

Patch should be duplicated, because name will now contain the
sequence number and although the patch contents would be duplicated,
the order of application might not be.

Mike 
> * how many digits should the sequence number have? I now that
> git-format-patch uses 4 digits (0001) but really isn't necessary for
> buildroot since the number of patches we'll have for each package is
> limited. A package with 99 patches would already be extraneous, so I'd
> say 01 (2 digits) is enough.
> This may seem like a detail, but discussing this should keep things
> consistent throughout the future.
> 
> Thanks,
> Thomas
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-17 13:05           ` Thomas De Schampheleire
  2011-11-17 21:23             ` Arnout Vandecappelle
@ 2011-11-18 16:41             ` Thomas Petazzoni
  1 sibling, 0 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-11-18 16:41 UTC (permalink / raw)
  To: buildroot

Le Thu, 17 Nov 2011 14:05:47 +0100,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> Although you are right that there is no such thing as a generic patch,
> I still feel there is a conceptual difference between:
> * modifications intended for all versions of a package, for example
> modifications that we know will never go upstream because they are
> buildroot-specific, and
> * modifications which are intended for a specific version, for example
> a backport of a change that was already fixed in a later version.

I get your point.

> To remove our reliance on 'luck', I agree it may be best to add
> version numbers to either type of patch. If a package has multiple
> versions, the patch may simply be duplicated with separate version
> numbers.
> 
> But, should we mark the difference between such patches in a
> different way?

Yes, through comments in the respective patch descriptions.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-17 21:42               ` Thomas Petazzoni
  2011-11-18  6:53                 ` Thomas De Schampheleire
@ 2011-11-18 19:27                 ` Arnout Vandecappelle
  2011-11-19  9:26                   ` Bjørn Forsman
  1 sibling, 1 reply; 21+ messages in thread
From: Arnout Vandecappelle @ 2011-11-18 19:27 UTC (permalink / raw)
  To: buildroot

On Thursday 17 November 2011 21:42:11 Thomas Petazzoni wrote:
> Having a comment inside a patch seems to be enough. Sometimes when the
> patch is pushed upstream, it's merged in the Git repo of the upstream
> project, but there isn't yet a release with the modification, so it
> would be hard to known which "nextversion" the patch will be in.

 Most projects have a single trunk, so you can predict that a patch in trunk will appear in the next release.  Even if a project has a more complex release scheme, it will typically have branches.  The only issue could be that you don't know if it will be 1.9 or 2.0, but then tagging it as 1.9 is a safe bet.

 I find a release number tag much more useful than e.g. a commit ID.  With the commit ID, it is still pretty difficult to find out if a partical release contains it.

 Oh, and patches which have not (yet) been accepted upstream should not be tagged.

 Anyway, as a version bumper, I would use that tag just as a hint.

> We already have comments in patches, those comments can carry the
> upstream status of the patch, which is also compatible with what we
> intend to do with the send-patches.org project.

 Is there going to be a tag for it, similar to Reviewed-by?

 Regards,
 Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-18  6:53                 ` Thomas De Schampheleire
  2011-11-18  7:05                   ` Thomas Petazzoni
  2011-11-18 12:24                   ` Michael S. Zick
@ 2011-11-18 19:44                   ` Arnout Vandecappelle
  2 siblings, 0 replies; 21+ messages in thread
From: Arnout Vandecappelle @ 2011-11-18 19:44 UTC (permalink / raw)
  To: buildroot

On Friday 18 November 2011 06:53:22 Thomas De Schampheleire wrote:
> - for packages that have multiple versions at once in buildroot,
> patches go into package/foo/foo-version, but have the same filename:
> <pkg>-<seqnum>-<description>.patch
[snip]
> * how many digits should the sequence number have? I now that
> git-format-patch uses 4 digits (0001) but really isn't necessary for
> buildroot since the number of patches we'll have for each package is
> limited. A package with 99 patches would already be extraneous, so I'd
> say 01 (2 digits) is enough.

 After going through ThomasP's ELC presentation, I've started using the approach of a <PKG>_OVERRIDE_SRCDIR which contains a git clone to generate the patches.  Works out great, much more convenient than my previous approach of copying the source tree to .orig and diffing.

 Except for one little detail: the patch naming.  git format-patch doesn't give much leeway in how the patches are named, so you end up needing to do a rename of the resulting patch set.

 For prepending a <pkg>- prefix, it is still doable with a simple
for i in 00*patch; do mv $i linux-$i; done
But with only two digits, you almost need a shell script (or mmv or mrename) to do it...

 Bottom line: I would prefer to see a git format-patch compatible filename:
<pkg>/0000-<description>.patch
<pkg>/<version>/0000-<description>.patch

 The extra zeroes are not really in the way IMO, and the package name is really redundant anyway.

 Note that people who prefer to name their patches in any other way can still do it.  The only thing you can't do is to mix patches of two different packages (or two different versions) in the same directory.


 Regards,
 Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-18 19:27                 ` Arnout Vandecappelle
@ 2011-11-19  9:26                   ` Bjørn Forsman
  2011-11-19 12:13                     ` Arnout Vandecappelle
  0 siblings, 1 reply; 21+ messages in thread
From: Bjørn Forsman @ 2011-11-19  9:26 UTC (permalink / raw)
  To: buildroot

On 18 November 2011 20:27, Arnout Vandecappelle <arnout@mind.be> wrote:
> ?With the commit ID, it is still pretty difficult to find out if a partical release contains it.

git describe --contains <commit-id>

I just learned it! :-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Buildroot] RFC: package patching
  2011-11-19  9:26                   ` Bjørn Forsman
@ 2011-11-19 12:13                     ` Arnout Vandecappelle
  0 siblings, 0 replies; 21+ messages in thread
From: Arnout Vandecappelle @ 2011-11-19 12:13 UTC (permalink / raw)
  To: buildroot

On Saturday 19 November 2011 10:26:17 Bj?rn Forsman wrote:
> On 18 November 2011 20:27, Arnout Vandecappelle <arnout@mind.be> wrote:
> >  With the commit ID, it is still pretty difficult to find out if a partical release contains it.
                                                                       ^^^^^^^^
                                                           I meant "particular"
> 
> git describe --contains <commit-id>
> 
> I just learned it! :-)

 Yes, but then you have to clone the repository, while normally when you're
bumping a version you just download the release tar.

 But thanks for the tip.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20111119/031e6465/attachment.html>

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2011-11-19 12:13 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-11-15  8:33 [Buildroot] RFC: package patching Thomas De Schampheleire
2011-11-15  8:45 ` Peter Korsgaard
2011-11-15 19:14   ` Arnout Vandecappelle
2011-11-15 21:28     ` Thomas Petazzoni
2011-11-16  6:18       ` Sergey Naumov
2011-11-16  6:50         ` Thomas De Schampheleire
2011-11-16  6:44       ` Thomas De Schampheleire
2011-11-16 18:03         ` Thomas Petazzoni
2011-11-17 13:05           ` Thomas De Schampheleire
2011-11-17 21:23             ` Arnout Vandecappelle
2011-11-17 21:42               ` Thomas Petazzoni
2011-11-18  6:53                 ` Thomas De Schampheleire
2011-11-18  7:05                   ` Thomas Petazzoni
2011-11-18  7:34                     ` Peter Korsgaard
2011-11-18  9:26                       ` Thomas De Schampheleire
2011-11-18 12:24                   ` Michael S. Zick
2011-11-18 19:44                   ` Arnout Vandecappelle
2011-11-18 19:27                 ` Arnout Vandecappelle
2011-11-19  9:26                   ` Bjørn Forsman
2011-11-19 12:13                     ` Arnout Vandecappelle
2011-11-18 16:41             ` Thomas Petazzoni

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox