[PATCH v2] builddeb: remove debian/files before build

[PATCH v2] builddeb: remove debian/files before build

[riku.voipio]

From: Riku Voipio <riku.voipio@linaro.org>

As debian/files keeps accumulating entries, the changes file
will end up growing indefinelty. Remove the file in clean
rule and before build starts.

Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Reported-by: Doug Smythies <dsmythies@telus.net>
---
 scripts/package/builddeb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index 0cd46e1..b967e4f 100755
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -115,7 +115,7 @@ esac
 BUILD_DEBUG="$(grep -s '^CONFIG_DEBUG_INFO=y' $KCONFIG_CONFIG || true)"
 
 # Setup the directory structure
-rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir"
+rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir" $objtree/debian/files
 mkdir -m 755 -p "$tmpdir/DEBIAN"
 mkdir -p "$tmpdir/lib" "$tmpdir/boot"
 mkdir -p "$fwdir/lib/firmware/$version/"
@@ -408,7 +408,7 @@ binary-arch:
 	\$(MAKE) KDEB_SOURCENAME=${sourcename} KDEB_PKGVERSION=${packageversion} bindeb-pkg
 
 clean:
-	rm -rf debian/*tmp
+	rm -rf debian/*tmp debian/files
 	mv debian/ debian.backup # debian/ might be cleaned away
 	\$(MAKE) clean
 	mv debian.backup debian
-- 
2.1.4

Re: [PATCH v2] builddeb: remove debian/files before build

[Kalle Valo]

<riku.voipio@linaro.org> writes:

> From: Riku Voipio <riku.voipio@linaro.org>
>
> As debian/files keeps accumulating entries, the changes file
> will end up growing indefinelty. Remove the file in clean
> rule and before build starts.
>
> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
> Reported-by: Doug Smythies <dsmythies@telus.net>

Thanks, I had the same problem and this fixed it.

Tested-by: Kalle Valo <kvalo@codeaurora.org>

But it would be nice to have a fixes line to point out what commit this
fixes:

Fixes: 3716001bcb7f ("deb-pkg: add source package")

-- 
Kalle Valo

RE: [PATCH v2] builddeb: remove debian/files before build

[Doug Smythies]

On 2015.09.15 11:22 Kalle Valo wrote:
> <riku.voipio@linaro.org> writes:
>> From: Riku Voipio <riku.voipio@linaro.org>
>>
>> As debian/files keeps accumulating entries, the changes file
>> will end up growing indefinelty. Remove the file in clean
>> rule and before build starts.
>>
>> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
>> Reported-by: Doug Smythies <dsmythies@telus.net>

> Thanks, I had the same problem and this fixed it.
> Tested-by: Kalle Valo <kvalo@codeaurora.org>

I tested also. Thanks.

For an incremental compile, where only one source file
has changed, the time went from 7 minutes and 47 seconds
to 4 minutes and 33 seconds on my test server.
And the spew of warnings and such are gone.

For reference a clean compile typically takes about
23 minutes.

The overhead for making the .changes file is now on the
order of 2 to 4 seconds.

Obviously, results will vary widely for others, depending
on their hardware.

Re: [PATCH v2] builddeb: remove debian/files before build

[Riku Voipio]

On 15 September 2015 at 22:11, Doug Smythies <dsmythies@telus.net> wrote:
> On 2015.09.15 11:22 Kalle Valo wrote:
>> <riku.voipio@linaro.org> writes:
>>> From: Riku Voipio <riku.voipio@linaro.org>
>>>
>>> As debian/files keeps accumulating entries, the changes file
>>> will end up growing indefinelty. Remove the file in clean
>>> rule and before build starts.
>>>
>>> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
>>> Reported-by: Doug Smythies <dsmythies@telus.net>

>> Thanks, I had the same problem and this fixed it.
>> Tested-by: Kalle Valo <kvalo@codeaurora.org>

> I tested also. Thanks.

Michal, can you get this as fix for 4.3, since this is a regression?

Riku

> For an incremental compile, where only one source file
> has changed, the time went from 7 minutes and 47 seconds
> to 4 minutes and 33 seconds on my test server.
> And the spew of warnings and such are gone.
>
> For reference a clean compile typically takes about
> 23 minutes.
>
> The overhead for making the .changes file is now on the
> order of 2 to 4 seconds.
>
> Obviously, results will vary widely for others, depending
> on their hardware.
>
>

Re: [PATCH v2] builddeb: remove debian/files before build

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

On Tue, 2015-09-15 at 21:11 +0300, riku.voipio@linaro.org wrote:
> From: Riku Voipio <riku.voipio@linaro.org>
> 
> As debian/files keeps accumulating entries, the changes file
> will end up growing indefinelty. Remove the file in clean
> rule and before build starts.
> 
> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
> Reported-by: Doug Smythies <dsmythies@telus.net>
[...]

FWIW:

Acked-by: Ben Hutchings <ben@decadent.org.uk>

Ben.

-- 
Ben Hutchings
If the facts do not conform to your theory, they must be disposed of.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: [PATCH v2] builddeb: remove debian/files before build

[Kalle Valo]

Ben Hutchings <ben@decadent.org.uk> writes:

> On Tue, 2015-09-15 at 21:11 +0300, riku.voipio@linaro.org wrote:
>> From: Riku Voipio <riku.voipio@linaro.org>
>> 
>> As debian/files keeps accumulating entries, the changes file
>> will end up growing indefinelty. Remove the file in clean
>> rule and before build starts.
>> 
>> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
>> Reported-by: Doug Smythies <dsmythies@telus.net>
> [...]
>
> FWIW:
>
> Acked-by: Ben Hutchings <ben@decadent.org.uk>

Marek, this is a pretty annoying regression. Is it possible to get this
to 4.3?

-- 
Kalle Valo

RE: [PATCH v2] builddeb: remove debian/files before build

[Doug Smythies]

On 2015.09.22 04:25 Kalle Valo wrote:
> Ben Hutchings <ben@decadent.org.uk> writes:
>> On Tue, 2015-09-15 at 21:11 +0300, riku.voipio@linaro.org wrote:
>>> From: Riku Voipio <riku.voipio@linaro.org>
>>> 
>>> As debian/files keeps accumulating entries, the changes file
>>> will end up growing indefinelty. Remove the file in clean
>>> rule and before build starts.
>>> 
>>> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
>>> Reported-by: Doug Smythies <dsmythies@telus.net>
>> [...]
>>
>> FWIW:
>>
>> Acked-by: Ben Hutchings <ben@decadent.org.uk>
>
> Marek, this is a pretty annoying regression. Is it possible to get this
> to 4.3?

Perhaps I do not understand the process here, but having reported the issue
within a day of kernel 4.3-rc1 being released, and then what seems to be an
acceptable patch being suggested, and tested, with plenty of time to make
into -rc2, why hasn't it been included yet (-rc4)?

Re: [PATCH v2] builddeb: remove debian/files before build

[Riku Voipio]

On 5 October 2015 at 04:03, Doug Smythies <dsmythies@telus.net> wrote:
> On 2015.09.22 04:25 Kalle Valo wrote:
>> Ben Hutchings <ben@decadent.org.uk> writes:
>>> On Tue, 2015-09-15 at 21:11 +0300, riku.voipio@linaro.org wrote:
>>>> From: Riku Voipio <riku.voipio@linaro.org>
>>>>
>>>> As debian/files keeps accumulating entries, the changes file
>>>> will end up growing indefinelty. Remove the file in clean
>>>> rule and before build starts.
>>>>
>>>> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
>>>> Reported-by: Doug Smythies <dsmythies@telus.net>
>>> [...]
>>>
>>> FWIW:
>>>
>>> Acked-by: Ben Hutchings <ben@decadent.org.uk>
>>
>> Marek, this is a pretty annoying regression. Is it possible to get this
>> to 4.3?
>
> Perhaps I do not understand the process here, but having reported the issue
> within a day of kernel 4.3-rc1 being released, and then what seems to be an
> acceptable patch being suggested, and tested, with plenty of time to make
> into -rc2, why hasn't it been included yet (-rc4)?

Hi Andrew,

Could you pass this patch in your series, as the regression doesn't
seem to be getting through via the maintainer. It would be
embarrassing to have it in 4.3 since it was reported and fixed timely.
The  patch is at:

http://permalink.gmane.org/gmane.linux.kbuild.devel/13710

Tests and acks at:

http://comments.gmane.org/gmane.linux.kbuild.devel/13710


Riku

Re: [PATCH v2] builddeb: remove debian/files before build

[Andrew Morton]

On Mon, 5 Oct 2015 15:39:40 +0300 Riku Voipio <riku.voipio@linaro.org> wrote:

> On 5 October 2015 at 04:03, Doug Smythies <dsmythies@telus.net> wrote:
> > On 2015.09.22 04:25 Kalle Valo wrote:
> >> Ben Hutchings <ben@decadent.org.uk> writes:
> >>> On Tue, 2015-09-15 at 21:11 +0300, riku.voipio@linaro.org wrote:
> >>>> From: Riku Voipio <riku.voipio@linaro.org>
> >>>>
> >>>> As debian/files keeps accumulating entries, the changes file
> >>>> will end up growing indefinelty. Remove the file in clean
> >>>> rule and before build starts.
> >>>>
> >>>> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
> >>>> Reported-by: Doug Smythies <dsmythies@telus.net>
> >>> [...]
> >>>
> >>> FWIW:
> >>>
> >>> Acked-by: Ben Hutchings <ben@decadent.org.uk>
> >>
> >> Marek, this is a pretty annoying regression. Is it possible to get this
> >> to 4.3?
> >
> > Perhaps I do not understand the process here, but having reported the issue
> > within a day of kernel 4.3-rc1 being released, and then what seems to be an
> > acceptable patch being suggested, and tested, with plenty of time to make
> > into -rc2, why hasn't it been included yet (-rc4)?
> 
> Hi Andrew,
> 
> Could you pass this patch in your series, as the regression doesn't
> seem to be getting through via the maintainer. It would be
> embarrassing to have it in 4.3 since it was reported and fixed timely.
> The  patch is at:
> 
> http://permalink.gmane.org/gmane.linux.kbuild.devel/13710
> 
> Tests and acks at:
> 
> http://comments.gmane.org/gmane.linux.kbuild.devel/13710

I'm not subscribed to linux-kbuild and the browser won't do
save-as-text, so I typed it all in again.  Please check the below.

I'll put this in my mainline-later queue, which means I'll send it
Linuswards next week if it doesn't pop up in linux-next before then.

The changelog doesn't actually describe this patch as a regression fix.
It makes the patch look like a regular old bugfix.  And the changelog
doesn't describe the severity of the bug either.  Please do pay
attention to such details to guide others in deciding which kernel
version(s) need fixing.


From: Riku Voipio <riku.voipio@linaro.org>
Subject: builddeb: remove debian/files before build

As debian/files keeps accumulating entries, the changes file will end up
growing indefinelty.  Remove the file in clean rule and before build
starts.

Fixes: 3716001bcb7f ("deb-pkg: add source package")
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Reported-by: Doug Smythies <dsmythies@telus.net>
Tested-by: Doug Smythies <dsmythies@telus.net>
Tested-by: Kalle Valo <kvalo@codeaurora.org>
Acked-by: Ben Hutchings <ben@decadent.org.uk>
Cc: Michal Marek <mmarek@suse.cz>
Cc: maximilian attems <maks@stro.at>
Cc: Chris J Arges <chris.j.arges@canonical.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 scripts/package/builddeb |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff -puN scripts/package/builddeb~a scripts/package/builddeb
--- a/scripts/package/builddeb~a
+++ a/scripts/package/builddeb
@@ -115,7 +115,7 @@ esac
 BUILD_DEBUG="$(grep -s '^CONFIG_DEBUG_INFO=y' $KCONFIG_CONFIG || true)"
 
 # Setup the directory structure
-rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir"
+rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir" $objtree/debian/files
 mkdir -m 755 -p "$tmpdir/DEBIAN"
 mkdir -p "$tmpdir/lib" "$tmpdir/boot"
 mkdir -p "$fwdir/lib/firmware/$version/"
@@ -408,7 +408,7 @@ binary-arch:
 	\$(MAKE) KDEB_SOURCENAME=${sourcename} KDEB_PKGVERSION=${packageversion} bindeb-pkg
 
 clean:
-	rm -rf debian/*tmp
+	rm -rf debian/*tmp debian/files
 	mv debian/ debian.backup # debian/ might be cleaned away
 	\$(MAKE) clean
 	mv debian.backup debian
_

[PATCH v3] builddeb: remove debian/files before build

[riku.voipio]

From: Riku Voipio <riku.voipio@linaro.org>

Commit 3716001bcb7f added the ability to create a debian
changelog file. This exposed that previously the builddeb
script hasn't cleared debian/files between builds.

As debian/files keeps accumulating entries, the changes file
will end up growing indefinelty. With outdated entries in
debian/files, builddeb script will exit with failure. This
regression impacts those who use "make deb-pkg" target to
build kernel into a .deb package and never use "make mrproper"
or other means to clean kernel tree from generated directories.

To fix the regression, remove debian/files before starting
build and in the generated clean rule.

Fixes: 3716001bcb7f ("deb-pkg: add source package")
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Reported-by: Doug Smythies <dsmythies@telus.net>
Tested-by: Doug Smythies <dsmythies@telus.net>
Tested-by: Kalle Valo <kvalo@codeaurora.org>
Acked-by: Ben Hutchings <ben@decadent.org.uk>
Cc: Michal Marek <mmarek@suse.cz>
Cc: maximilian attems <maks@stro.at>
Cc: Chris J Arges <chris.j.arges@canonical.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
v3: improved commit message
 scripts/package/builddeb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index 0cd46e1..b967e4f 100755
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -115,7 +115,7 @@ esac
 BUILD_DEBUG="$(grep -s '^CONFIG_DEBUG_INFO=y' $KCONFIG_CONFIG || true)"
 
 # Setup the directory structure
-rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir"
+rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir" $objtree/debian/files
 mkdir -m 755 -p "$tmpdir/DEBIAN"
 mkdir -p "$tmpdir/lib" "$tmpdir/boot"
 mkdir -p "$fwdir/lib/firmware/$version/"
@@ -408,7 +408,7 @@ binary-arch:
 	\$(MAKE) KDEB_SOURCENAME=${sourcename} KDEB_PKGVERSION=${packageversion} bindeb-pkg
 
 clean:
-	rm -rf debian/*tmp
+	rm -rf debian/*tmp debian/files
 	mv debian/ debian.backup # debian/ might be cleaned away
 	\$(MAKE) clean
 	mv debian.backup debian
-- 
2.1.4