[PATCH v6 0/4] deb-pkg: bring it a little closer to debian packaging

[PATCH v6 0/4] deb-pkg: bring it a little closer to debian packaging

[Anisse Astier]

These little fixes should bring debian packaging closer to the way it's done in debian.

Changes in:
v2:
 - only build debug packages when CONFIG_DEBUG_INFO=y
 - build debug package last.
 - more verbose debug package description
 - put package in section debug
v3:
 - remove duplicate code from v2
v4:
 - fixes thanks to Ben Hutchings' review
 - use KCONFIG_CONFIG instead of .config
 - use installed path specific to each architecture instead of hard-coded
   /boot/vmlinuz-$version, based on
   http://anonscm.debian.org/viewvc/kernel/dists/wheezy/linux/debian/rules.real?view=markup
v5:
 - fixes thanks to Ben Hutchings' review
 - don't use $objtree as it's redundant in build scripts
 - add symbolic links to vmlinux to work with other tools than perf
 - pick-up reviewed-by from ml for patch 3 and 4
v6:
 - first patch: handle .config path properly inside package archive
 - pick-up reviewed-by and acked-by's


Anisse Astier (4):
  deb-pkg: use KCONFIG_CONFIG instead of .config file directly
  deb-pkg: split debug symbols in their own package
  deb-pkg: fix installed image path on parisc, mips and powerpc
  deb-pkg: add a hook argument to match debian hooks parameters

 scripts/package/builddeb | 92 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 77 insertions(+), 15 deletions(-)

-- 
1.8.3.rc1

[PATCH v6 1/4] deb-pkg: use KCONFIG_CONFIG instead of .config file directly

[Anisse Astier]

Signed-off-by: Anisse Astier <anisse@astier.eu>
Reviewed-by: Ben Hutchings <ben@decadent.org.uk>
---
 scripts/package/builddeb | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index acb8650..a8662ef 100644
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -41,9 +41,9 @@ create_package() {
 	parisc*)
 		debarch=hppa ;;
 	mips*)
-		debarch=mips$(grep -q CPU_LITTLE_ENDIAN=y .config && echo el) ;;
+		debarch=mips$(grep -q CPU_LITTLE_ENDIAN=y $KCONFIG_CONFIG && echo el) ;;
 	arm*)
-		debarch=arm$(grep -q CONFIG_AEABI=y .config && echo el) ;;
+		debarch=arm$(grep -q CONFIG_AEABI=y $KCONFIG_CONFIG && echo el) ;;
 	*)
 		echo "" >&2
 		echo "** ** **  WARNING  ** ** **" >&2
@@ -106,12 +106,12 @@ fi
 if [ "$ARCH" = "um" ] ; then
 	$MAKE linux
 	cp System.map "$tmpdir/usr/lib/uml/modules/$version/System.map"
-	cp .config "$tmpdir/usr/share/doc/$packagename/config"
+	cp $KCONFIG_CONFIG "$tmpdir/usr/share/doc/$packagename/config"
 	gzip "$tmpdir/usr/share/doc/$packagename/config"
 	cp $KBUILD_IMAGE "$tmpdir/usr/bin/linux-$version"
 else 
 	cp System.map "$tmpdir/boot/System.map-$version"
-	cp .config "$tmpdir/boot/config-$version"
+	cp $KCONFIG_CONFIG "$tmpdir/boot/config-$version"
 	# Not all arches include the boot path in KBUILD_IMAGE
 	if [ -e $KBUILD_IMAGE ]; then
 		cp $KBUILD_IMAGE "$tmpdir/boot/vmlinuz-$version"
@@ -120,7 +120,7 @@ else
 	fi
 fi
 
-if grep -q '^CONFIG_MODULES=y' .config ; then
+if grep -q '^CONFIG_MODULES=y' $KCONFIG_CONFIG ; then
 	INSTALL_MOD_PATH="$tmpdir" $MAKE KBUILD_SRC= modules_install
 	rm -f "$tmpdir/lib/modules/$version/build"
 	rm -f "$tmpdir/lib/modules/$version/source"
@@ -245,11 +245,12 @@ fi
 # Build header package
 (cd $srctree; find . -name Makefile\* -o -name Kconfig\* -o -name \*.pl > "$objtree/debian/hdrsrcfiles")
 (cd $srctree; find arch/$SRCARCH/include include scripts -type f >> "$objtree/debian/hdrsrcfiles")
-(cd $objtree; find arch/$SRCARCH/include .config Module.symvers include scripts -type f >> "$objtree/debian/hdrobjfiles")
+(cd $objtree; find arch/$SRCARCH/include Module.symvers include scripts -type f >> "$objtree/debian/hdrobjfiles")
 destdir=$kernel_headers_dir/usr/src/linux-headers-$version
 mkdir -p "$destdir"
 (cd $srctree; tar -c -f - -T "$objtree/debian/hdrsrcfiles") | (cd $destdir; tar -xf -)
 (cd $objtree; tar -c -f - -T "$objtree/debian/hdrobjfiles") | (cd $destdir; tar -xf -)
+(cd $objtree; cp $KCONFIG_CONFIG $destdir/.config) # copy .config manually to be where it's expected to be
 ln -sf "/usr/src/linux-headers-$version" "$kernel_headers_dir/lib/modules/$version/build"
 rm -f "$objtree/debian/hdrsrcfiles" "$objtree/debian/hdrobjfiles"
 arch=$(dpkg --print-architecture)
-- 
1.8.3.rc1

[PATCH v6 2/4] deb-pkg: split debug symbols in their own package

[Anisse Astier]

This can reduce almost 3 times the size of the linux-image package,
while keeping the debug symbols available for this particular build, in
their own package.

This mimics the way kernels are built in debian, ubuntu, or with
make-kpkg, and comes at the price of a small slowdown in the building of
packages.

Signed-off-by: Anisse Astier <anisse@astier.eu>
Cc: Ben Hutchings <ben@decadent.org.uk>
Acked-by: maximilian attems <max@stro.at>
---
 scripts/package/builddeb | 50 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index a8662ef..541a1cf 100644
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -78,17 +78,21 @@ tmpdir="$objtree/debian/tmp"
 fwdir="$objtree/debian/fwtmp"
 kernel_headers_dir="$objtree/debian/hdrtmp"
 libc_headers_dir="$objtree/debian/headertmp"
+dbg_dir="$objtree/debian/dbgtmp"
 packagename=linux-image-$version
 fwpackagename=linux-firmware-image
 kernel_headers_packagename=linux-headers-$version
 libc_headers_packagename=linux-libc-dev
+dbg_packagename=$packagename-dbg
 
 if [ "$ARCH" = "um" ] ; then
 	packagename=user-mode-linux-$version
 fi
 
+BUILD_DEBUG="$(grep -s '^CONFIG_DEBUG_INFO=y' $KCONFIG_CONFIG || true)"
+
 # Setup the directory structure
-rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir"
+rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir" "$libc_headers_dir" "$dbg_dir"
 mkdir -m 755 -p "$tmpdir/DEBIAN"
 mkdir -p  "$tmpdir/lib" "$tmpdir/boot" "$tmpdir/usr/share/doc/$packagename"
 mkdir -m 755 -p "$fwdir/DEBIAN"
@@ -101,6 +105,10 @@ mkdir -p "$kernel_headers_dir/lib/modules/$version/"
 if [ "$ARCH" = "um" ] ; then
 	mkdir -p "$tmpdir/usr/lib/uml/modules/$version" "$tmpdir/usr/bin"
 fi
+if [ -n "$BUILD_DEBUG" ] ; then
+	mkdir -p "$dbg_dir/usr/share/doc/$dbg_packagename"
+	mkdir -m 755 -p "$dbg_dir/DEBIAN"
+fi
 
 # Build and install the kernel
 if [ "$ARCH" = "um" ] ; then
@@ -128,6 +136,20 @@ if grep -q '^CONFIG_MODULES=y' $KCONFIG_CONFIG ; then
 		mv "$tmpdir/lib/modules/$version"/* "$tmpdir/usr/lib/uml/modules/$version/"
 		rmdir "$tmpdir/lib/modules/$version"
 	fi
+	if [ -n "$BUILD_DEBUG" ] ; then
+		(
+			cd $tmpdir
+			for module in $(find lib/modules/ -name *.ko); do
+				mkdir -p $(dirname $dbg_dir/usr/lib/debug/$module)
+				# only keep debug symbols in the debug file
+				objcopy --only-keep-debug $module $dbg_dir/usr/lib/debug/$module
+				# strip original module from debug symbols
+				objcopy --strip-debug $module
+				# then add a link to those
+				objcopy --add-gnu-debuglink=$dbg_dir/usr/lib/debug/$module $module
+			done
+		)
+	fi
 fi
 
 if [ "$ARCH" != "um" ]; then
@@ -300,4 +322,30 @@ fi
 
 create_package "$packagename" "$tmpdir"
 
+if [ -n "$BUILD_DEBUG" ] ; then
+	# Build debug package
+	# Different tools want the image in different locations
+	# perf
+	mkdir -p $dbg_dir/usr/lib/debug/lib/modules/$version/
+	cp vmlinux $dbg_dir/usr/lib/debug/lib/modules/$version/
+	# systemtap
+	mkdir -p $dbg_dir/usr/lib/debug/boot/
+	ln -s ../lib/modules/$version/vmlinux $dbg_dir/usr/lib/debug/boot/vmlinux-$version
+	# kdump-tools
+	ln -s lib/modules/$version/vmlinux $dbg_dir/usr/lib/debug/vmlinux-$version
+
+	cat <<EOF >> debian/control
+
+Package: $dbg_packagename
+Section: debug
+Provides: linux-debug, linux-debug-$version
+Architecture: any
+Description: Linux kernel debugging symbols for $version
+ This package will come in handy if you need to debug the kernel. It provides
+ all the necessary debug symbols for the kernel and its modules.
+EOF
+
+	create_package "$dbg_packagename" "$dbg_dir"
+fi
+
 exit 0
-- 
1.8.3.rc1

Re: [PATCH v6 2/4] deb-pkg: split debug symbols in their own package

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On Wed, 2013-07-03 at 16:02 +0200, Anisse Astier wrote:
> This can reduce almost 3 times the size of the linux-image package,
> while keeping the debug symbols available for this particular build, in
> their own package.
> 
> This mimics the way kernels are built in debian, ubuntu, or with
> make-kpkg, and comes at the price of a small slowdown in the building of
> packages.
> 
> Signed-off-by: Anisse Astier <anisse@astier.eu>
> Cc: Ben Hutchings <ben@decadent.org.uk>
> Acked-by: maximilian attems <max@stro.at>
[...]

Reviewed-by: Ben Hutchings <ben@decadent.org.uk>

-- 
Ben Hutchings
Tomorrow will be cancelled due to lack of interest.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [PATCH v6 2/4] deb-pkg: split debug symbols in their own package

[Anisse Astier]

Michal,

Every patch has now been reviewed, do you think it would be possible to
take this series for 3.12 ?

Thanks,

Anisse

On Thu, 04 Jul 2013 02:46:00 +0100, Ben Hutchings <ben@decadent.org.uk> wrote :

> On Wed, 2013-07-03 at 16:02 +0200, Anisse Astier wrote:
> > This can reduce almost 3 times the size of the linux-image package,
> > while keeping the debug symbols available for this particular build, in
> > their own package.
> > 
> > This mimics the way kernels are built in debian, ubuntu, or with
> > make-kpkg, and comes at the price of a small slowdown in the building of
> > packages.
> > 
> > Signed-off-by: Anisse Astier <anisse@astier.eu>
> > Cc: Ben Hutchings <ben@decadent.org.uk>
> > Acked-by: maximilian attems <max@stro.at>
> [...]
> 
> Reviewed-by: Ben Hutchings <ben@decadent.org.uk>
> 

Re: [PATCH v6 2/4] deb-pkg: split debug symbols in their own package

[Michal Marek]

Dne 10.7.2013 15:09, Anisse Astier napsal(a):
> Michal,
> 
> Every patch has now been reviewed, do you think it would be possible to
> take this series for 3.12 ?

Yes, I will merge it as soon as this merge window closes.

Michal

Re: [PATCH v6 2/4] deb-pkg: split debug symbols in their own package

[Michal Marek]

On 10.7.2013 16:21, Michal Marek wrote:
> Dne 10.7.2013 15:09, Anisse Astier napsal(a):
>> Michal,
>>
>> Every patch has now been reviewed, do you think it would be possible to
>> take this series for 3.12 ?
> 
> Yes, I will merge it as soon as this merge window closes.

This is now in kbuild.git#misc. Thanks a lot to everyone!

Michal

[PATCH v6 3/4] deb-pkg: fix installed image path on parisc, mips and powerpc

[Anisse Astier]

Signed-off-by: Anisse Astier <anisse@astier.eu>
Reviewed-by: Ben Hutchings <ben@decadent.org.uk>
---
 scripts/package/builddeb | 27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index 541a1cf..aebc66e 100644
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -89,6 +89,20 @@ if [ "$ARCH" = "um" ] ; then
 	packagename=user-mode-linux-$version
 fi
 
+# Not all arches have the same installed path in debian
+# XXX: have each arch Makefile export a variable of the canonical image install
+# path instead
+case $ARCH in
+um)
+	installed_image_path="usr/bin/linux-$version"
+	;;
+parisc|mips|powerpc)
+	installed_image_path="boot/vmlinux-$version"
+	;;
+*)
+	installed_image_path="boot/vmlinuz-$version"
+esac
+
 BUILD_DEBUG="$(grep -s '^CONFIG_DEBUG_INFO=y' $KCONFIG_CONFIG || true)"
 
 # Setup the directory structure
@@ -116,16 +130,15 @@ if [ "$ARCH" = "um" ] ; then
 	cp System.map "$tmpdir/usr/lib/uml/modules/$version/System.map"
 	cp $KCONFIG_CONFIG "$tmpdir/usr/share/doc/$packagename/config"
 	gzip "$tmpdir/usr/share/doc/$packagename/config"
-	cp $KBUILD_IMAGE "$tmpdir/usr/bin/linux-$version"
 else 
 	cp System.map "$tmpdir/boot/System.map-$version"
 	cp $KCONFIG_CONFIG "$tmpdir/boot/config-$version"
-	# Not all arches include the boot path in KBUILD_IMAGE
-	if [ -e $KBUILD_IMAGE ]; then
-		cp $KBUILD_IMAGE "$tmpdir/boot/vmlinuz-$version"
-	else
-		cp arch/$ARCH/boot/$KBUILD_IMAGE "$tmpdir/boot/vmlinuz-$version"
-	fi
+fi
+# Not all arches include the boot path in KBUILD_IMAGE
+if [ -e $KBUILD_IMAGE ]; then
+	cp $KBUILD_IMAGE "$tmpdir/$installed_image_path"
+else
+	cp arch/$ARCH/boot/$KBUILD_IMAGE "$tmpdir/$installed_image_path"
 fi
 
 if grep -q '^CONFIG_MODULES=y' $KCONFIG_CONFIG ; then
-- 
1.8.3.rc1

[PATCH v6 4/4] deb-pkg: add a hook argument to match debian hooks parameters

[Anisse Astier]

We now provide the installed image path to the kernel hooks.

This should allow the package to better integrate with debian hooks, and
should not be too disruptive of hooks supporting only one parameter.

Signed-off-by: Anisse Astier <anisse@astier.eu>
Reviewed-by: Ben Hutchings <ben@decadent.org.uk>
Acked-by: maximilian attems <max@stro.at>
---
 scripts/package/builddeb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/package/builddeb b/scripts/package/builddeb
index aebc66e..de89923 100644
--- a/scripts/package/builddeb
+++ b/scripts/package/builddeb
@@ -184,7 +184,7 @@ set -e
 # Pass maintainer script parameters to hook scripts
 export DEB_MAINT_PARAMS="\$*"
 
-test -d $debhookdir/$script.d && run-parts --arg="$version" $debhookdir/$script.d
+test -d $debhookdir/$script.d && run-parts --arg="$version" --arg="/$installed_image_path" $debhookdir/$script.d
 exit 0
 EOF
 	chmod 755 "$tmpdir/DEBIAN/$script"
-- 
1.8.3.rc1