[Buildroot] [PATCH V1] core: add kconfig option to archive an un-stripped filesystem

[Buildroot] [PATCH V1] core: add kconfig option to archive an un-stripped filesystem

[Colin Foster]

It is reasonable to expect that deployed systems don't contain debugging
symbols. At the very least, it will bloat image size unnecessarily.

There might also be scenarios where a previously released image might
generate core dumps, or require attaching a debugger to a process
well after the filesystem image has been made.

These two goals contradict each other, and is addressed here.

Just before stripping target binaries, allow the option to tar the
target filesystem for archiving. This will create a single image -
rootfs_syms.tgz - with un-stripped target binaries, and allow post-build
debugging and core dump analysis. The final image will be unaffected.

Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
---
 Config.in | 10 ++++++++++
 Makefile  |  3 +++
 2 files changed, 13 insertions(+)

diff --git a/Config.in b/Config.in
index 0d7641633c..a6343f6336 100644
--- a/Config.in
+++ b/Config.in
@@ -443,6 +443,16 @@ config BR2_STRIP_strip
 	  on the target are needed for native debugging, but not when
 	  remote debugging is used.
 
+config BR2_TAR_UNSTRIPPED_FILESYSTEM
+    bool "create filesystem image with unstripped binaries"
+    default n
+    depends on BR2_STRIP_strip
+    help
+      Create an image of the filesystem prior to the 'strip' command.
+      This will allow binaries with debugging symbols to be archived,
+      and should aid in remote debugging and core dump analysis while
+      not taking up space on the target filesystem.
+
 config BR2_STRIP_EXCLUDE_FILES
 	string "executables that should not be stripped"
 	default ""
diff --git a/Makefile b/Makefile
index 09b2066f05..9a5165c24e 100644
--- a/Makefile
+++ b/Makefile
@@ -746,6 +746,9 @@ endif
 	rmdir $(TARGET_DIR)/usr/share 2>/dev/null || true
 ifneq ($(BR2_ENABLE_DEBUG):$(BR2_STRIP_strip),y:)
 	rm -rf $(TARGET_DIR)/lib/debug $(TARGET_DIR)/usr/lib/debug
+endif
+ifeq ($(BR2_TAR_UNSTRIPPED_FILESYSTEM),y)
+	tar -czvf $(BINARIES_DIR)/rootfs_syms.tgz -C $(TARGET_DIR) .
 endif
 	$(STRIP_FIND_CMD) | xargs -0 $(STRIPCMD) 2>/dev/null || true
 	$(STRIP_FIND_SPECIAL_LIBS_CMD) | xargs -0 -r $(STRIPCMD) $(STRIP_STRIP_DEBUG) 2>/dev/null || true
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH V1] core: add kconfig option to archive an un-stripped filesystem

[Thomas Petazzoni via buildroot]

Hello Colin,

On Fri, 28 Jul 2023 16:18:39 -0700
Colin Foster <colin.foster@in-advantage.com> wrote:

> It is reasonable to expect that deployed systems don't contain debugging
> symbols. At the very least, it will bloat image size unnecessarily.
> 
> There might also be scenarios where a previously released image might
> generate core dumps, or require attaching a debugger to a process
> well after the filesystem image has been made.
> 
> These two goals contradict each other, and is addressed here.
> 
> Just before stripping target binaries, allow the option to tar the
> target filesystem for archiving. This will create a single image -
> rootfs_syms.tgz - with un-stripped target binaries, and allow post-build
> debugging and core dump analysis. The final image will be unaffected.
> 
> Signed-off-by: Colin Foster <colin.foster@in-advantage.com>

I understand the reasoning, and what you proposed is indeed a simple
solution that can work today.

However, I think the "grand plan" to solve this issue was to install
everything to $(STAGING_DIR), not only packages installing libraries.
In $(STAGING_DIR), packages are installed unstripped, and with
debugging symbols if BR2_ENABLE_DEBUG=y. One issue with this solution
is that it requires a lot of effort to implement, so nobody ever
tackled that.

Perhaps one concern with your proposal is that the tar you're
generating is not exactly the same as the generated filesystem with
binaries unstripped. Indeed, after the point where you generate this
tarball, a number of other things will happen in the rootfs, most
notably all what happens within fakeroot (permission/ownership tweaks,
etc.). I understand these generally don't cause any issue for the
specific use-case of having binaries with debugging symbols, but it
shows that this option does not really provide a "rootfs with debugging
symbols", but rather a "almost complete but not quite entirely rootfs
with debugging symbols".

Let's see what others think. The advantage of your proposal is that it
is actionable now. For now, what we tell people is that libraries with
debugging symbols are in $(STAGING_DIR), and for packages that don't
install to staging, the debugging symbols are in the build dir of the
package: output/build/foo-1.0.0/foo.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH V1] core: add kconfig option to archive an un-stripped filesystem

[Colin Foster]

Hi Thomas,

On Sat, Jul 29, 2023 at 09:49:52AM +0200, Thomas Petazzoni wrote:
> Hello Colin,
> 
> On Fri, 28 Jul 2023 16:18:39 -0700
> Colin Foster <colin.foster@in-advantage.com> wrote:
> 
> > It is reasonable to expect that deployed systems don't contain debugging
> > symbols. At the very least, it will bloat image size unnecessarily.
> > 
> > There might also be scenarios where a previously released image might
> > generate core dumps, or require attaching a debugger to a process
> > well after the filesystem image has been made.
> > 
> > These two goals contradict each other, and is addressed here.
> > 
> > Just before stripping target binaries, allow the option to tar the
> > target filesystem for archiving. This will create a single image -
> > rootfs_syms.tgz - with un-stripped target binaries, and allow post-build
> > debugging and core dump analysis. The final image will be unaffected.
> > 
> > Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
> 
> I understand the reasoning, and what you proposed is indeed a simple
> solution that can work today.
> 
> However, I think the "grand plan" to solve this issue was to install
> everything to $(STAGING_DIR), not only packages installing libraries.
> In $(STAGING_DIR), packages are installed unstripped, and with
> debugging symbols if BR2_ENABLE_DEBUG=y. One issue with this solution
> is that it requires a lot of effort to implement, so nobody ever
> tackled that.

Thanks for the quick response / review!

When I started thinking about "how could this be done" I was thinking it
would be quite difficult. That would likely be the $(STAGING_DIR)
implementation. But I stumbled upon this not-quite-perfect solution and
figured it was worth getting feedback.

> 
> Perhaps one concern with your proposal is that the tar you're
> generating is not exactly the same as the generated filesystem with
> binaries unstripped. Indeed, after the point where you generate this
> tarball, a number of other things will happen in the rootfs, most
> notably all what happens within fakeroot (permission/ownership tweaks,
> etc.). I understand these generally don't cause any issue for the
> specific use-case of having binaries with debugging symbols, but it
> shows that this option does not really provide a "rootfs with debugging
> symbols", but rather a "almost complete but not quite entirely rootfs
> with debugging symbols".

I fully agree. It could be misleading, and maybe any subtleties should
be clearly defined in the Kconfig help. I've also found the
"THIS_IS_NOT_YOUR_ROOT_FILESYSTEM" file to be very helpful :-)

> Let's see what others think. The advantage of your proposal is that it
> is actionable now. For now, what we tell people is that libraries with
> debugging symbols are in $(STAGING_DIR), and for packages that don't
> install to staging, the debugging symbols are in the build dir of the
> package: output/build/foo-1.0.0/foo.

That sounds good, and I look forward to any feedback / suggestions.

Something to consider is the file / directory sizes. From a quick
Beaglebone build, the output/build directory is 9GB, while the
rootfs_syms.tgz is 33MB (and it is 90MB uncompressed). I suppose I could
compress and drag around the build directory (which compressed down to
~2GB) for completeness. But realistically there's a lot of baggage there
that won't be needed.


Colin
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH V1] core: add kconfig option to archive an un-stripped filesystem

[Thomas Petazzoni via buildroot]

On Sat, 29 Jul 2023 13:43:22 -0700
Colin Foster <colin.foster@in-advantage.com> wrote:

> Something to consider is the file / directory sizes. From a quick
> Beaglebone build, the output/build directory is 9GB, while the
> rootfs_syms.tgz is 33MB (and it is 90MB uncompressed). I suppose I could
> compress and drag around the build directory (which compressed down to
> ~2GB) for completeness. But realistically there's a lot of baggage there
> that won't be needed.

Agreed, and that's why installing everything to staging is most likely
the way to go (staging is much smaller than output/build, and the
contents of staging are distributed within the SDK, which makes sense
for debugging purposes).

The thing is that installing everything in staging is a major change in
Buildroot... At the very least all generic-package packages need to be
modified.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot