[Buildroot] [Bug 11411] New: check-uniq-files issue

[Buildroot] [Bug 11411] New: check-uniq-files issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

            Bug ID: 11411
           Summary: check-uniq-files issue
           Product: buildroot
           Version: 2018.02.4
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned at buildroot.uclibc.org
          Reporter: jpcartal at free.fr
                CC: buildroot at uclibc.org
  Target Milestone: ---

Hello,

The check-uniq-files script is returning lots of false positive when successive
builds are launched.

Those detected in the target directory seems to be coming from automatic
stripping that is done in the final step in the target directory:
STRIP_FIND_CMD = find $(TARGET_DIR)
ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
STRIP_FIND_CMD += \( $(call finddirclauses,$(TARGET_DIR),$(call
qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
endif
STRIP_FIND_CMD += -type f \( -perm /111 -o -name '*.so*' \)

When run this commmand modify the modification date of the stripped files, so
on next build, the check-uniq-files script will detect all modified files as
being touched by all packages e.g.:

Warning: target file "./usr/lib/libdevmapper-event-lvm2thin.so" is touched by
more than one package: [u'lvm2', u'popt', u'util-linux', u'cryptsetup',
u'expat', u'dbus', u'e2fsprogs', u'kmod', u'eudev', u'libpng', u'freetype',
u'libopenssl', u'ffmpeg', u'giflib', u'icu', u'libffi', u'pcre', u'libglib2',
u'harfbuzz', u'hdparm', u'ifupdown-scripts', u'initscripts', u'iperf', u'iw',
u'jpeg-turbo', u'jpeg', u'json-glib', u'libcap', u'openssl', u'libcurl',
u'libevent', u'libexif', u'libnspr', u'sqlite', u'libnss', u'liburiparser',
u'libxml2', u'monit', u'openssh', u'udev', u'pciutils', u'strace', u'sysvinit',
u'wireless-regdb', u'wpa_supplicant', u'netgem_bluez5_utils', u'brcm-patchram',
u'netgem-azure-c-shared-utility', u'netgem-azure-uamqp-c',
u'netgem-azure-umqtt-c', u'netgem-azure-iot-sdk-c', u'netgem_boot-bin',
u'netgem_boot', u'netgem_erpc', u'cobalt', u'netgem-bootloader',
u'netgem_nexus_standby', u'netgem_secure_dma_code',
u'realtek_8822BE_BT_driver', u'busybox']

Maybe the -p strip option could be used to avoid this issue ?

Regards.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #1 from Arnout Vandecappelle <arnout@mind.be> ---
Sounds like an excellent idea! The -p option has existed since 2000 so no
problem there. Care to send a patch to add it?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #2 from Jean-pierre Cartal <jpcartal@free.fr> ---
Here is the patch that I tested on my builds:

diff --git a/package/Makefile.in b/package/Makefile.in
index 58af2ef242..7d3d7c9028 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -216,7 +216,7 @@ TARGET_OBJDUMP  = $(TARGET_CROSS)objdump
 ifeq ($(BR2_STRIP_strip),y)
 STRIP_STRIP_DEBUG := --strip-debug
 TARGET_STRIP = $(TARGET_CROSS)strip
-STRIPCMD = $(TARGET_CROSS)strip --remove-section=.comment
--remove-section=.note
+STRIPCMD = $(TARGET_CROSS)strip --remove-section=.comment
--remove-section=.note -p
 else
 TARGET_STRIP = /bin/true
 STRIPCMD = $(TARGET_STRIP)


BTW, there are also a lot of false positive in staging because of this command
:
$(Q)find $(STAGING_DIR)/usr/lib* -name "*.la" | xargs --no-run-if-empty \
                $(SED) "s:$(BASE_DIR):@BASE_DIR@:g" \
                        -e "s:$(STAGING_DIR):@STAGING_DIR@:g" \
                        $(if $(TOOLCHAIN_EXTERNAL_INSTALL_DIR),\
                                -e
"s:$(TOOLCHAIN_EXTERNAL_INSTALL_DIR):@TOOLCHAIN_EXTERNAL_INSTALL_DIR@:g") \
                        -e "s:\(['= ]\)/usr:\\1 at STAGING_DIR@/usr:g" \
                        $(if $(TOOLCHAIN_EXTERNAL_INSTALL_DIR),\
                                -e
"s:@TOOLCHAIN_EXTERNAL_INSTALL_DIR@:$(TOOLCHAIN_EXTERNAL_INSTALL_DIR):g") \
                        -e "s:@STAGING_DIR@:$(STAGING_DIR):g" \
                        -e "s:@BASE_DIR@:$(BASE_DIR):g"

Do you think a patch using stat/touch to keep the original modification time of
impacted files be interesting ?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #3 from Arnout Vandecappelle <arnout@mind.be> ---
The originally reported problem actually only occurs when you do a
package-reinstall. Not really the most important use case I'd say. If you do a
package rebuild, it works correctly.

For the *.la files, the problem indeed occurs, but it already occurs on the
first run. Indeed, after every package, we fix up the .la files, and even if
they are not really modified, their timestamp is updated. So that really has to
be fixed. However, a better fix is to not update all the .la files, but only
the .la files really touched by that package.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #4 from Jean-pierre Cartal <jpcartal@free.fr> ---
Regarding the original issue, it was not my impression that the bug was only
happening on package reinstall.

For instance the example I gave with 
Warning: target file "./usr/lib/libdevmapper-event-lvm2thin.so" was not
triggered by a reinstallation of the lvm package if that's what you mean ?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #5 from Arnout Vandecappelle <arnout@mind.be> ---
As I said, I couldn't reproduce it, so how did you trigger it?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files target issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

Jean-pierre Cartal <jpcartal@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|check-uniq-files issue      |check-uniq-files target
                   |                            |issue

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files target issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #6 from Jean-pierre Cartal <jpcartal@free.fr> ---
You're correct, this is happening when I force a package reinstallation, a full
package rebuild does not trigger the issue.

However, I think that adding the -p option would still be useful to avoid this
issue when reinstalling a package.

BTW I created a separate ticket for warnings in staging area (#11416)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files target issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at buildroot.uclibc |yann.morin.1998 at free.fr
                   |.org                        |

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files target issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

--- Comment #7 from Jean-pierre Cartal <jpcartal@free.fr> ---
Hi,

Is there any hope for this patch to be added to main buildroot release ?

Regards.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 11411] check-uniq-files target issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=11411

Peter Korsgaard <jacmet@uclibc.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #8 from Peter Korsgaard <jacmet@uclibc.org> ---
(In reply to Jean-pierre Cartal from comment #7)

We have instead removed the check-uniq-files logic:

https://git.buildroot.org/buildroot/commit/?id=2496189a4207173e4cd5bbab90256f911175ee57

-- 
You are receiving this mail because:
You are on the CC list for the bug.