[PATCH 0/1] More fixes for a distro with a merged /usr

[PATCH 0/1] More fixes for a distro with a merged /usr

[Ian Ray]

When building a distro with a merged /usr[1] the linux-firmware
recipe causes /lib to be re-created because of hard-coded paths.

Tested by building an image and verifying that rootfs contains a
link from lib to usr/lib as expected.

1. https://wiki.freedesktop.org/www/Software/systemd/TheCaseForTheUsrMerge/

Ian Ray (1):
  linux-firmware: remove hard-coded paths

 .../linux-firmware/linux-firmware_git.bb           | 134 ++++++++++-----------
 1 file changed, 67 insertions(+), 67 deletions(-)

-- 
2.4.5

[PATCH 1/1] linux-firmware: remove hard-coded paths

[Ian Ray]

The recipe uses hard-coded paths (specifically /lib) in do_install
and in FILES, however on a merged /usr system this directory might
not exist. Prefer base_libdir.

Signed-off-by: Ian Ray <ian.ray@ge.com>
---
 .../linux-firmware/linux-firmware_git.bb           | 134 ++++++++++-----------
 1 file changed, 67 insertions(+), 67 deletions(-)

diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
index 0878ab1..4d2cc80 100644
--- a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
+++ b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
@@ -141,24 +141,24 @@ do_compile() {
 }
 
 do_install() {
-	install -d  ${D}/lib/firmware/
-	cp -r * ${D}/lib/firmware/
+	install -d  ${D}${base_libdir}/firmware/
+	cp -r * ${D}${base_libdir}/firmware/
 
 	# Avoid Makefile to be deployed
-	rm ${D}/lib/firmware/Makefile
+	rm ${D}${base_libdir}/firmware/Makefile
 
 	# Remove unbuild firmware which needs cmake and bash
-	rm ${D}/lib/firmware/carl9170fw -rf
+	rm ${D}${base_libdir}/firmware/carl9170fw -rf
 
 	# Remove pointless bash script
-	rm ${D}/lib/firmware/configure
+	rm ${D}${base_libdir}/firmware/configure
 
 	# Libertas sd8686
-	ln -sf libertas/sd8686_v9.bin ${D}/lib/firmware/sd8686.bin
-	ln -sf libertas/sd8686_v9_helper.bin ${D}/lib/firmware/sd8686_helper.bin
+	ln -sf libertas/sd8686_v9.bin ${D}${base_libdir}/firmware/sd8686.bin
+	ln -sf libertas/sd8686_v9_helper.bin ${D}${base_libdir}/firmware/sd8686_helper.bin
 
 	# fixup wl12xx location, after 2.6.37 the kernel searches a different location for it
-	( cd ${D}/lib/firmware ; ln -sf ti-connectivity/* . )
+	( cd ${D}${base_libdir}/firmware ; ln -sf ti-connectivity/* . )
 }
 
 
@@ -188,21 +188,21 @@ LICENSE_${PN}-ar3k = "Firmware-atheros_firmware"
 LICENSE_${PN}-ath6k = "Firmware-atheros_firmware"
 LICENSE_${PN}-ath9k = "Firmware-atheros_firmware"
 
-FILES_${PN}-atheros-license = "/lib/firmware/LICENCE.atheros_firmware"
+FILES_${PN}-atheros-license = "${base_libdir}/firmware/LICENCE.atheros_firmware"
 FILES_${PN}-ar9170 = " \
-  /lib/firmware/ar9170*.fw \
+  ${base_libdir}/firmware/ar9170*.fw \
 "
 FILES_${PN}-ar3k = " \
-  /lib/firmware/ar3k \
+  ${base_libdir}/firmware/ar3k \
 "
 FILES_${PN}-ath6k = " \
-  /lib/firmware/ath6k \
+  ${base_libdir}/firmware/ath6k \
 "
 FILES_${PN}-ath9k = " \
-  /lib/firmware/ar9271.fw \
-  /lib/firmware/ar7010*.fw \
-  /lib/firmware/htc_9271.fw \
-  /lib/firmware/htc_7010.fw \
+  ${base_libdir}/firmware/ar9271.fw \
+  ${base_libdir}/firmware/ar7010*.fw \
+  ${base_libdir}/firmware/htc_9271.fw \
+  ${base_libdir}/firmware/htc_7010.fw \
 "
 
 RDEPENDS_${PN}-ar9170 += "${PN}-atheros-license"
@@ -213,9 +213,9 @@ RDEPENDS_${PN}-ath9k += "${PN}-atheros-license"
 # For ralink
 LICENSE_${PN}-ralink = "Firmware-ralink-firmware"
 
-FILES_${PN}-ralink-license = "/lib/firmware/LICENCE.ralink-firmware.txt"
+FILES_${PN}-ralink-license = "${base_libdir}/firmware/LICENCE.ralink-firmware.txt"
 FILES_${PN}-ralink = " \
-  /lib/firmware/rt*.bin \
+  ${base_libdir}/firmware/rt*.bin \
 "
 
 RDEPENDS_${PN}-ralink += "${PN}-ralink-license"
@@ -223,9 +223,9 @@ RDEPENDS_${PN}-ralink += "${PN}-ralink-license"
 # For radeon
 LICENSE_${PN}-radeon = "Firmware-radeon"
 
-FILES_${PN}-radeon-license = "/lib/firmware/LICENSE.radeon"
+FILES_${PN}-radeon-license = "${base_libdir}/firmware/LICENSE.radeon"
 FILES_${PN}-radeon = " \
-  /lib/firmware/radeon \
+  ${base_libdir}/firmware/radeon \
 "
 
 RDEPENDS_${PN}-radeon += "${PN}-radeon-license"
@@ -235,16 +235,16 @@ LICENSE_${PN}-sd8686 = "Firmware-Marvell"
 LICENSE_${PN}-sd8787 = "Firmware-Marvell"
 LICENSE_${PN}-sd8797 = "Firmware-Marvell"
 
-FILES_${PN}-marvell-license = "/lib/firmware/LICENCE.Marvell"
+FILES_${PN}-marvell-license = "${base_libdir}/firmware/LICENCE.Marvell"
 FILES_${PN}-sd8686 = " \
-  /lib/firmware/libertas/sd8686_v9* \
-  /lib/firmware/sd8686* \
+  ${base_libdir}/firmware/libertas/sd8686_v9* \
+  ${base_libdir}/firmware/sd8686* \
 "
 FILES_${PN}-sd8787 = " \
-  /lib/firmware/mrvl/sd8787_uapsta.bin \
+  ${base_libdir}/firmware/mrvl/sd8787_uapsta.bin \
 "
 FILES_${PN}-sd8797 = " \
-  /lib/firmware/mrvl/sd8797_uapsta.bin \
+  ${base_libdir}/firmware/mrvl/sd8797_uapsta.bin \
 "
 
 RDEPENDS_${PN}-sd8686 += "${PN}-marvell-license"
@@ -257,16 +257,16 @@ LICENSE_${PN}-rtl8192ce = "Firmware-rtlwifi_firmware"
 LICENSE_${PN}-rtl8192su = "Firmware-rtlwifi_firmware"
 
 FILES_${PN}-rtl-license = " \
-  /lib/firmware/LICENCE.rtlwifi_firmware.txt \
+  ${base_libdir}/firmware/LICENCE.rtlwifi_firmware.txt \
 "
 FILES_${PN}-rtl8192cu = " \
-  /lib/firmware/rtlwifi/rtl8192cufw*.bin \
+  ${base_libdir}/firmware/rtlwifi/rtl8192cufw*.bin \
 "
 FILES_${PN}-rtl8192ce = " \
-  /lib/firmware/rtlwifi/rtl8192cfw*.bin \
+  ${base_libdir}/firmware/rtlwifi/rtl8192cfw*.bin \
 "
 FILES_${PN}-rtl8192su = " \
-  /lib/firmware/rtlwifi/rtl8712u.bin \
+  ${base_libdir}/firmware/rtlwifi/rtl8712u.bin \
 "
 
 RDEPENDS_${PN}-rtl8192ce += "${PN}-rtl-license"
@@ -277,16 +277,16 @@ RDEPENDS_${PN}-rtl8192su = "${PN}-rtl-license"
 LICENSE_${PN}-wl12xx = "Firmware-ti-connectivity"
 LICENSE_${PN}-wl18xx = "Firmware-ti-connectivity"
 
-FILES_${PN}-ti-connectivity-license = "/lib/firmware/LICENCE.ti-connectivity"
+FILES_${PN}-ti-connectivity-license = "${base_libdir}/firmware/LICENCE.ti-connectivity"
 FILES_${PN}-wl12xx = " \
-  /lib/firmware/wl12* \
-  /lib/firmware/TI* \
-  /lib/firmware/ti-connectivity \
+  ${base_libdir}/firmware/wl12* \
+  ${base_libdir}/firmware/TI* \
+  ${base_libdir}/firmware/ti-connectivity \
 "
 FILES_${PN}-wl18xx = " \
-  /lib/firmware/wl18* \
-  /lib/firmware/TI* \
-  /lib/firmware/ti-connectivity \
+  ${base_libdir}/firmware/wl18* \
+  ${base_libdir}/firmware/TI* \
+  ${base_libdir}/firmware/ti-connectivity \
 "
 
 RDEPENDS_${PN}-wl12xx = "${PN}-ti-connectivity-license"
@@ -295,9 +295,9 @@ RDEPENDS_${PN}-wl18xx = "${PN}-ti-connectivity-license"
 # For vt6656
 LICENSE_${PN}-vt6656 = "Firmware-via_vt6656"
 
-FILES_${PN}-vt6656-license = "/lib/firmware/LICENCE.via_vt6656"
+FILES_${PN}-vt6656-license = "${base_libdir}/firmware/LICENCE.via_vt6656"
 FILES_${PN}-vt6656 = " \
-  /lib/firmware/vntwusb.fw \
+  ${base_libdir}/firmware/vntwusb.fw \
 "
 
 RDEPENDS_${PN}-vt6656 = "${PN}-vt6656-license"
@@ -314,31 +314,31 @@ LICENSE_${PN}-bcm4334 = "Firmware-broadcom_bcm43xx"
 LICENSE_${PN}-bcm4354 = "Firmware-broadcom_bcm43xx"
 
 FILES_${PN}-broadcom-license = " \
-  /lib/firmware/LICENCE.broadcom_bcm43xx \
+  ${base_libdir}/firmware/LICENCE.broadcom_bcm43xx \
 "
 FILES_${PN}-bcm4329 = " \
-  /lib/firmware/brcm/brcmfmac4329-sdio.bin \
+  ${base_libdir}/firmware/brcm/brcmfmac4329-sdio.bin \
 "
 FILES_${PN}-bcm4330 = " \
-  /lib/firmware/brcm/brcmfmac4330-sdio.bin \
+  ${base_libdir}/firmware/brcm/brcmfmac4330-sdio.bin \
 "
 FILES_${PN}-bcm4334 = " \
-  /lib/firmware/brcm/brcmfmac4334-sdio.bin \
+  ${base_libdir}/firmware/brcm/brcmfmac4334-sdio.bin \
 "
 FILES_${PN}-bcm4354 = " \
-  /lib/firmware/brcm/brcmfmac4354-sdio.bin \
+  ${base_libdir}/firmware/brcm/brcmfmac4354-sdio.bin \
 "
 
-ALTERNATIVE_LINK_NAME[brcmfmac-sdio.bin] = "/lib/firmware/brcm/brcmfmac-sdio.bin"
+ALTERNATIVE_LINK_NAME[brcmfmac-sdio.bin] = "${base_libdir}/firmware/brcm/brcmfmac-sdio.bin"
 
 ALTERNATIVE_linux-firmware-bcm4334 = "brcmfmac-sdio.bin"
-ALTERNATIVE_TARGET_linux-firmware-bcm4334[brcmfmac-sdio.bin] = "/lib/firmware/brcm/brcmfmac4334-sdio.bin"
+ALTERNATIVE_TARGET_linux-firmware-bcm4334[brcmfmac-sdio.bin] = "${base_libdir}/firmware/brcm/brcmfmac4334-sdio.bin"
 ALTERNATIVE_linux_firmware-bcm4354 = "brcmfmac-sdio.bin"
-ALTERNATIVE_TARGET_linux-firmware-bcm4354[brcmfmac-sdio.bin] = "/lib/firmware/brcm/brcmfmac4354-sdio.bin"
+ALTERNATIVE_TARGET_linux-firmware-bcm4354[brcmfmac-sdio.bin] = "${base_libdir}/firmware/brcm/brcmfmac4354-sdio.bin"
 ALTERNATIVE_linux-firmware-bcm4329 = "brcmfmac-sdio.bin"
-ALTERNATIVE_TARGET_linux-firmware-bcm4329[brcmfmac-sdio.bin] = "/lib/firmware/brcm/brcmfmac4329-sdio.bin"
+ALTERNATIVE_TARGET_linux-firmware-bcm4329[brcmfmac-sdio.bin] = "${base_libdir}/firmware/brcm/brcmfmac4329-sdio.bin"
 ALTERNATIVE_linux-firmware-bcm4330 = "brcmfmac-sdio.bin"
-ALTERNATIVE_TARGET_linux-firmware-bcm4330[brcmfmac-sdio.bin] = "/lib/firmware/brcm/brcmfmac4330-sdio.bin"
+ALTERNATIVE_TARGET_linux-firmware-bcm4330[brcmfmac-sdio.bin] = "${base_libdir}/firmware/brcm/brcmfmac4330-sdio.bin"
 
 RDEPENDS_${PN}-bcm4329 += "${PN}-broadcom-license"
 RDEPENDS_${PN}-bcm4330 += "${PN}-broadcom-license"
@@ -363,23 +363,23 @@ LICENSE_${PN}-iwlwifi-7260-9    = "Firmware-iwlwifi_firmware"
 LICENSE_${PN}-iwlwifi-7265-8    = "Firmware-iwlwifi_firmware"
 LICENSE_${PN}-iwlwifi-7265-9    = "Firmware-iwlwifi_firmware"
 
-FILES_${PN}-iwlwifi-license = "/lib/firmware/LICENCE.iwlwifi_firmware"
-FILES_${PN}-iwlwifi-135-6 = "/lib/firmware/iwlwifi-135-6.ucode"
-FILES_${PN}-iwlwifi-3160-7 = "/lib/firmware/iwlwifi-3160-7.ucode"
-FILES_${PN}-iwlwifi-3160-8 = "/lib/firmware/iwlwifi-3160-8.ucode"
-FILES_${PN}-iwlwifi-3160-9 = "/lib/firmware/iwlwifi-3160-9.ucode"
-FILES_${PN}-iwlwifi-6000-4 = "/lib/firmware/iwlwifi-6000-4.ucode"
-FILES_${PN}-iwlwifi-6000g2a-5 = "/lib/firmware/iwlwifi-6000g2a-5.ucode"
-FILES_${PN}-iwlwifi-6000g2a-6 = "/lib/firmware/iwlwifi-6000g2a-6.ucode"
-FILES_${PN}-iwlwifi-6000g2b-5 = "/lib/firmware/iwlwifi-6000g2b-5.ucode"
-FILES_${PN}-iwlwifi-6000g2b-6 = "/lib/firmware/iwlwifi-6000g2b-6.ucode"
-FILES_${PN}-iwlwifi-6050-4 = "/lib/firmware/iwlwifi-6050-4.ucode"
-FILES_${PN}-iwlwifi-6050-5 = "/lib/firmware/iwlwifi-6050-5.ucode"
-FILES_${PN}-iwlwifi-7260-7 = "/lib/firmware/iwlwifi-7260-7.ucode"
-FILES_${PN}-iwlwifi-7260-8 = "/lib/firmware/iwlwifi-7260-8.ucode"
-FILES_${PN}-iwlwifi-7260-9 = "/lib/firmware/iwlwifi-7260-9.ucode"
-FILES_${PN}-iwlwifi-7265-8 = "/lib/firmware/iwlwifi-7265-8.ucode"
-FILES_${PN}-iwlwifi-7265-9 = "/lib/firmware/iwlwifi-7265-9.ucode"
+FILES_${PN}-iwlwifi-license = "${base_libdir}/firmware/LICENCE.iwlwifi_firmware"
+FILES_${PN}-iwlwifi-135-6 = "${base_libdir}/firmware/iwlwifi-135-6.ucode"
+FILES_${PN}-iwlwifi-3160-7 = "${base_libdir}/firmware/iwlwifi-3160-7.ucode"
+FILES_${PN}-iwlwifi-3160-8 = "${base_libdir}/firmware/iwlwifi-3160-8.ucode"
+FILES_${PN}-iwlwifi-3160-9 = "${base_libdir}/firmware/iwlwifi-3160-9.ucode"
+FILES_${PN}-iwlwifi-6000-4 = "${base_libdir}/firmware/iwlwifi-6000-4.ucode"
+FILES_${PN}-iwlwifi-6000g2a-5 = "${base_libdir}/firmware/iwlwifi-6000g2a-5.ucode"
+FILES_${PN}-iwlwifi-6000g2a-6 = "${base_libdir}/firmware/iwlwifi-6000g2a-6.ucode"
+FILES_${PN}-iwlwifi-6000g2b-5 = "${base_libdir}/firmware/iwlwifi-6000g2b-5.ucode"
+FILES_${PN}-iwlwifi-6000g2b-6 = "${base_libdir}/firmware/iwlwifi-6000g2b-6.ucode"
+FILES_${PN}-iwlwifi-6050-4 = "${base_libdir}/firmware/iwlwifi-6050-4.ucode"
+FILES_${PN}-iwlwifi-6050-5 = "${base_libdir}/firmware/iwlwifi-6050-5.ucode"
+FILES_${PN}-iwlwifi-7260-7 = "${base_libdir}/firmware/iwlwifi-7260-7.ucode"
+FILES_${PN}-iwlwifi-7260-8 = "${base_libdir}/firmware/iwlwifi-7260-8.ucode"
+FILES_${PN}-iwlwifi-7260-9 = "${base_libdir}/firmware/iwlwifi-7260-9.ucode"
+FILES_${PN}-iwlwifi-7265-8 = "${base_libdir}/firmware/iwlwifi-7265-8.ucode"
+FILES_${PN}-iwlwifi-7265-9 = "${base_libdir}/firmware/iwlwifi-7265-9.ucode"
 
 RDEPENDS_${PN}-iwlwifi-135-6     = "${PN}-iwlwifi-license"
 RDEPENDS_${PN}-iwlwifi-3160-7    = "${PN}-iwlwifi-license"
@@ -429,8 +429,8 @@ LICENSE_${PN} = "\
     & Firmware-xc5000 \
     & Firmware-xc5000c \
 "
-FILES_${PN}-license += "/lib/firmware/LICEN*"
-FILES_${PN} += "/lib/firmware/*"
+FILES_${PN}-license += "${base_libdir}/firmware/LICEN*"
+FILES_${PN} += "${base_libdir}/firmware/*"
 RDEPENDS_${PN} += "${PN}-license"
 
 # Make linux-firmware depend on all of the split-out packages.
-- 
2.4.5

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Matthias Schiffer]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

On 01/04/2016 02:14 PM, Ian Ray wrote:
> The recipe uses hard-coded paths (specifically /lib) in do_install
> and in FILES, however on a merged /usr system this directory might
> not exist. Prefer base_libdir.
> 
> Signed-off-by: Ian Ray <ian.ray@ge.com>
> ---

This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
ppc64, which is not where the firmware is expected.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Mark Hatle]

On 1/4/16 10:11 AM, Matthias Schiffer wrote:
> On 01/04/2016 02:14 PM, Ian Ray wrote:
>> The recipe uses hard-coded paths (specifically /lib) in do_install
>> and in FILES, however on a merged /usr system this directory might
>> not exist. Prefer base_libdir.
>>
>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>> ---
> 
> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
> ppc64, which is not where the firmware is expected.
> 

At a minimum, I would agree nonarch_base_libdir, however..

I believe that the kernel loader/modules/tools themselves actually have '/lib'
hard coded into them.  This is the reason why /lib/firmware was used and not one
of the variables.

This is one of the cases were /lib is actually correct, since that is what the
system is expecting.  We can make some kind of accommodation for systems where
/lib -> /usr/lib... but that should be done inside of the filesystem setup
processing and not the package itself.  (I'm referring to the
'meta/files/fs-perms.txt' file.

--Mark

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Matthias Schiffer]

[-- Attachment #1: Type: text/plain, Size: 1967 bytes --]

On 01/04/2016 05:32 PM, Mark Hatle wrote:
> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>> and in FILES, however on a merged /usr system this directory might
>>> not exist. Prefer base_libdir.
>>>
>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>> ---
>>
>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>> ppc64, which is not where the firmware is expected.
>>
> 
> At a minimum, I would agree nonarch_base_libdir, however..
> 
> I believe that the kernel loader/modules/tools themselves actually have '/lib'
> hard coded into them.  This is the reason why /lib/firmware was used and not one
> of the variables.
> 
> This is one of the cases were /lib is actually correct, since that is what the
> system is expecting.  We can make some kind of accommodation for systems where
> /lib -> /usr/lib... but that should be done inside of the filesystem setup
> processing and not the package itself.  (I'm referring to the
> 'meta/files/fs-perms.txt' file.
> 
> --Mark
> 

There seem to be some intresting ideas going around about what can or
should be done via fs-perms.txt... AFAICT, fs-perms.txt can't move
around files, so moving files form /lib to /usr/lib must be done in the
package recipes themselves. (In my opinion, fs-perms.txt is a bad hack
for broken recipes that shouldn't exist anyways, but that's another
discussion)

I think if a distro config changes any of the base paths
({nonarch_,}base_libdir, base_{,s}bindir), *all* packages should respect
this. It's the distro's reponsiblity to create symlinks so everything is
found again at the expected paths (other examples for such hardcoded
paths: /bin/sh; the dynamic linker). See also my patchset I submitted to
this mailing list, which introduces a distro feature to have such
symlinks created by base-files.

Matthias


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Mark Hatle]

On 1/4/16 4:26 PM, Matthias Schiffer wrote:
> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>> and in FILES, however on a merged /usr system this directory might
>>>> not exist. Prefer base_libdir.
>>>>
>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>> ---
>>>
>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>> ppc64, which is not where the firmware is expected.
>>>
>>
>> At a minimum, I would agree nonarch_base_libdir, however..
>>
>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>> of the variables.
>>
>> This is one of the cases were /lib is actually correct, since that is what the
>> system is expecting.  We can make some kind of accommodation for systems where
>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>> processing and not the package itself.  (I'm referring to the
>> 'meta/files/fs-perms.txt' file.
>>
>> --Mark
>>
> 
> There seem to be some intresting ideas going around about what can or
> should be done via fs-perms.txt... AFAICT, fs-perms.txt can't move
> around files, so moving files form /lib to /usr/lib must be done in the
> package recipes themselves. (In my opinion, fs-perms.txt is a bad hack
> for broken recipes that shouldn't exist anyways, but that's another
> discussion)

Since I wrote fs-perms.txt, I'll explain the purpose.  Individual packages don't
know if something is a directory, symbolic link, or what owner/group/permissions
a system level directory should be set to.

The entire purpose of it is to declare a common set of -system- directories.
(Packages/layers can amend and override this as necessary to add their own
system directories.)

FYI System directories are things like /usr/bin.  Having every package in the
system need to define /usr/bin as a directory with an owner/group of root:root
and permission of 0755 is a REALLY bad practice.. but putting this knowledge
into a single file that synchronizes everything is very practical.

When the system level directories are mapped to symlinks.. the case where
everyone is trying to folks /usr -> / or /bin -> /usr/bin.. then it can
AUTOMATICALLY map and move the files in these places..

> I think if a distro config changes any of the base paths
> ({nonarch_,}base_libdir, base_{,s}bindir), *all* packages should respect
> this. It's the distro's reponsiblity to create symlinks so everything is
> found again at the expected paths (other examples for such hardcoded
> paths: /bin/sh; the dynamic linker). See also my patchset I submitted to
> this mailing list, which introduces a distro feature to have such
> symlinks created by base-files.

When this was written it was heavily argued against this knowledge being in
base-files or base-dirs (suggested at the time) packages.

Defining a base setup, and then using a synchronization pass using the
fs-perms.txt was the way to go.

Note, fs-perms process is absolutely supposed to move files around -if- a
symlink is generated.. i.e.:

/lib -> /usr/lib

if you write to /lib/firmware, the code is supposed to see the directory of
'/lib', create a new /usr/lib (set perms properly) and move the contents if /lib
to /usr/lib, then replace the directory with a symbolic link.

If it's NOT doing that, lets fix it.

--Mark

> Matthias
> 
> 
> 

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Matthias Schiffer]

[-- Attachment #1: Type: text/plain, Size: 4381 bytes --]

On 01/04/2016 11:56 PM, Mark Hatle wrote:
> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>> and in FILES, however on a merged /usr system this directory might
>>>>> not exist. Prefer base_libdir.
>>>>>
>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>> ---
>>>>
>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>> ppc64, which is not where the firmware is expected.
>>>>
>>>
>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>
>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>>> of the variables.
>>>
>>> This is one of the cases were /lib is actually correct, since that is what the
>>> system is expecting.  We can make some kind of accommodation for systems where
>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>> processing and not the package itself.  (I'm referring to the
>>> 'meta/files/fs-perms.txt' file.
>>>
>>> --Mark
>>>
>>
>> There seem to be some intresting ideas going around about what can or
>> should be done via fs-perms.txt... AFAICT, fs-perms.txt can't move
>> around files, so moving files form /lib to /usr/lib must be done in the
>> package recipes themselves. (In my opinion, fs-perms.txt is a bad hack
>> for broken recipes that shouldn't exist anyways, but that's another
>> discussion)
> 
> Since I wrote fs-perms.txt, I'll explain the purpose.  Individual packages don't
> know if something is a directory, symbolic link, or what owner/group/permissions
> a system level directory should be set to.
> 
> The entire purpose of it is to declare a common set of -system- directories.
> (Packages/layers can amend and override this as necessary to add their own
> system directories.)
> 
> FYI System directories are things like /usr/bin.  Having every package in the
> system need to define /usr/bin as a directory with an owner/group of root:root
> and permission of 0755 is a REALLY bad practice.. but putting this knowledge
> into a single file that synchronizes everything is very practical.
> 
> When the system level directories are mapped to symlinks.. the case where
> everyone is trying to folks /usr -> / or /bin -> /usr/bin.. then it can
> AUTOMATICALLY map and move the files in these places..

Thanks for the explanation, I asked a similar question a few weeks ago
and didn't get a satisfactory answer about what fs-perms can do. I'll
rethink my approach for the merged-usr patchset.

So the remaining issue is how to conditionalize this. I'd like to find a
solution which doesn't require distros to define their own fs-perms when
they just want to use a merged /usr dir.

> 
>> I think if a distro config changes any of the base paths
>> ({nonarch_,}base_libdir, base_{,s}bindir), *all* packages should respect
>> this. It's the distro's reponsiblity to create symlinks so everything is
>> found again at the expected paths (other examples for such hardcoded
>> paths: /bin/sh; the dynamic linker). See also my patchset I submitted to
>> this mailing list, which introduces a distro feature to have such
>> symlinks created by base-files.
> 
> When this was written it was heavily argued against this knowledge being in
> base-files or base-dirs (suggested at the time) packages.

Is that discussion archived somewhere? I'm interested in the
argumentation. Do any non-OE distros have a similar feature?

> 
> Defining a base setup, and then using a synchronization pass using the
> fs-perms.txt was the way to go.
> 
> Note, fs-perms process is absolutely supposed to move files around -if- a
> symlink is generated.. i.e.:
> 
> /lib -> /usr/lib
> 
> if you write to /lib/firmware, the code is supposed to see the directory of
> '/lib', create a new /usr/lib (set perms properly) and move the contents if /lib
> to /usr/lib, then replace the directory with a symbolic link.
> 
> If it's NOT doing that, lets fix it.

I didn't try yet as I didn't now that it is supposed to to that.

> 
> --Mark
> 
>> Matthias
>>
>>
>>
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Mark Hatle]

On 1/4/16 5:57 PM, Matthias Schiffer wrote:
> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>> not exist. Prefer base_libdir.
>>>>>>
>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>> ---
>>>>>
>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>> ppc64, which is not where the firmware is expected.
>>>>>
>>>>
>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>
>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>>>> of the variables.
>>>>
>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>> system is expecting.  We can make some kind of accommodation for systems where
>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>> processing and not the package itself.  (I'm referring to the
>>>> 'meta/files/fs-perms.txt' file.
>>>>
>>>> --Mark
>>>>
>>>
>>> There seem to be some intresting ideas going around about what can or
>>> should be done via fs-perms.txt... AFAICT, fs-perms.txt can't move
>>> around files, so moving files form /lib to /usr/lib must be done in the
>>> package recipes themselves. (In my opinion, fs-perms.txt is a bad hack
>>> for broken recipes that shouldn't exist anyways, but that's another
>>> discussion)
>>
>> Since I wrote fs-perms.txt, I'll explain the purpose.  Individual packages don't
>> know if something is a directory, symbolic link, or what owner/group/permissions
>> a system level directory should be set to.
>>
>> The entire purpose of it is to declare a common set of -system- directories.
>> (Packages/layers can amend and override this as necessary to add their own
>> system directories.)
>>
>> FYI System directories are things like /usr/bin.  Having every package in the
>> system need to define /usr/bin as a directory with an owner/group of root:root
>> and permission of 0755 is a REALLY bad practice.. but putting this knowledge
>> into a single file that synchronizes everything is very practical.
>>
>> When the system level directories are mapped to symlinks.. the case where
>> everyone is trying to folks /usr -> / or /bin -> /usr/bin.. then it can
>> AUTOMATICALLY map and move the files in these places..
> 
> Thanks for the explanation, I asked a similar question a few weeks ago
> and didn't get a satisfactory answer about what fs-perms can do. I'll
> rethink my approach for the merged-usr patchset.

(I've been out since near the beginning of December, but I'm back now..)

> So the remaining issue is how to conditionalize this. I'd like to find a
> solution which doesn't require distros to define their own fs-perms when
> they just want to use a merged /usr dir.

You conditionalize it by providing different default fs-perms files -- OR since
more then one file can be loaded -- additional fs-perms with the permutations
you desire.

I could easily see having an files/fs-perms-usr-only.txt and an
files/fs-perms-no-usr.txt

Where the first would be something like:

# Define symlinks from base directories to their prefix versions
/bin	link ${bindir}
/sbin	link ${sbindir}
...

fs-perms-no-user would be:

# Define that $prefix as simply pointing back to root for compatibility
/usr	link /


Then in your distro.conf file:

FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt"
FILESYSTEM_PERMS_TABLES_append_usronly = " files/fs-perms-usr-only.txt"
FILESYSTEM_PERMS_TABLES_append_nousr = " files/fs-perms-no-user.txt"


The above will cause packages to produce the symlinks ONLY if the package itself
creates a matching directory.  If the package is moving all of it's files to the
new configured location (from the distro.conf file) then no link is present.

Assuming you want the links, I would add some kind of a change or configuration
to the base-files to do this.

(I did argue in the past, unsuccessfully, that base-files or something it calls
should process the fs-perms files and simply generate a base configuration based
on this setup.  Ensuring symlinks and final directories were always created..
but that was rejected at the time..)

>>
>>> I think if a distro config changes any of the base paths
>>> ({nonarch_,}base_libdir, base_{,s}bindir), *all* packages should respect
>>> this. It's the distro's reponsiblity to create symlinks so everything is
>>> found again at the expected paths (other examples for such hardcoded
>>> paths: /bin/sh; the dynamic linker). See also my patchset I submitted to
>>> this mailing list, which introduces a distro feature to have such
>>> symlinks created by base-files.
>>
>> When this was written it was heavily argued against this knowledge being in
>> base-files or base-dirs (suggested at the time) packages.
> 
> Is that discussion archived somewhere? I'm interested in the
> argumentation. Do any non-OE distros have a similar feature?

This would have been discussed back in the original oe-core days.  Likely around
2010 on the oe-core list....

>>
>> Defining a base setup, and then using a synchronization pass using the
>> fs-perms.txt was the way to go.
>>
>> Note, fs-perms process is absolutely supposed to move files around -if- a
>> symlink is generated.. i.e.:
>>
>> /lib -> /usr/lib
>>
>> if you write to /lib/firmware, the code is supposed to see the directory of
>> '/lib', create a new /usr/lib (set perms properly) and move the contents if /lib
>> to /usr/lib, then replace the directory with a symbolic link.
>>
>> If it's NOT doing that, lets fix it.
> 
> I didn't try yet as I didn't now that it is supposed to to that.

This is a good example where the upstream software is always going to use
'/lib', but you want it to go somewhere else.  Without this fs-perms mechanism,
you would need to patch this package and potentially others.  But by using
fs-perms, they can continue to rely on their special knowledge of '/lib', and
the system will fix it up before packaging to where the distribution actually
wants the files.

--Mark

>>
>> --Mark
>>
>>> Matthias
>>>
>>>
>>>
>>
> 
> 

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Matthias Schiffer]

[-- Attachment #1: Type: text/plain, Size: 7607 bytes --]

On 01/05/2016 01:28 AM, Mark Hatle wrote:
> On 1/4/16 5:57 PM, Matthias Schiffer wrote:
>> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>>> not exist. Prefer base_libdir.
>>>>>>>
>>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>>> ---
>>>>>>
>>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>>> ppc64, which is not where the firmware is expected.
>>>>>>
>>>>>
>>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>>
>>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>>>>> of the variables.
>>>>>
>>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>>> system is expecting.  We can make some kind of accommodation for systems where
>>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>>> processing and not the package itself.  (I'm referring to the
>>>>> 'meta/files/fs-perms.txt' file.
>>>>>
>>>>> --Mark
>>>>>
>>>>
>>>> There seem to be some intresting ideas going around about what can or
>>>> should be done via fs-perms.txt... AFAICT, fs-perms.txt can't move
>>>> around files, so moving files form /lib to /usr/lib must be done in the
>>>> package recipes themselves. (In my opinion, fs-perms.txt is a bad hack
>>>> for broken recipes that shouldn't exist anyways, but that's another
>>>> discussion)
>>>
>>> Since I wrote fs-perms.txt, I'll explain the purpose.  Individual packages don't
>>> know if something is a directory, symbolic link, or what owner/group/permissions
>>> a system level directory should be set to.
>>>
>>> The entire purpose of it is to declare a common set of -system- directories.
>>> (Packages/layers can amend and override this as necessary to add their own
>>> system directories.)
>>>
>>> FYI System directories are things like /usr/bin.  Having every package in the
>>> system need to define /usr/bin as a directory with an owner/group of root:root
>>> and permission of 0755 is a REALLY bad practice.. but putting this knowledge
>>> into a single file that synchronizes everything is very practical.
>>>
>>> When the system level directories are mapped to symlinks.. the case where
>>> everyone is trying to folks /usr -> / or /bin -> /usr/bin.. then it can
>>> AUTOMATICALLY map and move the files in these places..
>>
>> Thanks for the explanation, I asked a similar question a few weeks ago
>> and didn't get a satisfactory answer about what fs-perms can do. I'll
>> rethink my approach for the merged-usr patchset.
> 
> (I've been out since near the beginning of December, but I'm back now..)
> 
>> So the remaining issue is how to conditionalize this. I'd like to find a
>> solution which doesn't require distros to define their own fs-perms when
>> they just want to use a merged /usr dir.
> 
> You conditionalize it by providing different default fs-perms files -- OR since
> more then one file can be loaded -- additional fs-perms with the permutations
> you desire.
> 
> I could easily see having an files/fs-perms-usr-only.txt and an
> files/fs-perms-no-usr.txt
> 
> Where the first would be something like:
> 
> # Define symlinks from base directories to their prefix versions
> /bin	link ${bindir}
> /sbin	link ${sbindir}
> ...
> 
> fs-perms-no-user would be:
> 
> # Define that $prefix as simply pointing back to root for compatibility
> /usr	link /
> 
> 
> Then in your distro.conf file:
> 
> FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt"
> FILESYSTEM_PERMS_TABLES_append_usronly = " files/fs-perms-usr-only.txt"
> FILESYSTEM_PERMS_TABLES_append_nousr = " files/fs-perms-no-user.txt"
> 
> 
> The above will cause packages to produce the symlinks ONLY if the package itself
> creates a matching directory.  If the package is moving all of it's files to the
> new configured location (from the distro.conf file) then no link is present.
> 
> Assuming you want the links, I would add some kind of a change or configuration
> to the base-files to do this.
> 
> (I did argue in the past, unsuccessfully, that base-files or something it calls
> should process the fs-perms files and simply generate a base configuration based
> on this setup.  Ensuring symlinks and final directories were always created..
> but that was rejected at the time..)

I see. Do you think it would make sense to provide a fs-perms snippet in
oe-core for the Fedora-style merged-usr setup?

> 
>>>
>>>> I think if a distro config changes any of the base paths
>>>> ({nonarch_,}base_libdir, base_{,s}bindir), *all* packages should respect
>>>> this. It's the distro's reponsiblity to create symlinks so everything is
>>>> found again at the expected paths (other examples for such hardcoded
>>>> paths: /bin/sh; the dynamic linker). See also my patchset I submitted to
>>>> this mailing list, which introduces a distro feature to have such
>>>> symlinks created by base-files.
>>>
>>> When this was written it was heavily argued against this knowledge being in
>>> base-files or base-dirs (suggested at the time) packages.
>>
>> Is that discussion archived somewhere? I'm interested in the
>> argumentation. Do any non-OE distros have a similar feature?
> 
> This would have been discussed back in the original oe-core days.  Likely around
> 2010 on the oe-core list....

Okay, I've found at least some fragments of the discussion.

> 
>>>
>>> Defining a base setup, and then using a synchronization pass using the
>>> fs-perms.txt was the way to go.
>>>
>>> Note, fs-perms process is absolutely supposed to move files around -if- a
>>> symlink is generated.. i.e.:
>>>
>>> /lib -> /usr/lib
>>>
>>> if you write to /lib/firmware, the code is supposed to see the directory of
>>> '/lib', create a new /usr/lib (set perms properly) and move the contents if /lib
>>> to /usr/lib, then replace the directory with a symbolic link.
>>>
>>> If it's NOT doing that, lets fix it.
>>
>> I didn't try yet as I didn't now that it is supposed to to that.
> 
> This is a good example where the upstream software is always going to use
> '/lib', but you want it to go somewhere else.  Without this fs-perms mechanism,
> you would need to patch this package and potentially others.  But by using
> fs-perms, they can continue to rely on their special knowledge of '/lib', and
> the system will fix it up before packaging to where the distribution actually
> wants the files.
> 

I guess I'm fine with fs-perms.txt fixing permissions and owners, but
moving around files goes a bit too far in my opinion. Doing so will
potentially break relative symlinks and other relative paths used by
packages. I'd have implemented the "link" feature as a QA-only thing:
make the build fail if there are any files where a symlink is supposed
to go, but don't try to fix it up.

Another (more easily fixable) issue is that the automagic renaming
doesn't work if the target dir already exists (if I understand the code
correctly). So fixing up a package containing both files in /bin and
/usr/bin, where /bin is supposed to link to /usr/bin, will fail.

Matthias



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Mark Hatle]

On 1/4/16 7:26 PM, Matthias Schiffer wrote:
> On 01/05/2016 01:28 AM, Mark Hatle wrote:
>> On 1/4/16 5:57 PM, Matthias Schiffer wrote:
>>> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>>>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>>>> not exist. Prefer base_libdir.
>>>>>>>>
>>>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>>>> ---
>>>>>>>
>>>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>>>> ppc64, which is not where the firmware is expected.
>>>>>>>
>>>>>>
>>>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>>>
>>>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>>>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>>>>>> of the variables.
>>>>>>
>>>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>>>> system is expecting.  We can make some kind of accommodation for systems where
>>>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>>>> processing and not the package itself.  (I'm referring to the
>>>>>> 'meta/files/fs-perms.txt' file.
>>>>>>
>>>>>> --Mark
>>>>>>
>>>>>
>>>>> There seem to be some intresting ideas going around about what can or
>>>>> should be done via fs-perms.txt... AFAICT, fs-perms.txt can't move
>>>>> around files, so moving files form /lib to /usr/lib must be done in the
>>>>> package recipes themselves. (In my opinion, fs-perms.txt is a bad hack
>>>>> for broken recipes that shouldn't exist anyways, but that's another
>>>>> discussion)
>>>>
>>>> Since I wrote fs-perms.txt, I'll explain the purpose.  Individual packages don't
>>>> know if something is a directory, symbolic link, or what owner/group/permissions
>>>> a system level directory should be set to.
>>>>
>>>> The entire purpose of it is to declare a common set of -system- directories.
>>>> (Packages/layers can amend and override this as necessary to add their own
>>>> system directories.)
>>>>
>>>> FYI System directories are things like /usr/bin.  Having every package in the
>>>> system need to define /usr/bin as a directory with an owner/group of root:root
>>>> and permission of 0755 is a REALLY bad practice.. but putting this knowledge
>>>> into a single file that synchronizes everything is very practical.
>>>>
>>>> When the system level directories are mapped to symlinks.. the case where
>>>> everyone is trying to folks /usr -> / or /bin -> /usr/bin.. then it can
>>>> AUTOMATICALLY map and move the files in these places..
>>>
>>> Thanks for the explanation, I asked a similar question a few weeks ago
>>> and didn't get a satisfactory answer about what fs-perms can do. I'll
>>> rethink my approach for the merged-usr patchset.
>>
>> (I've been out since near the beginning of December, but I'm back now..)
>>
>>> So the remaining issue is how to conditionalize this. I'd like to find a
>>> solution which doesn't require distros to define their own fs-perms when
>>> they just want to use a merged /usr dir.
>>
>> You conditionalize it by providing different default fs-perms files -- OR since
>> more then one file can be loaded -- additional fs-perms with the permutations
>> you desire.
>>
>> I could easily see having an files/fs-perms-usr-only.txt and an
>> files/fs-perms-no-usr.txt
>>
>> Where the first would be something like:
>>
>> # Define symlinks from base directories to their prefix versions
>> /bin	link ${bindir}
>> /sbin	link ${sbindir}
>> ...
>>
>> fs-perms-no-user would be:
>>
>> # Define that $prefix as simply pointing back to root for compatibility
>> /usr	link /
>>
>>
>> Then in your distro.conf file:
>>
>> FILESYSTEM_PERMS_TABLES = "files/fs-perms.txt"
>> FILESYSTEM_PERMS_TABLES_append_usronly = " files/fs-perms-usr-only.txt"
>> FILESYSTEM_PERMS_TABLES_append_nousr = " files/fs-perms-no-user.txt"
>>
>>
>> The above will cause packages to produce the symlinks ONLY if the package itself
>> creates a matching directory.  If the package is moving all of it's files to the
>> new configured location (from the distro.conf file) then no link is present.
>>
>> Assuming you want the links, I would add some kind of a change or configuration
>> to the base-files to do this.
>>
>> (I did argue in the past, unsuccessfully, that base-files or something it calls
>> should process the fs-perms files and simply generate a base configuration based
>> on this setup.  Ensuring symlinks and final directories were always created..
>> but that was rejected at the time..)
> 
> I see. Do you think it would make sense to provide a fs-perms snippet in
> oe-core for the Fedora-style merged-usr setup?

This has always been a question, and I leave the answer more up to the oe-core
community as a whole.

Right now oe-core has a basic distribution configuration, and it's expected that
developers will override this configuration with their own requirements.  (I.e.
the Yocto Project does this by providing the poky configuration.)

If oe-core provides multiple configurations, then we'll need a test plan and
resources to test them (not necessarily a bad thing, but there are ramifications
outside of the code base when making these kinds of changes.)

The immediate alternative is create a distribution layer that implements this
work.. it can be used to show everything is functional or highlight areas where
there are bugs in oe-core so they can be fixed.  (Supporting this workflow has
always been intended, so I'm fully behind finding and fixing bugs with this kind
of filesystem merge.)

>>
>>>>
>>>>> I think if a distro config changes any of the base paths
>>>>> ({nonarch_,}base_libdir, base_{,s}bindir), *all* packages should respect
>>>>> this. It's the distro's reponsiblity to create symlinks so everything is
>>>>> found again at the expected paths (other examples for such hardcoded
>>>>> paths: /bin/sh; the dynamic linker). See also my patchset I submitted to
>>>>> this mailing list, which introduces a distro feature to have such
>>>>> symlinks created by base-files.
>>>>
>>>> When this was written it was heavily argued against this knowledge being in
>>>> base-files or base-dirs (suggested at the time) packages.
>>>
>>> Is that discussion archived somewhere? I'm interested in the
>>> argumentation. Do any non-OE distros have a similar feature?
>>
>> This would have been discussed back in the original oe-core days.  Likely around
>> 2010 on the oe-core list....
> 
> Okay, I've found at least some fragments of the discussion.
> 
>>
>>>>
>>>> Defining a base setup, and then using a synchronization pass using the
>>>> fs-perms.txt was the way to go.
>>>>
>>>> Note, fs-perms process is absolutely supposed to move files around -if- a
>>>> symlink is generated.. i.e.:
>>>>
>>>> /lib -> /usr/lib
>>>>
>>>> if you write to /lib/firmware, the code is supposed to see the directory of
>>>> '/lib', create a new /usr/lib (set perms properly) and move the contents if /lib
>>>> to /usr/lib, then replace the directory with a symbolic link.
>>>>
>>>> If it's NOT doing that, lets fix it.
>>>
>>> I didn't try yet as I didn't now that it is supposed to to that.
>>
>> This is a good example where the upstream software is always going to use
>> '/lib', but you want it to go somewhere else.  Without this fs-perms mechanism,
>> you would need to patch this package and potentially others.  But by using
>> fs-perms, they can continue to rely on their special knowledge of '/lib', and
>> the system will fix it up before packaging to where the distribution actually
>> wants the files.
>>
> 
> I guess I'm fine with fs-perms.txt fixing permissions and owners, but
> moving around files goes a bit too far in my opinion. Doing so will
> potentially break relative symlinks and other relative paths used by
> packages. I'd have implemented the "link" feature as a QA-only thing:
> make the build fail if there are any files where a symlink is supposed
> to go, but don't try to fix it up.
> 
> Another (more easily fixable) issue is that the automagic renaming
> doesn't work if the target dir already exists (if I understand the code
> correctly). So fixing up a package containing both files in /bin and
> /usr/bin, where /bin is supposed to link to /usr/bin, will fail.

I would consider this a bug we should fix in the code.  The only failure should
be if moving the contents results in two non-identical files clashing.

--Mark

> Matthias
> 
> 

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Ray, Ian (GE Healthcare)]

On 01/05/2016 03:26 AM, Matthias Schiffer wrote:
> On 01/05/2016 01:28 AM, Mark Hatle wrote:
>> On 1/4/16 5:57 PM, Matthias Schiffer wrote:
>>> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>>>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>>>> not exist. Prefer base_libdir.
>>>>>>>>
>>>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>>>> ---
>>>>>>>
>>>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>>>> ppc64, which is not where the firmware is expected.
>>>>>>>
>>>>>>
>>>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>>>
>>>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>>>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>>>>>> of the variables.
>>>>>>
>>>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>>>> system is expecting.  We can make some kind of accommodation for systems where
>>>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>>>> processing and not the package itself.  (I'm referring to the
>>>>>> 'meta/files/fs-perms.txt' file.
>>>>>>
>>>>>> --Mark

Ah, that makes sense.

<snip>

>> You conditionalize it by providing different default fs-perms files -- OR since
>> more then one file can be loaded -- additional fs-perms with the permutations
>> you desire.
>>
>> I could easily see having an files/fs-perms-usr-only.txt and an
>> files/fs-perms-no-usr.txt
>>
>> Where the first would be something like:
>>
>> # Define symlinks from base directories to their prefix versions
>> /bin	link ${bindir}
>> /sbin	link ${sbindir}
>> ...

This is very interesting -- thank you.

I tried patching our additional fs-perms file as Mark suggested. This resulted in
a build failure during packaging of a recipe. The package directory looks odd:

ls -l /home/ian/myproduct/build/tmp/work/myproduct-linux-gnueabi/
myrecipe/4.3-r0/package
total 4
drwxr-xr-x. 2 ian ian 4096 Jan  5 11:30 boot
drwxr-xr-x. 4 ian ian   56 Jan  5 11:30 etc
lrwxrwxrwx. 1 ian ian    8 Jan  5 11:33 lib -> /usr/lib
drwxr-xr-x. 3 ian ian   16 Jan  5 11:33 usr

This can be fixed by teaching classes/package.bbclass to _not_ use the leading
slash when making the link. Not sure if that is unacceptably hacky? But it must
be better than leaking to host filesystem.

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index a86b680..59ef447 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -774,6 +774,7 @@ python fixup_perms () {
         link = fs_perms_table[dir].link
         if link[0] == "/":
             target = dvar + link
+            link = link[1:]
             ptarget = link
         else:
             target = os.path.join(os.path.dirname(origin), link)

Once past that problem there is a QA problem since do_package now installs
to /usr/lib (because of fs-perms) instead of /lib which is mentioned in FILES.

Joshua's patches for merged /usr touched FILES in _some_ cases (for example
73a6fe958f47642d18ba0098cfd45c3520d53560) but linux-firmware would require
considerably more effort. Is that an argument in favour of nonarch_base_libdir
instead of literal "/lib" or is there some other technique to address this?

Ian

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Mark Hatle]

On 1/5/16 8:04 AM, Ray, Ian (GE Healthcare) wrote:
> On 01/05/2016 03:26 AM, Matthias Schiffer wrote:
>> On 01/05/2016 01:28 AM, Mark Hatle wrote:
>>> On 1/4/16 5:57 PM, Matthias Schiffer wrote:
>>>> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>>>>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>>>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>>>>> not exist. Prefer base_libdir.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>>>>> ---
>>>>>>>>
>>>>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>>>>> ppc64, which is not where the firmware is expected.
>>>>>>>>
>>>>>>>
>>>>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>>>>
>>>>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>>>>> hard coded into them.  This is the reason why /lib/firmware was used and not one
>>>>>>> of the variables.
>>>>>>>
>>>>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>>>>> system is expecting.  We can make some kind of accommodation for systems where
>>>>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>>>>> processing and not the package itself.  (I'm referring to the
>>>>>>> 'meta/files/fs-perms.txt' file.
>>>>>>>
>>>>>>> --Mark
> 
> Ah, that makes sense.
> 
> <snip>
> 
>>> You conditionalize it by providing different default fs-perms files -- OR since
>>> more then one file can be loaded -- additional fs-perms with the permutations
>>> you desire.
>>>
>>> I could easily see having an files/fs-perms-usr-only.txt and an
>>> files/fs-perms-no-usr.txt
>>>
>>> Where the first would be something like:
>>>
>>> # Define symlinks from base directories to their prefix versions
>>> /bin	link ${bindir}
>>> /sbin	link ${sbindir}
>>> ...
> 
> This is very interesting -- thank you.
> 
> I tried patching our additional fs-perms file as Mark suggested. This resulted in
> a build failure during packaging of a recipe. The package directory looks odd:
> 
> ls -l /home/ian/myproduct/build/tmp/work/myproduct-linux-gnueabi/
> myrecipe/4.3-r0/package
> total 4
> drwxr-xr-x. 2 ian ian 4096 Jan  5 11:30 boot
> drwxr-xr-x. 4 ian ian   56 Jan  5 11:30 etc
> lrwxrwxrwx. 1 ian ian    8 Jan  5 11:33 lib -> /usr/lib
> drwxr-xr-x. 3 ian ian   16 Jan  5 11:33 usr

We should look into this further.  Pseudo (the fakeroot wrapper) should be
changing '/usr/lib' [in the cross build filesystem] to a relative path... It
will only be '/usr/lib' in the target version if we're in the pseudo environment
or the translation is off.  (Maybe I'm remembering incorrectly, but that was my
memory.)

> This can be fixed by teaching classes/package.bbclass to _not_ use the leading
> slash when making the link. Not sure if that is unacceptably hacky? But it must
> be better than leaking to host filesystem.
> 
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index a86b680..59ef447 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -774,6 +774,7 @@ python fixup_perms () {
>          link = fs_perms_table[dir].link
>          if link[0] == "/":
>              target = dvar + link
> +            link = link[1:]
>              ptarget = link
>          else:
>              target = os.path.join(os.path.dirname(origin), link)

I'm definitely interested in understanding what may or may not be working
properly here.

> Once past that problem there is a QA problem since do_package now installs
> to /usr/lib (because of fs-perms) instead of /lib which is mentioned in FILES.

What was the QA issue?  About the absolute path name or something else?

> Joshua's patches for merged /usr touched FILES in _some_ cases (for example
> 73a6fe958f47642d18ba0098cfd45c3520d53560) but linux-firmware would require
> considerably more effort. Is that an argument in favour of nonarch_base_libdir
> instead of literal "/lib" or is there some other technique to address this?

For linux-firmware specifically.. we need to make sure the kernel and modules
(which use the firmware) use the same filesystem notion as what the
linux-firmware package is doing.  In the past when I've looked, pretty much
everything was just hardcoded as '/lib/firmware'.  So using that in the
packaging was correct for all three systems.  (I don't believe patching the
three to use bitbake variables is necessarily a good idea -- since I'm sure
there are more instances of this type of usage that I'm unaware of.)

This is where using the fs-perms mechanism to move the files and setup the
symlink is the right approach in my opinion.  The key thing is figuring out the
permutations of the libdir, nonarch libdir (since they may or may not be the
same) and dealing with any QA errors that may occur.

> Ian
> 

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Ray, Ian (GE Healthcare)]

On 01/05/2016 4:33 PM, Mark Hatle wrote:
> On 1/5/16 8:04 AM, Ray, Ian (GE Healthcare) wrote:
>> On 01/05/2016 03:26 AM, Matthias Schiffer wrote:
>>> On 01/05/2016 01:28 AM, Mark Hatle wrote:
>>>> On 1/4/16 5:57 PM, Matthias Schiffer wrote:
>>>>> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>>>>>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>>>>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>>>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>>>>>> not exist. Prefer base_libdir.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>>>>>> ppc64, which is not where the firmware is expected.
>>>>>>>>>
>>>>>>>>
>>>>>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>>>>>
>>>>>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>>>>>> hard coded into them.  This is the reason why /lib/firmware was used and
>>>>>>>> not one
>>>>>>>> of the variables.
>>>>>>>>
>>>>>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>>>>>> system is expecting.  We can make some kind of accommodation for
>>>>>>>> systems where
>>>>>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>>>>>> processing and not the package itself.  (I'm referring to the
>>>>>>>> 'meta/files/fs-perms.txt' file.
>>>>>>>>
>>>>>>>> --Mark
>>
>> Ah, that makes sense.
>>
>> <snip>
>>
>>>> You conditionalize it by providing different default fs-perms files -- OR since
>>>> more then one file can be loaded -- additional fs-perms with the permutations
>>>> you desire.
>>>>
>>>> I could easily see having an files/fs-perms-usr-only.txt and an
>>>> files/fs-perms-no-usr.txt
>>>>
>>>> Where the first would be something like:
>>>>
>>>> # Define symlinks from base directories to their prefix versions
>>>> /bin	link ${bindir}
>>>> /sbin	link ${sbindir}
>>>> ...
>>
>> This is very interesting -- thank you.
>>
>> I tried patching our additional fs-perms file as Mark suggested. This resulted in
>> a build failure during packaging of a recipe. The package directory looks odd:
>>
>> ls -l /home/ian/myproduct/build/tmp/work/myproduct-linux-gnueabi/
>> myrecipe/4.3-r0/package
>> total 4
>> drwxr-xr-x. 2 ian ian 4096 Jan  5 11:30 boot
>> drwxr-xr-x. 4 ian ian   56 Jan  5 11:30 etc
>> lrwxrwxrwx. 1 ian ian    8 Jan  5 11:33 lib -> /usr/lib
>> drwxr-xr-x. 3 ian ian   16 Jan  5 11:33 usr
>
> We should look into this further.  Pseudo (the fakeroot wrapper) should be
> changing '/usr/lib' [in the cross build filesystem] to a relative path... It
> will only be '/usr/lib' in the target version if we're in the pseudo environment
> or the translation is off.  (Maybe I'm remembering incorrectly, but that was my
> memory.)
>
>> This can be fixed by teaching classes/package.bbclass to _not_ use the leading
>> slash when making the link. Not sure if that is unacceptably hacky? But it must
>> be better than leaking to host filesystem.
>>
>> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
>> index a86b680..59ef447 100644
>> --- a/meta/classes/package.bbclass
>> +++ b/meta/classes/package.bbclass
>> @@ -774,6 +774,7 @@ python fixup_perms () {
>>          link = fs_perms_table[dir].link
>>          if link[0] == "/":
>>              target = dvar + link
>> +            link = link[1:]
>>              ptarget = link
>>          else:
>>              target = os.path.join(os.path.dirname(origin), link)
>
> I'm definitely interested in understanding what may or may not be working
> properly here.
>
>> Once past that problem there is a QA problem since do_package now installs
>> to /usr/lib (because of fs-perms) instead of /lib which is mentioned in FILES.
>
> What was the QA issue?  About the absolute path name or something else?

ERROR: QA Issue: linux-advantech: Files/directories were installed but not shipped in any package:
  /usr
  /usr/lib
  /usr/lib/modules
  /usr/lib/modules/4.4.0-rc3-next-20151202
  /usr/lib/modules/4.4.0-rc3-next-20151202/modules.order
  /usr/lib/modules/4.4.0-rc3-next-20151202/modules.builtin
  ...

>> Joshua's patches for merged /usr touched FILES in _some_ cases (for example
>> 73a6fe958f47642d18ba0098cfd45c3520d53560) but linux-firmware would require
>> considerably more effort. Is that an argument in favour of nonarch_base_libdir
>> instead of literal "/lib" or is there some other technique to address this?
>
> For linux-firmware specifically.. we need to make sure the kernel and modules
> (which use the firmware) use the same filesystem notion as what the
> linux-firmware package is doing.  In the past when I've looked, pretty much
> everything was just hardcoded as '/lib/firmware'.  So using that in the
> packaging was correct for all three systems.  (I don't believe patching the
> three to use bitbake variables is necessarily a good idea -- since I'm sure
> there are more instances of this type of usage that I'm unaware of.)
>
> This is where using the fs-perms mechanism to move the files and setup the
> symlink is the right approach in my opinion.  The key thing is figuring out the
> permutations of the libdir, nonarch libdir (since they may or may not be the
> same) and dealing with any QA errors that may occur.

This makes sense, but I'm struggling to understand how FILES is affected,
when it lists /lib/foo but we install to /usr/lib/foo.

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Mark Hatle]

On 1/5/16 8:41 AM, Ray, Ian (GE Healthcare) wrote:
> On 01/05/2016 4:33 PM, Mark Hatle wrote:
>> On 1/5/16 8:04 AM, Ray, Ian (GE Healthcare) wrote:
>>> On 01/05/2016 03:26 AM, Matthias Schiffer wrote:
>>>> On 01/05/2016 01:28 AM, Mark Hatle wrote:
>>>>> On 1/4/16 5:57 PM, Matthias Schiffer wrote:
>>>>>> On 01/04/2016 11:56 PM, Mark Hatle wrote:
>>>>>>> On 1/4/16 4:26 PM, Matthias Schiffer wrote:
>>>>>>>> On 01/04/2016 05:32 PM, Mark Hatle wrote:
>>>>>>>>> On 1/4/16 10:11 AM, Matthias Schiffer wrote:
>>>>>>>>>> On 01/04/2016 02:14 PM, Ian Ray wrote:
>>>>>>>>>>> The recipe uses hard-coded paths (specifically /lib) in do_install
>>>>>>>>>>> and in FILES, however on a merged /usr system this directory might
>>>>>>>>>>> not exist. Prefer base_libdir.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Ian Ray <ian.ray@ge.com>
>>>>>>>>>>> ---
>>>>>>>>>>
>>>>>>>>>> This should use nonarch_base_libdir, base_libdir defaults to /lib64 on
>>>>>>>>>> ppc64, which is not where the firmware is expected.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> At a minimum, I would agree nonarch_base_libdir, however..
>>>>>>>>>
>>>>>>>>> I believe that the kernel loader/modules/tools themselves actually have '/lib'
>>>>>>>>> hard coded into them.  This is the reason why /lib/firmware was used and
>>>>>>>>> not one
>>>>>>>>> of the variables.
>>>>>>>>>
>>>>>>>>> This is one of the cases were /lib is actually correct, since that is what the
>>>>>>>>> system is expecting.  We can make some kind of accommodation for
>>>>>>>>> systems where
>>>>>>>>> /lib -> /usr/lib... but that should be done inside of the filesystem setup
>>>>>>>>> processing and not the package itself.  (I'm referring to the
>>>>>>>>> 'meta/files/fs-perms.txt' file.
>>>>>>>>>
>>>>>>>>> --Mark
>>>
>>> Ah, that makes sense.
>>>
>>> <snip>
>>>
>>>>> You conditionalize it by providing different default fs-perms files -- OR since
>>>>> more then one file can be loaded -- additional fs-perms with the permutations
>>>>> you desire.
>>>>>
>>>>> I could easily see having an files/fs-perms-usr-only.txt and an
>>>>> files/fs-perms-no-usr.txt
>>>>>
>>>>> Where the first would be something like:
>>>>>
>>>>> # Define symlinks from base directories to their prefix versions
>>>>> /bin	link ${bindir}
>>>>> /sbin	link ${sbindir}
>>>>> ...
>>>
>>> This is very interesting -- thank you.
>>>
>>> I tried patching our additional fs-perms file as Mark suggested. This resulted in
>>> a build failure during packaging of a recipe. The package directory looks odd:
>>>
>>> ls -l /home/ian/myproduct/build/tmp/work/myproduct-linux-gnueabi/
>>> myrecipe/4.3-r0/package
>>> total 4
>>> drwxr-xr-x. 2 ian ian 4096 Jan  5 11:30 boot
>>> drwxr-xr-x. 4 ian ian   56 Jan  5 11:30 etc
>>> lrwxrwxrwx. 1 ian ian    8 Jan  5 11:33 lib -> /usr/lib
>>> drwxr-xr-x. 3 ian ian   16 Jan  5 11:33 usr
>>
>> We should look into this further.  Pseudo (the fakeroot wrapper) should be
>> changing '/usr/lib' [in the cross build filesystem] to a relative path... It
>> will only be '/usr/lib' in the target version if we're in the pseudo environment
>> or the translation is off.  (Maybe I'm remembering incorrectly, but that was my
>> memory.)
>>
>>> This can be fixed by teaching classes/package.bbclass to _not_ use the leading
>>> slash when making the link. Not sure if that is unacceptably hacky? But it must
>>> be better than leaking to host filesystem.
>>>
>>> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
>>> index a86b680..59ef447 100644
>>> --- a/meta/classes/package.bbclass
>>> +++ b/meta/classes/package.bbclass
>>> @@ -774,6 +774,7 @@ python fixup_perms () {
>>>          link = fs_perms_table[dir].link
>>>          if link[0] == "/":
>>>              target = dvar + link
>>> +            link = link[1:]
>>>              ptarget = link
>>>          else:
>>>              target = os.path.join(os.path.dirname(origin), link)
>>
>> I'm definitely interested in understanding what may or may not be working
>> properly here.
>>
>>> Once past that problem there is a QA problem since do_package now installs
>>> to /usr/lib (because of fs-perms) instead of /lib which is mentioned in FILES.
>>
>> What was the QA issue?  About the absolute path name or something else?
> 
> ERROR: QA Issue: linux-advantech: Files/directories were installed but not shipped in any package:
>   /usr
>   /usr/lib
>   /usr/lib/modules
>   /usr/lib/modules/4.4.0-rc3-next-20151202
>   /usr/lib/modules/4.4.0-rc3-next-20151202/modules.order
>   /usr/lib/modules/4.4.0-rc3-next-20151202/modules.builtin
>   ...

In this case, this is actually what I would expect....  I suspect we may have
some issues with the way the FILES_ entries are setup in classes and packages.
Biggest problem though is /lib/modules and /lib/firmware are a bit unique in
that they really expect to be 'at that location'...... so moving them requires
adjusting the FILES entry as well.. Hmm.

I'm not really sure I like the idea, but it might also make sense for code
related to the fs-perms processing to adjust the FILES entries accordingly...
but Matthias's changes to the kernel setup bbclass might be the right fix (and
avoid more complex matching code..)

>>> Joshua's patches for merged /usr touched FILES in _some_ cases (for example
>>> 73a6fe958f47642d18ba0098cfd45c3520d53560) but linux-firmware would require
>>> considerably more effort. Is that an argument in favour of nonarch_base_libdir
>>> instead of literal "/lib" or is there some other technique to address this?
>>
>> For linux-firmware specifically.. we need to make sure the kernel and modules
>> (which use the firmware) use the same filesystem notion as what the
>> linux-firmware package is doing.  In the past when I've looked, pretty much
>> everything was just hardcoded as '/lib/firmware'.  So using that in the
>> packaging was correct for all three systems.  (I don't believe patching the
>> three to use bitbake variables is necessarily a good idea -- since I'm sure
>> there are more instances of this type of usage that I'm unaware of.)
>>
>> This is where using the fs-perms mechanism to move the files and setup the
>> symlink is the right approach in my opinion.  The key thing is figuring out the
>> permutations of the libdir, nonarch libdir (since they may or may not be the
>> same) and dealing with any QA errors that may occur.
> 
> This makes sense, but I'm struggling to understand how FILES is affected,
> when it lists /lib/foo but we install to /usr/lib/foo.
> 

fs-perms doesn't affect the FILES entries at all.  The original design
assumption is that if a file was listed at a specific location it was fixed due
to the design of the application -- while 90% of things used the bitbake
variables which could change easily.

This is a situation where we've got hard coded '/lib' entries in FILES, but
we're trying to move the directory elsewhere.... 'adjusting' FILES automatically
is probably more error prone and complicated then desired.. but I'm not sure
switching to nonarch_baselib_dir is right either.. argh.  (But it may certainly
be the less evil in this case.)

--Mark

Re: [PATCH 1/1] linux-firmware: remove hard-coded paths

[Phil Blundell]

On Tue, 2016-01-05 at 11:06 -0600, Mark Hatle wrote:
> This is a situation where we've got hard coded '/lib' entries in FILES, but
> we're trying to move the directory elsewhere.... 'adjusting' FILES automatically
> is probably more error prone and complicated then desired.. but I'm not sure
> switching to nonarch_baselib_dir is right either.. argh.  (But it may certainly
> be the less evil in this case.)

Isn't this more or less exactly what ${nonarch_baselib_dir} is for?  If
there's some disconnect between firmware and other users of that
variable (mostly mdev/udev/systemd I guess) then maybe we should
introduce a separate ${firmware_loader_dir} or something.  But either
way, it seems fairly clear that the right thing is to simply have a
variable that the distro can set to the right place, and then have both
do_install() and FILES defined in terms of that same variable, rather
than doing some sort of subsequent post-processing step that tries to
move the contents around and then patch FILES up to match.

p.