read-only-rootfs and opkg

read-only-rootfs and opkg

[Martin Siegumfeldt]

[-- Attachment #1: Type: text/plain, Size: 2444 bytes --]

Hi all,

I am trying to realize a rootfs backed by opkg as package manager. It appears that the opkg meta-data is directed to a volatile mount (triggered by read-only-rootfs feature in order to achieve write access I assume):

$ tail -5 /etc/opkg/opkg.conf
# Default destination for installed packages
dest root /
option lists_dir   /var/lib/opkg/lists
option info_dir    /var/lib/opkg/info
option status_file /var/lib/opkg/status

$ mount | grep var/lib
overlay on /var/lib type overlay (rw,relatime,lowerdir=/var/lib,upperdir=/var/volatile/lib,workdir=/var/volatile/.lib-work)

$ find /var/lib/opkg/ | head
/var/lib/opkg/
/var/lib/opkg/status
/var/lib/opkg/info
/var/lib/opkg/info/vim-tiny.list
/var/lib/opkg/info/libdaemon0.list
/var/lib/opkg/info/coreutils.prerm
/var/lib/opkg/info/busybox-syslog.control
/var/lib/opkg/info/libc6.postinst
/var/lib/opkg/info/libmount1.list
/var/lib/opkg/info/libpcap1.postinst

Consequently, any updated meta-data do no persist across reboots.

I have attempted to assign 'OPKGLIBDIR' to a R/W mount:

OPKGLIBDIR ?= "/data-os/opkg"

, which does appear to update the conf file:

$ tail -5 /etc/opkg/opkg.conf
# Default destination for installed packages
dest root /
option lists_dir   /data-os/opkg/opkg/lists
option info_dir    /data-os/opkg/opkg/info
option status_file /data-os/opkg/opkg/status
but meta-data still appears to reside with '/var/lib'

but meta-data still appears residing within '/var/lib':

$ find /data-os/opkg/
/data-os/opkg/
/data-os/opkg/opkg
/data-os/opkg/opkg/info

$ find /var/lib/opkg/ | head
/var/lib/opkg/
/var/lib/opkg/status
/var/lib/opkg/info
/var/lib/opkg/info/vim-tiny.list
/var/lib/opkg/info/libdaemon0.list
/var/lib/opkg/info/coreutils.prerm
/var/lib/opkg/info/busybox-syslog.control
/var/lib/opkg/info/libc6.postinst
/var/lib/opkg/info/libmount1.list
/var/lib/opkg/info/libpcap1.postinst

and opkg appears broken (as nothing is indicated installed):

$ opkg list-installed; echo $?
0

To clarify, I imagine a process where the ipk file is uploaded to the device, after which the rootfs is remounted R/W and ipk installed/updated through opkg. Installation/update do succeed, but the meta-data do not persist as inteded.

Note, I am on baseline Langdale, no obvious changes seem to have been pushed to master though.

Am I missing any available configuration to achieve this?
Thanks,
Martin

[-- Attachment #2: Type: text/html, Size: 3164 bytes --]

Re: [EXTERNAL] [yocto] read-only-rootfs and opkg

[Stewart, Alex]

Hey Martin,

The opkg config files used during OE rootfs creation and at-runtime are different. The former is supposed to be generated by the opkg package_manager implementation [1]. The latter is written by the opkg recipe via the do_install (that you identified).

The files you're seeing installed to he incorrect location (`/var/lib`) are the product of the rootfs installation, I think. So the issue seems to imply an issue in config [1].

Both the recipe and package_manager config files are supposed to handle the OPKGLIBDIR variable. But they might not be handling them *at the same place* in the OE build process.

My intuition is that the weak-default OPKGLIBDIR assignment is being immediately-expanded to the upstream value (/var/lib) in the package_manager implementation prior to your bbappend's weak-assignment taking effect [2].

Does this issue reproduce if you assign OPKGLIBDIR more strongly? eg.
```
OPKGLIBDIR = "/data-os/opkg"
```

If it does still, try debugging the values of the package_manager's `self.opkg_dir` attribute [1].

[1] https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/package_manager/ipk/__init__.py#n111

[2] https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html?highlight=operators#setting-a-weak-default-value

---
Alex Stewart
Software Engineer. PR&D Security, RTOS.
T&M, Emerson Electric Corp. (NI)

________________________________________
From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> on behalf of Martin Siegumfeldt <mns@gomspace.com>
Sent: Monday, September 16, 2024 06:00
To: yocto@lists.yoctoproject.org
Subject: [EXTERNAL] [yocto] read-only-rootfs and opkg

Hi all, I am trying to realize a rootfs backed by opkg as package manager. It appears that the opkg meta-data is directed to a volatile mount (triggered by read-only-rootfs feature in order to achieve write access I assume): $ tail -5 /etc/opkg/opkg. conf

Hi all,

I am trying to realize a rootfs backed by opkg as package manager. It appears that the opkg meta-data is directed to a volatile mount (triggered by read-only-rootfs feature in order to achieve write access I assume):

$ tail -5 /etc/opkg/opkg.conf
# Default destination for installed packages
dest root /
option lists_dir   /var/lib/opkg/lists
option info_dir    /var/lib/opkg/info
option status_file /var/lib/opkg/status

$ mount | grep var/lib
overlay on /var/lib type overlay (rw,relatime,lowerdir=/var/lib,upperdir=/var/volatile/lib,workdir=/var/volatile/.lib-work)

$ find /var/lib/opkg/ | head
/var/lib/opkg/
/var/lib/opkg/status
/var/lib/opkg/info
/var/lib/opkg/info/vim-tiny.list
/var/lib/opkg/info/libdaemon0.list
/var/lib/opkg/info/coreutils.prerm
/var/lib/opkg/info/busybox-syslog.control
/var/lib/opkg/info/libc6.postinst
/var/lib/opkg/info/libmount1.list
/var/lib/opkg/info/libpcap1.postinst

Consequently, any updated meta-data do no persist across reboots.

I have attempted to assign 'OPKGLIBDIR' to a R/W mount:

OPKGLIBDIR ?= "/data-os/opkg"

, which does appear to update the conf file:

$ tail -5 /etc/opkg/opkg.conf
# Default destination for installed packages
dest root /
option lists_dir   /data-os/opkg/opkg/lists
option info_dir    /data-os/opkg/opkg/info
option status_file /data-os/opkg/opkg/status
but meta-data still appears to reside with '/var/lib'

but meta-data still appears residing within '/var/lib':

$ find /data-os/opkg/
/data-os/opkg/
/data-os/opkg/opkg
/data-os/opkg/opkg/info

$ find /var/lib/opkg/ | head
/var/lib/opkg/
/var/lib/opkg/status
/var/lib/opkg/info
/var/lib/opkg/info/vim-tiny.list
/var/lib/opkg/info/libdaemon0.list
/var/lib/opkg/info/coreutils.prerm
/var/lib/opkg/info/busybox-syslog.control
/var/lib/opkg/info/libc6.postinst
/var/lib/opkg/info/libmount1.list
/var/lib/opkg/info/libpcap1.postinst

and opkg appears broken (as nothing is indicated installed):

$ opkg list-installed; echo $?
0


To clarify, I imagine a process where the ipk file is uploaded to the device, after which the rootfs is remounted R/W and ipk installed/updated through opkg. Installation/update do succeed, but the meta-data do not persist as inteded.

Note, I am on baseline Langdale, no obvious changes seem to have been pushed to master though.

Am I missing any available configuration to achieve this?
Thanks,
Martin

Re: [yocto] read-only-rootfs and opkg

[Martin Siegumfeldt]

[-- Attachment #1: Type: text/plain, Size: 1110 bytes --]

Hi Alex,

Thanks for your input - highly appreciated.

It appears you have a point regarding the variable assignment - 'bitbake-getvar OPKGLIBDIR' indeed indicates the intended assignment does not occur:
$ bitbake-getvar --value OPKGLIBDIR
NOTE: Starting bitbake server...
/var/lib

Doing the strong assignment (OPKGLIBDIR = "/data-os/opkg") does not make a difference, which puzzles me as all assigments (I am able to grep) are weak:

$ rg ^OPKGLIBDIR
meta/classes-recipe/rootfs_ipk.bbclass
30:OPKGLIBDIR ??= "${localstatedir}/lib"
meta/recipes-devtools/opkg/opkg_0.6.1.bb
32:OPKGLIBDIR ??= "${target_localstatedir}/lib"
meta/classes-global/package_ipk.bbclass
24:OPKGLIBDIR ??= "${localstatedir}/lib"

However, based on your reply, I am starting to wonder whether what I am trying to achieve is at all possible. The mount '/data-os' is a dedicated partition mounted R/W and used by overlayfs-etc, my assumption was to configure opkg to use this for the meta-data. But if the meta-data is rendered build-time during rootfs creation, I do not see how this is achievable?

Thanks,
Martin

[-- Attachment #2: Type: text/html, Size: 1384 bytes --]

Re: [EXTERNAL] Re: [yocto] read-only-rootfs and opkg

[Stewart, Alex]

[-- Attachment #1: Type: text/plain, Size: 2526 bytes --]

RE: variable expansion. Where are you assigning OPKGLIBDIR its new value? If it is in an opkg bbappends in your own layer, I wonder if the bbappends is not being parsed during rootfs creation.


RE: the partitions. Hmm, yeah. I agree that it probably isn't sensible to expect the opkg state information to be placed into /data-os during rootfs creation.

Does your usecase support running opkg at runtime? If not, you might just accept that the state information stays on the read-only partition.

If runtime opkg operations are supported, you might make a one-time initscript that runs on first-boot and moves the sstate information to /data-os.

Sorry if my emails are strangely formatted by the way. I'm in a weird temporary email setup.


---
Alex Stewart
Software Engineer. PR&D Security, RTOS.
T&M, Emerson Electric Corp. (NI)

________________________________________
From: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> on behalf of Martin Siegumfeldt <mns@gomspace.com>
Sent: Wednesday, September 18, 2024 04:39
To: Stewart, Alex; yocto@lists.yoctoproject.org
Subject: [EXTERNAL] Re: [yocto] read-only-rootfs and opkg

Hi Alex, Thanks for your input - highly appreciated. It appears you have a point regarding the variable assignment - 'bitbake-getvar OPKGLIBDIR' indeed indicates the intended assignment does not occur: $ bitbake-getvar --value OPKGLIBDIR NOTE: 

Hi Alex,

Thanks for your input - highly appreciated.

It appears you have a point regarding the variable assignment - 'bitbake-getvar OPKGLIBDIR' indeed indicates the intended assignment does not occur:
$ bitbake-getvar --value OPKGLIBDIR
NOTE: Starting bitbake server...
/var/lib

Doing the strong assignment (OPKGLIBDIR = "/data-os/opkg") does not make a difference, which puzzles me as all assigments (I am able to grep) are weak:

$ rg ^OPKGLIBDIR
meta/classes-recipe/rootfs_ipk.bbclass
30:OPKGLIBDIR ??= "${localstatedir}/lib"
meta/recipes-devtools/opkg/opkg_0.6.1.bb
32:OPKGLIBDIR ??= "${target_localstatedir}/lib"
meta/classes-global/package_ipk.bbclass
24:OPKGLIBDIR ??= "${localstatedir}/lib"

However, based on your reply, I am starting to wonder whether what I am trying to achieve is at all possible. The mount '/data-os' is a dedicated partition mounted R/W and used by overlayfs-etc, my assumption was to configure opkg to use this for the meta-data. But if the meta-data is rendered build-time during rootfs creation, I do not see how this is achievable?

Thanks,
Martin


[-- Attachment #2: ATT00001.txt --]
[-- Type: text/plain, Size: 784 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#63858): https://urldefense.com/v3/__https://lists.yoctoproject.org/g/yocto/message/63858__;!!OkyjEqkqqe9UhNQ!xaN31BlkIMYtsVPvKIVGPH0bIS1X45V8P3AS3cvykPFnNeAPXjUJjGlQbLjDmMn7FkXlQwsJMHlIdQw$ 
Mute This Topic: https://urldefense.com/v3/__https://lists.yoctoproject.org/mt/108478012/3616788__;!!OkyjEqkqqe9UhNQ!xaN31BlkIMYtsVPvKIVGPH0bIS1X45V8P3AS3cvykPFnNeAPXjUJjGlQbLjDmMn7FkXlQwsJaNEQ6Hg$ 
Group Owner: yocto+owner@lists.yoctoproject.org
Unsubscribe: https://urldefense.com/v3/__https://lists.yoctoproject.org/g/yocto/unsub__;!!OkyjEqkqqe9UhNQ!xaN31BlkIMYtsVPvKIVGPH0bIS1X45V8P3AS3cvykPFnNeAPXjUJjGlQbLjDmMn7FkXlQwsJMLshdPs$  [alex.stewart@emerson.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [yocto] read-only-rootfs and opkg

[Martin Siegumfeldt]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

Indeed, it was from a layer-local "opkg_%.bbappend" - moving to local.conf seems to do the trick.

Yes I would like to use opkg runtime to be able to update individual components and also keep the meta-data updated, such that e.g. 'opkg info <COMPONENT>' would indicate actuals.

Runtime moving of the meta-data from '/var/lib' to '/data-os' might be an option - at least I can't think of a better for now. Manual experiments indicates the concept functional.

Again, thanks for your ideas Alex - appreciated.

Have a nice weekend,
Martin

[-- Attachment #2: Type: text/html, Size: 704 bytes --]