[PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[dani.barra25]

From: Daniel Andrade <dani.barra25@gmail.com>

Using `install` in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses `cp` without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
Since you are using that variable everywhere, it is not the same value as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf` that is applied in every other file.

Daniel Andrade (1):
  wic: Content of the temporary updated fstab should be copied into the
    original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

[PATCH 1/1] wic: Content of the temporary updated fstab should be copied into the original not replacing it entirely.

[dani.barra25]

From: Daniel Andrade <dani.barra25@gmail.com>

Fixes [15947]
The current functionality to update fstab generates a new temporary fstab with the new partition configuration.
However, this file does not retain any metadata being it a completely new file.
Because of this, when the rootfs plugin under `poky/scripts/lib/wic/plugins/source/rootfs.py` copies the file, it overrides the original fstab metadata.
The patch removes implementation of msdos/ext code for fstab since it is not possible to append content without rewriting the file to preserve metadata.

The timestamp applied to fstab is not the same as every other file.
It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp SOURCE_DATE_EPOCH_FALLBACK.
Since that variable is used everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Signed-off-by: Daniel Andrade <dani.barra25@gmail.com>
---
 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index acf4e2d153..0581fc3564 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -684,8 +684,10 @@ export PYTHONHASHSEED = "0"
 export PERL_HASH_SEED = "0"
 export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
 # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
+# If these 2 are misaligned, when generating fstab later on wic, the timestamps will never match because
+# The repeatibility uses the REPRODUCIBLE_TIMESTAMP_ROOTFS while wic uses SOURCE_DATE_EPOCH (therefore its fallback)
 SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
-REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1302044400"
 
 ##################################################################
 # Settings used by bitbake-layers.
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index bf2c34d594..82d754835c 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index e29f3a4c2f..3848af2a91 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -223,8 +223,8 @@ class RootfsPlugin(SourcePlugin):
             part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab"))
             if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update:
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
-                # Assume that fstab should always be owned by root with fixed permissions
-                install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                # We dont want any metadata of the updated fstab, just the one that already existed
+                install_cmd = "cp --no-preserve=all %s %s" % (part.updated_fstab_path, fstab_path)
                 if new_pseudo:
                     pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
                 else:

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Mathieu Dubois-Briand]

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:
> From: Daniel Andrade <dani.barra25@gmail.com>
>
> Using `install` in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
> The fix just uses `cp` without preserving attributes from the temporary fstab since none of them are needed, just the content.
> Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.
>
> Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
> Since you are using that variable everywhere, it is not the same value as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf` that is applied in every other file.
>
> Daniel Andrade (1):
>   wic: Content of the temporary updated fstab should be copied into the
>     original not replacing it entirely.
>
>  meta/conf/bitbake.conf                   |  4 +++-
>  scripts/lib/wic/partition.py             | 15 +--------------
>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>  3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
    self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
  File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
    raise self.failureException(msg)
AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Can you fix it please?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 3748 bytes --]

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via 
lists.openembedded.org wrote:
> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:
>> From: Daniel Andrade<dani.barra25@gmail.com>
>>
>> Using `install` in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
>> The fix just uses `cp` without preserving attributes from the temporary fstab since none of them are needed, just the content.
>> Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.
>>
>> Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
>> Since you are using that variable everywhere, it is not the same value as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf` that is applied in every other file.
>>
>> Daniel Andrade (1):
>>    wic: Content of the temporary updated fstab should be copied into the
>>      original not replacing it entirely.
>>
>>   meta/conf/bitbake.conf                   |  4 +++-
>>   scripts/lib/wic/partition.py             | 15 +--------------
>>   scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>>   3 files changed, 6 insertions(+), 17 deletions(-)
> Hi Daniel,
>
> Thanks for your patch.
>
> It looks like it is breaking a test:
>
> 2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
> ...
> 2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>    File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
>      self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
>    File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
>      assertion_func(first, second, msg=msg)
>    File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
>      self.fail(self._formatMessage(msg, standardMsg))
>    File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
>      raise self.failureException(msg)
> AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
> - af3c087d6c9131735c8d1f270a226892
> + 9edb8255abd217fdb20e118833afb856
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
>
> Can you fix it please?
Ping?

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug 
review meeting
so that's why I'm sending this email.

../Randy


>
> Thanks,
> Mathieu
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#222962):https://lists.openembedded.org/g/openembedded-core/message/222962
> Mute This Topic:https://lists.openembedded.org/mt/115043355/3616765
> Group Owner:openembedded-core+owner@lists.openembedded.org
> Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 5758 bytes --]

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Daniel Andrade]

[-- Attachment #1: Type: text/plain, Size: 3660 bytes --]

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta, 11/09/2025
à(s) 17:56):

> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org
> wrote:
>
> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:
>
> From: Daniel Andrade <dani.barra25@gmail.com> <dani.barra25@gmail.com>
>
> Using `install` in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
> The fix just uses `cp` without preserving attributes from the temporary fstab since none of them are needed, just the content.
> Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.
>
> Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
> Since you are using that variable everywhere, it is not the same value as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf` that is applied in every other file.
>
> Daniel Andrade (1):
>   wic: Content of the temporary updated fstab should be copied into the
>     original not replacing it entirely.
>
>  meta/conf/bitbake.conf                   |  4 +++-
>  scripts/lib/wic/partition.py             | 15 +--------------
>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>  3 files changed, 6 insertions(+), 17 deletions(-)
>
> Hi Daniel,
>
> Thanks for your patch.
>
> It looks like it is breaking a test:
>
> 2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
> ...
> 2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>   File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
>   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
>     assertion_func(first, second, msg=msg)
>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
>     self.fail(self._formatMessage(msg, standardMsg))
>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
>     raise self.failureException(msg)
> AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
> - af3c087d6c9131735c8d1f270a226892
> + 9edb8255abd217fdb20e118833afb856
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
>
> Can you fix it please?
>
> Ping?
>
> I think this is being tracked by:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
>
> "WIC does not preserve metadata when updating fstab"
>
> Btw, we were just following the "Need Info" process during the bug review
> meeting
> so that's why I'm sending this email.
>
> ../Randy
>
> Thanks,
> Mathieu
>
>
>
> 
>
>
>
> --
> # Randy MacLeod
> # Wind River Linux
>
>

[-- Attachment #2: Type: text/html, Size: 5295 bytes --]

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Daniel Andrade]

[-- Attachment #1: Type: text/plain, Size: 4605 bytes --]

Good Afternoon,

I have been trying to figure it out and something seems off.
The specific test you refer to may be hiding a problem or maybe I have my
testbench compromised and I would like your help to test.
The test `test_no_fstab_update ` verifies if the fstab was updated or not
because on the wks file used to do it there is a --no-fstab-update. The
thing is, I think this test is passing because what is forcing it to not
update is a constraint that I removed previously on the patch (the one that
checks if the system is ext* or msdos) and not because the flag is
triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs and
indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <dani.barra25@gmail.com> escreveu (sábado, 13/09/2025 à(s)
11:02):

> Hello guys, sorry for the delay.
> I have been busy, but I hope to provide you with a fix in the next few
> days.
>
> Sorry about that,
> Daniel
>
> Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta, 11/09/2025
> à(s) 17:56):
>
>> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via
>> lists.openembedded.org wrote:
>>
>> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:
>>
>> From: Daniel Andrade <dani.barra25@gmail.com> <dani.barra25@gmail.com>
>>
>> Using `install` in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
>> The fix just uses `cp` without preserving attributes from the temporary fstab since none of them are needed, just the content.
>> Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.
>>
>> Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
>> Since you are using that variable everywhere, it is not the same value as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf` that is applied in every other file.
>>
>> Daniel Andrade (1):
>>   wic: Content of the temporary updated fstab should be copied into the
>>     original not replacing it entirely.
>>
>>  meta/conf/bitbake.conf                   |  4 +++-
>>  scripts/lib/wic/partition.py             | 15 +--------------
>>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>>  3 files changed, 6 insertions(+), 17 deletions(-)
>>
>> Hi Daniel,
>>
>> Thanks for your patch.
>>
>> It looks like it is breaking a test:
>>
>> 2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
>> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
>> ...
>> 2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>>   File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
>>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
>>   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
>>     assertion_func(first, second, msg=msg)
>>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
>>     self.fail(self._formatMessage(msg, standardMsg))
>>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
>>     raise self.failureException(msg)
>> AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
>> - af3c087d6c9131735c8d1f270a226892
>> + 9edb8255abd217fdb20e118833afb856
>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
>>
>> Can you fix it please?
>>
>> Ping?
>>
>> I think this is being tracked by:
>>
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
>>
>> "WIC does not preserve metadata when updating fstab"
>>
>> Btw, we were just following the "Need Info" process during the bug review
>> meeting
>> so that's why I'm sending this email.
>>
>> ../Randy
>>
>> Thanks,
>> Mathieu
>>
>>
>>
>> 
>>
>>
>>
>> --
>> # Randy MacLeod
>> # Wind River Linux
>>
>>

[-- Attachment #2: Type: text/html, Size: 6649 bytes --]

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Mathieu Dubois-Briand]

Hi Daniel,

Sorry, but I'm not sure to get your point.

I ran the test locally, and it does fail. It also fails if I modify it
to use squashfs instead of ext4.

My reproduction procedure:
git clone https://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-2320
. oe-init-build-env
echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
oe-selftest -r wic.Wic.test_no_fstab_update

Thanks,
Mathieu

On Tue Sep 23, 2025 at 5:05 PM CEST, Daniel Andrade wrote:
> Good Afternoon,
>
> I have been trying to figure it out and something seems off.
> The specific test you refer to may be hiding a problem or maybe I have my
> testbench compromised and I would like your help to test.
> The test `test_no_fstab_update ` verifies if the fstab was updated or not
> because on the wks file used to do it there is a --no-fstab-update. The
> thing is, I think this test is passing because what is forcing it to not
> update is a constraint that I removed previously on the patch (the one that
> checks if the system is ext* or msdos) and not because the flag is
> triggering its intended functionality.
> The crosscheck I did was modifying the fstype from ext4 to squashfs and
> indeed the test failed.
> Can someone also verify this?
>
> Daniel
>
> Daniel Andrade <dani.barra25@gmail.com> escreveu (sábado, 13/09/2025 à(s)
> 11:02):
>
>> Hello guys, sorry for the delay.
>> I have been busy, but I hope to provide you with a fix in the next few
>> days.
>>
>> Sorry about that,
>> Daniel
>>
>> Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta, 11/09/2025
>> à(s) 17:56):
>>
>>> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via
>>> lists.openembedded.org wrote:
>>>
>>> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:
>>>
>>> From: Daniel Andrade <dani.barra25@gmail.com> <dani.barra25@gmail.com>
>>>
>>> Using `install` in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
>>> The fix just uses `cp` without preserving attributes from the temporary fstab since none of them are needed, just the content.
>>> Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.
>>>
>>> Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
>>> Since you are using that variable everywhere, it is not the same value as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf` that is applied in every other file.
>>>
>>> Daniel Andrade (1):
>>>   wic: Content of the temporary updated fstab should be copied into the
>>>     original not replacing it entirely.
>>>
>>>  meta/conf/bitbake.conf                   |  4 +++-
>>>  scripts/lib/wic/partition.py             | 15 +--------------
>>>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>>>  3 files changed, 6 insertions(+), 17 deletions(-)
>>>
>>> Hi Daniel,
>>>
>>> Thanks for your patch.
>>>
>>> It looks like it is breaking a test:
>>>
>>> 2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
>>> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
>>> ...
>>> 2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>>>   File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
>>>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
>>>   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
>>>     assertion_func(first, second, msg=msg)
>>>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
>>>     self.fail(self._formatMessage(msg, standardMsg))
>>>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
>>>     raise self.failureException(msg)
>>> AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
>>> - af3c087d6c9131735c8d1f270a226892
>>> + 9edb8255abd217fdb20e118833afb856
>>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
>>>
>>> Can you fix it please?
>>>
>>> Ping?
>>>
>>> I think this is being tracked by:
>>>
>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
>>>
>>> "WIC does not preserve metadata when updating fstab"
>>>
>>> Btw, we were just following the "Need Info" process during the bug review
>>> meeting
>>> so that's why I'm sending this email.
>>>
>>> ../Randy
>>>
>>> Thanks,
>>> Mathieu
>>>
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#222962): https://lists.openembedded.org/g/openembedded-core/message/222962
>>> Mute This Topic: https://lists.openembedded.org/mt/115043355/3616765
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>>>
>>>
>>> --
>>> # Randy MacLeod
>>> # Wind River Linux
>>>
>>>




-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Daniel Andrade]

[-- Attachment #1: Type: text/plain, Size: 7509 bytes --]

Hello Mathieu,

Thank you for the quick answer.
Ok, I will start from the beginning.

The failing test is supposed to test `--no-fstab-update` flags on wks file.
According to the understanding of the code, the flag is being passed across
python scripts either as `self.no_fstab_update`, like in
scripts/lib/wic/partition.py, or `part.no_fstab_update`, on
scripts/lib/wic/plugins/source/rootfs.py.
The issue that is arising from the `oe-selftest -r
wic.Wic.test_no_fstab_update` is that it is not actually testing it,
because somehow the assigning of the previous variables is not correct,
which leads to a later assumption.
The thing that is really doing the "no update" is the line 134 of
`scripts/lib/wic/partition.py` where it excludes ext* and msdos partitions
from falling in the common plugin for Rootfs.
Because of this, I suggested someone to, in a clean and a different
environment compared to mine, change the wks file used to `squashfs` fstype
(or any other supported wic fstype) instead of `ext4` which indeed also
made the test fail.
In summary, the test is failing not because of the patch (even though maybe
something can be improved or fixed later), but because it is just checking
for ext4 partitions that fall in a different way of approaching fstab while
the real mechanism is not behaving as desired.

TLDR: The test is getting passed not because it is enforcing
`--no-fstab-update` but because the partition is ext* or msdos. Changing it
to squashfs, btrfs or any other wic supported partition, with or without
the patch, will cause this to fail. This is an extra thing non-related with
the proposed patch but that is preventing it to behave correctly as
intended by the test.

Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> escreveu (terça,
23/09/2025 à(s) 19:44):

> Hi Daniel,
>
> Sorry, but I'm not sure to get your point.
>
> I ran the test locally, and it does fail. It also fails if I modify it
> to use squashfs instead of ext4.
>
> My reproduction procedure:
> git clone https://git.yoctoproject.org/poky-ci-archive -b
> autobuilder.yoctoproject.org/valkyrie/a-full-2320
> . oe-init-build-env
> echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
> oe-selftest -r wic.Wic.test_no_fstab_update
>
> Thanks,
> Mathieu
>
> On Tue Sep 23, 2025 at 5:05 PM CEST, Daniel Andrade wrote:
> > Good Afternoon,
> >
> > I have been trying to figure it out and something seems off.
> > The specific test you refer to may be hiding a problem or maybe I have my
> > testbench compromised and I would like your help to test.
> > The test `test_no_fstab_update ` verifies if the fstab was updated or not
> > because on the wks file used to do it there is a --no-fstab-update. The
> > thing is, I think this test is passing because what is forcing it to not
> > update is a constraint that I removed previously on the patch (the one
> that
> > checks if the system is ext* or msdos) and not because the flag is
> > triggering its intended functionality.
> > The crosscheck I did was modifying the fstype from ext4 to squashfs and
> > indeed the test failed.
> > Can someone also verify this?
> >
> > Daniel
> >
> > Daniel Andrade <dani.barra25@gmail.com> escreveu (sábado, 13/09/2025
> à(s)
> > 11:02):
> >
> >> Hello guys, sorry for the delay.
> >> I have been busy, but I hope to provide you with a fix in the next few
> >> days.
> >>
> >> Sorry about that,
> >> Daniel
> >>
> >> Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta,
> 11/09/2025
> >> à(s) 17:56):
> >>
> >>> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via
> >>> lists.openembedded.org wrote:
> >>>
> >>> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via
> lists.openembedded.org wrote:
> >>>
> >>> From: Daniel Andrade <dani.barra25@gmail.com> <dani.barra25@gmail.com>
> >>>
> >>> Using `install` in the rootfs plugin forces fstab to be replaced
> entirely, meaning that even its Inodes will change, leading xattrs and
> SELinux context stored by pseudo not to be applied.
> >>> The fix just uses `cp` without preserving attributes from the
> temporary fstab since none of them are needed, just the content.
> >>> Same thing happens with the predefined mechanisms for ext4 and msdos.
> Using debugfs there is no way to replace contents while maintaining
> metadata, so the approach taken on the path was to remove the different
> fstab logic for those fstypes and also use the same modified cp command.
> Reviewing the builds I did it seems to work for all of the fstypes.
> >>>
> >>> Another problem is that the timestamp applied to fstab is not the same
> as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to
> the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
> >>> Since you are using that variable everywhere, it is not the same value
> as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf`
> that is applied in every other file.
> >>>
> >>> Daniel Andrade (1):
> >>>   wic: Content of the temporary updated fstab should be copied into the
> >>>     original not replacing it entirely.
> >>>
> >>>  meta/conf/bitbake.conf                   |  4 +++-
> >>>  scripts/lib/wic/partition.py             | 15 +--------------
> >>>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
> >>>  3 files changed, 6 insertions(+), 17 deletions(-)
> >>>
> >>> Hi Daniel,
> >>>
> >>> Thanks for your patch.
> >>>
> >>> It looks like it is breaking a test:
> >>>
> >>> 2025-09-04 15:28:11,112 - oe-selftest - INFO -
> wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
> >>> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
> >>> ...
> >>> 2025-09-04 15:28:11,114 - oe-selftest - INFO -
> testtools.testresult.real._StringException: Traceback (most recent call
> last):
> >>>   File
> "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py",
> line 859, in test_no_fstab_update
> >>>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
> >>>   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
> >>>     assertion_func(first, second, msg=msg)
> >>>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in
> assertMultiLineEqual
> >>>     self.fail(self._formatMessage(msg, standardMsg))
> >>>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
> >>>     raise self.failureException(msg)
> >>> AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=
> '9edb8255abd217fdb20e118833afb856'
> >>> - af3c087d6c9131735c8d1f270a226892
> >>> + 9edb8255abd217fdb20e118833afb856
> >>>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
> >>>
> >>> Can you fix it please?
> >>>
> >>> Ping?
> >>>
> >>> I think this is being tracked by:
> >>>
> >>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
> >>>
> >>> "WIC does not preserve metadata when updating fstab"
> >>>
> >>> Btw, we were just following the "Need Info" process during the bug
> review
> >>> meeting
> >>> so that's why I'm sending this email.
> >>>
> >>> ../Randy
> >>>
> >>> Thanks,
> >>> Mathieu
> >>>
> >>>
> >>>
> >>> 
> >>>
> >>>
> >>>
> >>> --
> >>> # Randy MacLeod
> >>> # Wind River Linux
> >>>
> >>>
>
>
>
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>

[-- Attachment #2: Type: text/html, Size: 9977 bytes --]

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Mathieu Dubois-Briand]

OK! So I misunderstood your previous mail.

I confirm that on master branch the test succeed, but fails if I change
the filesystem to squashfs or btrfs.

On a first glance, it looks like you are right, but I can't say I am
experienced with wic internals. We might want a second opinion here.

If the test is wrong, it has to be fixed. And so probably the tested
code also have to be fixed. I this something you can do?

Thanks,
Mathieu

On Wed Sep 24, 2025 at 6:04 PM CEST, Daniel Andrade wrote:
> Hello Mathieu,
>
> Thank you for the quick answer.
> Ok, I will start from the beginning.
>
> The failing test is supposed to test `--no-fstab-update` flags on wks file.
> According to the understanding of the code, the flag is being passed across
> python scripts either as `self.no_fstab_update`, like in
> scripts/lib/wic/partition.py, or `part.no_fstab_update`, on
> scripts/lib/wic/plugins/source/rootfs.py.
> The issue that is arising from the `oe-selftest -r
> wic.Wic.test_no_fstab_update` is that it is not actually testing it,
> because somehow the assigning of the previous variables is not correct,
> which leads to a later assumption.
> The thing that is really doing the "no update" is the line 134 of
> `scripts/lib/wic/partition.py` where it excludes ext* and msdos partitions
> from falling in the common plugin for Rootfs.
> Because of this, I suggested someone to, in a clean and a different
> environment compared to mine, change the wks file used to `squashfs` fstype
> (or any other supported wic fstype) instead of `ext4` which indeed also
> made the test fail.
> In summary, the test is failing not because of the patch (even though maybe
> something can be improved or fixed later), but because it is just checking
> for ext4 partitions that fall in a different way of approaching fstab while
> the real mechanism is not behaving as desired.
>
> TLDR: The test is getting passed not because it is enforcing
> `--no-fstab-update` but because the partition is ext* or msdos. Changing it
> to squashfs, btrfs or any other wic supported partition, with or without
> the patch, will cause this to fail. This is an extra thing non-related with
> the proposed patch but that is preventing it to behave correctly as
> intended by the test.
>
> Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> escreveu (terça,
> 23/09/2025 à(s) 19:44):
>
>> Hi Daniel,
>>
>> Sorry, but I'm not sure to get your point.
>>
>> I ran the test locally, and it does fail. It also fails if I modify it
>> to use squashfs instead of ext4.
>>
>> My reproduction procedure:
>> git clone https://git.yoctoproject.org/poky-ci-archive -b
>> autobuilder.yoctoproject.org/valkyrie/a-full-2320
>> . oe-init-build-env
>> echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
>> oe-selftest -r wic.Wic.test_no_fstab_update
>>
>> Thanks,
>> Mathieu
>>
>> On Tue Sep 23, 2025 at 5:05 PM CEST, Daniel Andrade wrote:
>> > Good Afternoon,
>> >
>> > I have been trying to figure it out and something seems off.
>> > The specific test you refer to may be hiding a problem or maybe I have my
>> > testbench compromised and I would like your help to test.
>> > The test `test_no_fstab_update ` verifies if the fstab was updated or not
>> > because on the wks file used to do it there is a --no-fstab-update. The
>> > thing is, I think this test is passing because what is forcing it to not
>> > update is a constraint that I removed previously on the patch (the one
>> that
>> > checks if the system is ext* or msdos) and not because the flag is
>> > triggering its intended functionality.
>> > The crosscheck I did was modifying the fstype from ext4 to squashfs and
>> > indeed the test failed.
>> > Can someone also verify this?
>> >
>> > Daniel
>> >
>> > Daniel Andrade <dani.barra25@gmail.com> escreveu (sábado, 13/09/2025
>> à(s)
>> > 11:02):
>> >
>> >> Hello guys, sorry for the delay.
>> >> I have been busy, but I hope to provide you with a fix in the next few
>> >> days.
>> >>
>> >> Sorry about that,
>> >> Daniel
>> >>
>> >> Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta,
>> 11/09/2025
>> >> à(s) 17:56):
>> >>
>> >>> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via
>> >>> lists.openembedded.org wrote:
>> >>>
>> >>> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via
>> lists.openembedded.org wrote:
>> >>>
>> >>> From: Daniel Andrade <dani.barra25@gmail.com> <dani.barra25@gmail.com>
>> >>>
>> >>> Using `install` in the rootfs plugin forces fstab to be replaced
>> entirely, meaning that even its Inodes will change, leading xattrs and
>> SELinux context stored by pseudo not to be applied.
>> >>> The fix just uses `cp` without preserving attributes from the
>> temporary fstab since none of them are needed, just the content.
>> >>> Same thing happens with the predefined mechanisms for ext4 and msdos.
>> Using debugfs there is no way to replace contents while maintaining
>> metadata, so the approach taken on the path was to remove the different
>> fstab logic for those fstypes and also use the same modified cp command.
>> Reviewing the builds I did it seems to work for all of the fstypes.
>> >>>
>> >>> Another problem is that the timestamp applied to fstab is not the same
>> as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to
>> the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
>> >>> Since you are using that variable everywhere, it is not the same value
>> as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf`
>> that is applied in every other file.
>> >>>
>> >>> Daniel Andrade (1):
>> >>>   wic: Content of the temporary updated fstab should be copied into the
>> >>>     original not replacing it entirely.
>> >>>
>> >>>  meta/conf/bitbake.conf                   |  4 +++-
>> >>>  scripts/lib/wic/partition.py             | 15 +--------------
>> >>>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>> >>>  3 files changed, 6 insertions(+), 17 deletions(-)
>> >>>
>> >>> Hi Daniel,
>> >>>
>> >>> Thanks for your patch.
>> >>>
>> >>> It looks like it is breaking a test:
>> >>>
>> >>> 2025-09-04 15:28:11,112 - oe-selftest - INFO -
>> wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
>> >>> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
>> >>> ...
>> >>> 2025-09-04 15:28:11,114 - oe-selftest - INFO -
>> testtools.testresult.real._StringException: Traceback (most recent call
>> last):
>> >>>   File
>> "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py",
>> line 859, in test_no_fstab_update
>> >>>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
>> >>>     assertion_func(first, second, msg=msg)
>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in
>> assertMultiLineEqual
>> >>>     self.fail(self._formatMessage(msg, standardMsg))
>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
>> >>>     raise self.failureException(msg)
>> >>> AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=
>> '9edb8255abd217fdb20e118833afb856'
>> >>> - af3c087d6c9131735c8d1f270a226892
>> >>> + 9edb8255abd217fdb20e118833afb856
>> >>>
>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
>> >>>
>> >>> Can you fix it please?
>> >>>
>> >>> Ping?
>> >>>
>> >>> I think this is being tracked by:
>> >>>
>> >>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
>> >>>
>> >>> "WIC does not preserve metadata when updating fstab"
>> >>>
>> >>> Btw, we were just following the "Need Info" process during the bug
>> review
>> >>> meeting
>> >>> so that's why I'm sending this email.
>> >>>
>> >>> ../Randy
>> >>>
>> >>> Thanks,
>> >>> Mathieu
>> >>>
>> >>>
>> >>>
>> >>> -=-=-=-=-=-=-=-=-=-=-=-
>> >>> Links: You receive all messages sent to this group.
>> >>> View/Reply Online (#222962):
>> https://lists.openembedded.org/g/openembedded-core/message/222962
>> >>> Mute This Topic: https://lists.openembedded.org/mt/115043355/3616765
>> >>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> >>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>> [randy.macleod@windriver.com]
>> >>> -=-=-=-=-=-=-=-=-=-=-=-
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> # Randy MacLeod
>> >>> # Wind River Linux
>> >>>
>> >>>
>>
>>
>>
>>
>> --
>> Mathieu Dubois-Briand, Bootlin
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
>>
>>




-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Mathieu Dubois-Briand]

Hi Trevor,

I'm not sure you have seen the following mail thread. It was suggested
we seek for your advices during today patch review call.

Do you have any opinion about this test, and if the behaviour we are
seeing is expected?

Thanks,
Mathieu

On Thu Sep 25, 2025 at 10:57 AM CEST, Mathieu Dubois-Briand wrote:
> OK! So I misunderstood your previous mail.
>
> I confirm that on master branch the test succeed, but fails if I change
> the filesystem to squashfs or btrfs.
>
> On a first glance, it looks like you are right, but I can't say I am
> experienced with wic internals. We might want a second opinion here.
>
> If the test is wrong, it has to be fixed. And so probably the tested
> code also have to be fixed. I this something you can do?
>
> Thanks,
> Mathieu
>
> On Wed Sep 24, 2025 at 6:04 PM CEST, Daniel Andrade wrote:
>> Hello Mathieu,
>>
>> Thank you for the quick answer.
>> Ok, I will start from the beginning.
>>
>> The failing test is supposed to test `--no-fstab-update` flags on wks file.
>> According to the understanding of the code, the flag is being passed across
>> python scripts either as `self.no_fstab_update`, like in
>> scripts/lib/wic/partition.py, or `part.no_fstab_update`, on
>> scripts/lib/wic/plugins/source/rootfs.py.
>> The issue that is arising from the `oe-selftest -r
>> wic.Wic.test_no_fstab_update` is that it is not actually testing it,
>> because somehow the assigning of the previous variables is not correct,
>> which leads to a later assumption.
>> The thing that is really doing the "no update" is the line 134 of
>> `scripts/lib/wic/partition.py` where it excludes ext* and msdos partitions
>> from falling in the common plugin for Rootfs.
>> Because of this, I suggested someone to, in a clean and a different
>> environment compared to mine, change the wks file used to `squashfs` fstype
>> (or any other supported wic fstype) instead of `ext4` which indeed also
>> made the test fail.
>> In summary, the test is failing not because of the patch (even though maybe
>> something can be improved or fixed later), but because it is just checking
>> for ext4 partitions that fall in a different way of approaching fstab while
>> the real mechanism is not behaving as desired.
>>
>> TLDR: The test is getting passed not because it is enforcing
>> `--no-fstab-update` but because the partition is ext* or msdos. Changing it
>> to squashfs, btrfs or any other wic supported partition, with or without
>> the patch, will cause this to fail. This is an extra thing non-related with
>> the proposed patch but that is preventing it to behave correctly as
>> intended by the test.
>>
>> Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> escreveu (terça,
>> 23/09/2025 à(s) 19:44):
>>
>>> Hi Daniel,
>>>
>>> Sorry, but I'm not sure to get your point.
>>>
>>> I ran the test locally, and it does fail. It also fails if I modify it
>>> to use squashfs instead of ext4.
>>>
>>> My reproduction procedure:
>>> git clone https://git.yoctoproject.org/poky-ci-archive -b
>>> autobuilder.yoctoproject.org/valkyrie/a-full-2320
>>> . oe-init-build-env
>>> echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
>>> oe-selftest -r wic.Wic.test_no_fstab_update
>>>
>>> Thanks,
>>> Mathieu
>>>
>>> On Tue Sep 23, 2025 at 5:05 PM CEST, Daniel Andrade wrote:
>>> > Good Afternoon,
>>> >
>>> > I have been trying to figure it out and something seems off.
>>> > The specific test you refer to may be hiding a problem or maybe I have my
>>> > testbench compromised and I would like your help to test.
>>> > The test `test_no_fstab_update ` verifies if the fstab was updated or not
>>> > because on the wks file used to do it there is a --no-fstab-update. The
>>> > thing is, I think this test is passing because what is forcing it to not
>>> > update is a constraint that I removed previously on the patch (the one
>>> that
>>> > checks if the system is ext* or msdos) and not because the flag is
>>> > triggering its intended functionality.
>>> > The crosscheck I did was modifying the fstype from ext4 to squashfs and
>>> > indeed the test failed.
>>> > Can someone also verify this?
>>> >
>>> > Daniel
>>> >
>>> > Daniel Andrade <dani.barra25@gmail.com> escreveu (sábado, 13/09/2025
>>> à(s)
>>> > 11:02):
>>> >
>>> >> Hello guys, sorry for the delay.
>>> >> I have been busy, but I hope to provide you with a fix in the next few
>>> >> days.
>>> >>
>>> >> Sorry about that,
>>> >> Daniel
>>> >>
>>> >> Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta,
>>> 11/09/2025
>>> >> à(s) 17:56):
>>> >>
>>> >>> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via
>>> >>> lists.openembedded.org wrote:
>>> >>>
>>> >>> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via
>>> lists.openembedded.org wrote:
>>> >>>
>>> >>> From: Daniel Andrade <dani.barra25@gmail.com> <dani.barra25@gmail.com>
>>> >>>
>>> >>> Using `install` in the rootfs plugin forces fstab to be replaced
>>> entirely, meaning that even its Inodes will change, leading xattrs and
>>> SELinux context stored by pseudo not to be applied.
>>> >>> The fix just uses `cp` without preserving attributes from the
>>> temporary fstab since none of them are needed, just the content.
>>> >>> Same thing happens with the predefined mechanisms for ext4 and msdos.
>>> Using debugfs there is no way to replace contents while maintaining
>>> metadata, so the approach taken on the path was to remove the different
>>> fstab logic for those fstypes and also use the same modified cp command.
>>> Reviewing the builds I did it seems to work for all of the fstypes.
>>> >>>
>>> >>> Another problem is that the timestamp applied to fstab is not the same
>>> as every other file. It seems like the `SOURCE_DATE_EPOCH` variable goes to
>>> the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
>>> >>> Since you are using that variable everywhere, it is not the same value
>>> as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf`
>>> that is applied in every other file.
>>> >>>
>>> >>> Daniel Andrade (1):
>>> >>>   wic: Content of the temporary updated fstab should be copied into the
>>> >>>     original not replacing it entirely.
>>> >>>
>>> >>>  meta/conf/bitbake.conf                   |  4 +++-
>>> >>>  scripts/lib/wic/partition.py             | 15 +--------------
>>> >>>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
>>> >>>  3 files changed, 6 insertions(+), 17 deletions(-)
>>> >>>
>>> >>> Hi Daniel,
>>> >>>
>>> >>> Thanks for your patch.
>>> >>>
>>> >>> It looks like it is breaking a test:
>>> >>>
>>> >>> 2025-09-04 15:28:11,112 - oe-selftest - INFO -
>>> wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
>>> >>> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
>>> >>> ...
>>> >>> 2025-09-04 15:28:11,114 - oe-selftest - INFO -
>>> testtools.testresult.real._StringException: Traceback (most recent call
>>> last):
>>> >>>   File
>>> "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py",
>>> line 859, in test_no_fstab_update
>>> >>>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
>>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
>>> >>>     assertion_func(first, second, msg=msg)
>>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in
>>> assertMultiLineEqual
>>> >>>     self.fail(self._formatMessage(msg, standardMsg))
>>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
>>> >>>     raise self.failureException(msg)
>>> >>> AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=
>>> '9edb8255abd217fdb20e118833afb856'
>>> >>> - af3c087d6c9131735c8d1f270a226892
>>> >>> + 9edb8255abd217fdb20e118833afb856
>>> >>>
>>> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
>>> >>>
>>> >>> Can you fix it please?
>>> >>>
>>> >>> Ping?
>>> >>>
>>> >>> I think this is being tracked by:
>>> >>>
>>> >>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
>>> >>>
>>> >>> "WIC does not preserve metadata when updating fstab"
>>> >>>
>>> >>> Btw, we were just following the "Need Info" process during the bug
>>> review
>>> >>> meeting
>>> >>> so that's why I'm sending this email.
>>> >>>
>>> >>> ../Randy
>>> >>>
>>> >>> Thanks,
>>> >>> Mathieu
>>> >>>
>>> >>>
>>> >>>
>>> >>> -=-=-=-=-=-=-=-=-=-=-=-
>>> >>> Links: You receive all messages sent to this group.
>>> >>> View/Reply Online (#222962):
>>> https://lists.openembedded.org/g/openembedded-core/message/222962
>>> >>> Mute This Topic: https://lists.openembedded.org/mt/115043355/3616765
>>> >>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> >>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>>> [randy.macleod@windriver.com]
>>> >>> -=-=-=-=-=-=-=-=-=-=-=-
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> # Randy MacLeod
>>> >>> # Wind River Linux
>>> >>>
>>> >>>
>>>
>>>
>>>
>>>
>>> --
>>> Mathieu Dubois-Briand, Bootlin
>>> Embedded Linux and Kernel engineering
>>> https://bootlin.com
>>>
>>>




-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 0/1] wic: updated fstab does not preserve metadata of the original file

[Daniel Andrade]

[-- Attachment #1: Type: text/plain, Size: 10009 bytes --]

Hi Trevor and Mathieu,

If, by any chance, you need more information, goal or objective with what I
am trying to accomplish, or any other thing you might find useful, please
let me know.
I am available to provide more explanation, either via this thread or via a
separate email or meeting.
For the moment I will halt the research/fixing I was looking/doing until
you provide further confirmation on the behaviour of that mechanism.

Have a nice day,
Daniel

Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> escreveu (quinta,
25/09/2025 à(s) 12:11):

> Hi Trevor,
>
> I'm not sure you have seen the following mail thread. It was suggested
> we seek for your advices during today patch review call.
>
> Do you have any opinion about this test, and if the behaviour we are
> seeing is expected?
>
> Thanks,
> Mathieu
>
> On Thu Sep 25, 2025 at 10:57 AM CEST, Mathieu Dubois-Briand wrote:
> > OK! So I misunderstood your previous mail.
> >
> > I confirm that on master branch the test succeed, but fails if I change
> > the filesystem to squashfs or btrfs.
> >
> > On a first glance, it looks like you are right, but I can't say I am
> > experienced with wic internals. We might want a second opinion here.
> >
> > If the test is wrong, it has to be fixed. And so probably the tested
> > code also have to be fixed. I this something you can do?
> >
> > Thanks,
> > Mathieu
> >
> > On Wed Sep 24, 2025 at 6:04 PM CEST, Daniel Andrade wrote:
> >> Hello Mathieu,
> >>
> >> Thank you for the quick answer.
> >> Ok, I will start from the beginning.
> >>
> >> The failing test is supposed to test `--no-fstab-update` flags on wks
> file.
> >> According to the understanding of the code, the flag is being passed
> across
> >> python scripts either as `self.no_fstab_update`, like in
> >> scripts/lib/wic/partition.py, or `part.no_fstab_update`, on
> >> scripts/lib/wic/plugins/source/rootfs.py.
> >> The issue that is arising from the `oe-selftest -r
> >> wic.Wic.test_no_fstab_update` is that it is not actually testing it,
> >> because somehow the assigning of the previous variables is not correct,
> >> which leads to a later assumption.
> >> The thing that is really doing the "no update" is the line 134 of
> >> `scripts/lib/wic/partition.py` where it excludes ext* and msdos
> partitions
> >> from falling in the common plugin for Rootfs.
> >> Because of this, I suggested someone to, in a clean and a different
> >> environment compared to mine, change the wks file used to `squashfs`
> fstype
> >> (or any other supported wic fstype) instead of `ext4` which indeed also
> >> made the test fail.
> >> In summary, the test is failing not because of the patch (even though
> maybe
> >> something can be improved or fixed later), but because it is just
> checking
> >> for ext4 partitions that fall in a different way of approaching fstab
> while
> >> the real mechanism is not behaving as desired.
> >>
> >> TLDR: The test is getting passed not because it is enforcing
> >> `--no-fstab-update` but because the partition is ext* or msdos.
> Changing it
> >> to squashfs, btrfs or any other wic supported partition, with or without
> >> the patch, will cause this to fail. This is an extra thing non-related
> with
> >> the proposed patch but that is preventing it to behave correctly as
> >> intended by the test.
> >>
> >> Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> escreveu
> (terça,
> >> 23/09/2025 à(s) 19:44):
> >>
> >>> Hi Daniel,
> >>>
> >>> Sorry, but I'm not sure to get your point.
> >>>
> >>> I ran the test locally, and it does fail. It also fails if I modify it
> >>> to use squashfs instead of ext4.
> >>>
> >>> My reproduction procedure:
> >>> git clone https://git.yoctoproject.org/poky-ci-archive -b
> >>> autobuilder.yoctoproject.org/valkyrie/a-full-2320
> >>> . oe-init-build-env
> >>> echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
> >>> oe-selftest -r wic.Wic.test_no_fstab_update
> >>>
> >>> Thanks,
> >>> Mathieu
> >>>
> >>> On Tue Sep 23, 2025 at 5:05 PM CEST, Daniel Andrade wrote:
> >>> > Good Afternoon,
> >>> >
> >>> > I have been trying to figure it out and something seems off.
> >>> > The specific test you refer to may be hiding a problem or maybe I
> have my
> >>> > testbench compromised and I would like your help to test.
> >>> > The test `test_no_fstab_update ` verifies if the fstab was updated
> or not
> >>> > because on the wks file used to do it there is a --no-fstab-update.
> The
> >>> > thing is, I think this test is passing because what is forcing it to
> not
> >>> > update is a constraint that I removed previously on the patch (the
> one
> >>> that
> >>> > checks if the system is ext* or msdos) and not because the flag is
> >>> > triggering its intended functionality.
> >>> > The crosscheck I did was modifying the fstype from ext4 to squashfs
> and
> >>> > indeed the test failed.
> >>> > Can someone also verify this?
> >>> >
> >>> > Daniel
> >>> >
> >>> > Daniel Andrade <dani.barra25@gmail.com> escreveu (sábado, 13/09/2025
> >>> à(s)
> >>> > 11:02):
> >>> >
> >>> >> Hello guys, sorry for the delay.
> >>> >> I have been busy, but I hope to provide you with a fix in the next
> few
> >>> >> days.
> >>> >>
> >>> >> Sorry about that,
> >>> >> Daniel
> >>> >>
> >>> >> Randy MacLeod <randy.macleod@windriver.com> escreveu (quinta,
> >>> 11/09/2025
> >>> >> à(s) 17:56):
> >>> >>
> >>> >>> On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via
> >>> >>> lists.openembedded.org wrote:
> >>> >>>
> >>> >>> On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via
> >>> lists.openembedded.org wrote:
> >>> >>>
> >>> >>> From: Daniel Andrade <dani.barra25@gmail.com> <
> dani.barra25@gmail.com>
> >>> >>>
> >>> >>> Using `install` in the rootfs plugin forces fstab to be replaced
> >>> entirely, meaning that even its Inodes will change, leading xattrs and
> >>> SELinux context stored by pseudo not to be applied.
> >>> >>> The fix just uses `cp` without preserving attributes from the
> >>> temporary fstab since none of them are needed, just the content.
> >>> >>> Same thing happens with the predefined mechanisms for ext4 and
> msdos.
> >>> Using debugfs there is no way to replace contents while maintaining
> >>> metadata, so the approach taken on the path was to remove the different
> >>> fstab logic for those fstypes and also use the same modified cp
> command.
> >>> Reviewing the builds I did it seems to work for all of the fstypes.
> >>> >>>
> >>> >>> Another problem is that the timestamp applied to fstab is not the
> same
> >>> as every other file. It seems like the `SOURCE_DATE_EPOCH` variable
> goes to
> >>> the fallback timestamp (`SOURCE_DATE_EPOCH_FALLBACK`).
> >>> >>> Since you are using that variable everywhere, it is not the same
> value
> >>> as ` REPRODUCIBLE_TIMESTAMP_ROOTFS` under `poky/meta/conf/bitbake.conf`
> >>> that is applied in every other file.
> >>> >>>
> >>> >>> Daniel Andrade (1):
> >>> >>>   wic: Content of the temporary updated fstab should be copied
> into the
> >>> >>>     original not replacing it entirely.
> >>> >>>
> >>> >>>  meta/conf/bitbake.conf                   |  4 +++-
> >>> >>>  scripts/lib/wic/partition.py             | 15 +--------------
> >>> >>>  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
> >>> >>>  3 files changed, 6 insertions(+), 17 deletions(-)
> >>> >>>
> >>> >>> Hi Daniel,
> >>> >>>
> >>> >>> Thanks for your patch.
> >>> >>>
> >>> >>> It looks like it is breaking a test:
> >>> >>>
> >>> >>> 2025-09-04 15:28:11,112 - oe-selftest - INFO -
> >>> wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
> >>> >>> 2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
> >>> >>> ...
> >>> >>> 2025-09-04 15:28:11,114 - oe-selftest - INFO -
> >>> testtools.testresult.real._StringException: Traceback (most recent call
> >>> last):
> >>> >>>   File
> >>>
> "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py",
> >>> line 859, in test_no_fstab_update
> >>> >>>     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
> >>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 885, in
> assertEqual
> >>> >>>     assertion_func(first, second, msg=msg)
> >>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 1251, in
> >>> assertMultiLineEqual
> >>> >>>     self.fail(self._formatMessage(msg, standardMsg))
> >>> >>>   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
> >>> >>>     raise self.failureException(msg)
> >>> >>> AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=
> >>> '9edb8255abd217fdb20e118833afb856'
> >>> >>> - af3c087d6c9131735c8d1f270a226892
> >>> >>> + 9edb8255abd217fdb20e118833afb856
> >>> >>>
> >>>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268
> >>> >>>
> >>> >>> Can you fix it please?
> >>> >>>
> >>> >>> Ping?
> >>> >>>
> >>> >>> I think this is being tracked by:
> >>> >>>
> >>> >>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947
> >>> >>>
> >>> >>> "WIC does not preserve metadata when updating fstab"
> >>> >>>
> >>> >>> Btw, we were just following the "Need Info" process during the bug
> >>> review
> >>> >>> meeting
> >>> >>> so that's why I'm sending this email.
> >>> >>>
> >>> >>> ../Randy
> >>> >>>
> >>> >>> Thanks,
> >>> >>> Mathieu
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> 
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> # Randy MacLeod
> >>> >>> # Wind River Linux
> >>> >>>
> >>> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Mathieu Dubois-Briand, Bootlin
> >>> Embedded Linux and Kernel engineering
> >>> https://bootlin.com
> >>>
> >>>
>
>
>
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>

[-- Attachment #2: Type: text/html, Size: 14562 bytes --]

Re: [OE-core] [PATCH 1/1] wic: Content of the temporary updated fstab should be copied into the original not replacing it entirely.

[Mathieu Dubois-Briand]

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:
> From: Daniel Andrade <dani.barra25@gmail.com>
>
> Fixes [15947]
> The current functionality to update fstab generates a new temporary fstab with the new partition configuration.
> However, this file does not retain any metadata being it a completely new file.
> Because of this, when the rootfs plugin under `poky/scripts/lib/wic/plugins/source/rootfs.py` copies the file, it overrides the original fstab metadata.
> The patch removes implementation of msdos/ext code for fstab since it is not possible to append content without rewriting the file to preserve metadata.
>
> The timestamp applied to fstab is not the same as every other file.
> It seems like the `SOURCE_DATE_EPOCH` variable goes to the fallback timestamp SOURCE_DATE_EPOCH_FALLBACK.
> Since that variable is used everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.
>
> Signed-off-by: Daniel Andrade <dani.barra25@gmail.com>
> ---

Hi Daniel,

I finally had a bit of time to have a deeper look at these changes and
at the wic.Wic.test_no_fstab_update test. It turns out the test is
correct and your patch is indeed introducing a regression.

So our first assumption was the test was wrong, as changing the
filesystem resulted in a test fail. But actually, if you change the
filesystem in the wic command file, you also have to change the command
extracting the file from the filesystem. E.g. for squashfs, you need to
replace "debugfs -R cat" with "sqfscat".

Now talking about your patch, my comments below.

> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index bf2c34d594..82d754835c 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -131,7 +131,7 @@ class Partition():
>          partition command parameters.
>          """
>          self.updated_fstab_path = updated_fstab_path
> -        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
> +        if self.updated_fstab_path:
>              self.update_fstab_in_rootfs = True
>  
>          if not self.source:
> @@ -295,15 +295,6 @@ class Partition():
>              (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
>          exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>  
> -        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
> -            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
> -            with open(debugfs_script_path, "w") as f:
> -                f.write("cd etc\n")
> -                f.write("rm fstab\n")
> -                f.write("write %s fstab\n" % (self.updated_fstab_path))
> -            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
> -            exec_native_cmd(debugfs_cmd, native_sysroot)
> -
>          mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
>          exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>  
> @@ -400,10 +391,6 @@ class Partition():
>          mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
>          exec_native_cmd(mcopy_cmd, native_sysroot)
>  
> -        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
> -            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
> -            exec_native_cmd(mcopy_cmd, native_sysroot)
> -
>          chmod_cmd = "chmod 644 %s" % rootfs
>          exec_cmd(chmod_cmd)
>  

So you are removing an optimisation on ext and fat filesystems. This
might raise further questions from other reviewers, but if that's
needed, why not.

> diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
> index e29f3a4c2f..3848af2a91 100644
> --- a/scripts/lib/wic/plugins/source/rootfs.py
> +++ b/scripts/lib/wic/plugins/source/rootfs.py
> @@ -223,8 +223,8 @@ class RootfsPlugin(SourcePlugin):
>              part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab"))
>              if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update:
>                  fstab_path = os.path.join(new_rootfs, "etc/fstab")
> -                # Assume that fstab should always be owned by root with fixed permissions
> -                install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
> +                # We dont want any metadata of the updated fstab, just the one that already existed
> +                install_cmd = "cp --no-preserve=all %s %s" % (part.updated_fstab_path, fstab_path)

Now this is what makes the test fail. So I understand the switch from
install to cp, as it will reuse the same inode instead of creating a new
one. But this is also what causes the test regression.

When creating a filesystem image with a modified fstab, a copy of the
filesystem content is first made, so file modifications only affect this
specific image. But in order to gain time and disk space, this copy is
made using hardlinks: see use of copyhardlinktree() in
do_prepare_partition() from scripts/lib/wic/plugins/source/rootfs.py. So
as cp will keep the same inode, it means you will modify the content on
both the copy and the reference, and so all further generated image.

So I'm sorry, we will have to find another way to implement this change.

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 1/1] wic: Content of the temporary updated fstab should be copied into the original not replacing it entirely.

[dani.barra25]

Hi Mathieu,

Thank you for the you time testing this thoroughly.
After some test, I was able to provide a new alternative (attached to this message if everything goes well).

For the performance part of ext* and msdos fstype I have nothing to counter argument. I did not find a way to add xattrs using those methods and, according to what seemed to be the natural way of
the code design of wic, I let all the fstype flal under rootfs.py.  Any other solution is welcome!

For the second part, this time, instead of using cp, I let the install be like it was before. However, instead of running it under the pseudo environment, I executed it as the host machine.
This way, I was able to recheck the database using pseudo, which will fail and therefore led the files.db to be rebuilt. This happens because he is able to detect the inodes of fstab entry on all
the tables do not match and update them accordingly. This way we don't run the risk of modifying the original content of fstab under the normal yocto output files because the file is actually not the same.

I also ran oe-selftest -r wic.Wic.test_no_fstab_update on a core-image-minimal image with the patch and it seemed to have passed.

Best Regards,
Daniel

[PATCH 1/1] Fstab: Fix xattrs not being maintained on fstab file when using wic fstab update funtionalities

[dani.barra25]

From: Daniel Andrade <dani.barra25@gmail.com>

The wic fstab generation capability was removing the xattrs of the /etc/fstab, possibly rendering the produced image unusable in some case (e.g.: SELinux enforced on a read-only rootfs).
The file appears as unlabeled no matter the fstype chosen.
To bypass this, some late procedures done on ext* and msdos fstype have been removed and all the fstab logic was passed to rootfs.py.
On this one, the idea is to still use the install command, but in this case, executed outside the pseudo environment. After the file as sucessfully been place, the idea was to check again for inconsistencies
in the database (now the fstab inode is different) and force it to rebuild with the correct inode, ensure fstab inode is correctly set on all the tables, including the xattrs one.

Signed-off-by: Daniel Andrade <dani.barra25@gmail.com>
---
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  8 ++++----
 2 files changed, 5 insertions(+), 18 deletions(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index bf2c34d594..82d754835c 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index e29f3a4c2f..c0d9a28b91 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -225,11 +225,11 @@ class RootfsPlugin(SourcePlugin):
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
                 # Assume that fstab should always be owned by root with fixed permissions
                 install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                exec_native_cmd(install_cmd, native_sysroot)
                 if new_pseudo:
-                    pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
-                else:
-                    pseudo = None
-                exec_native_cmd(install_cmd, native_sysroot, pseudo)
+                    pseudo_cmd = "%s -B " % (cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo))
+                    exec_native_cmd(pseudo_cmd, native_sysroot)
+
 
         part.prepare_rootfs(cr_workdir, oe_builddir,
                             new_rootfs or part.rootfs_dir, native_sysroot,

Re: [PATCH 1/1] Fstab: Fix xattrs not being maintained on fstab file when using wic fstab update funtionalities

[Mathieu Dubois-Briand]

On Fri Oct 17, 2025 at 1:04 PM CEST, dani.barra25 wrote:
> From: Daniel Andrade <dani.barra25@gmail.com>
>
> The wic fstab generation capability was removing the xattrs of the /etc/fstab, possibly rendering the produced image unusable in some case (e.g.: SELinux enforced on a read-only rootfs).
> The file appears as unlabeled no matter the fstype chosen.
> To bypass this, some late procedures done on ext* and msdos fstype have been removed and all the fstab logic was passed to rootfs.py.
> On this one, the idea is to still use the install command, but in this case, executed outside the pseudo environment. After the file as sucessfully been place, the idea was to check again for inconsistencies
> in the database (now the fstab inode is different) and force it to rebuild with the correct inode, ensure fstab inode is correctly set on all the tables, including the xattrs one.
>
> Signed-off-by: Daniel Andrade <dani.barra25@gmail.com>
> ---

Hi Daniel,

Thanks for the new version. It looks like this one is now breaking
wic.Wic.test_exclude_path and wic.Wic.test_include_path tests:

2025-10-19 07:55:18,010 - oe-selftest - INFO - wic.Wic.test_exclude_path (subunit.RemotedTestCase)
2025-10-19 07:55:18,011 - oe-selftest - INFO -  ... FAIL
...
AssertionError: Command 'wic create temp.wks -e core-image-minimal -o /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp' returned non-zero exit status 1:
...
ERROR: _exec_cmd: export PATH=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/hosttools:$PATH;export PSEUDO_PREFIX=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr;export PSEUDO_LOCALSTATEDIR=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/pseudo1;export PSEUDO_PASSWD=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/rootfs1;export PSEUDO_NOSYMLINKEXP=1;/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/sysroots-components/x86_64/pseudo-native/usr/bin/pseudo  -B  returned '1' instead of 0
output: PRAGMA journal_mode = OFF;: database is locked
PRAGMA synchronous = OFF;: database is locked
Failed: database is locked
Error getting 'logs.db' database.
PRAGMA journal_mode = OFF;: database is locked
PRAGMA synchronous = OFF;: database is locked
Failed: database is locked
Error getting 'files.db' database.
error during load from disk: database is locked
pdb_files: database error.
Couldn't start file list, can't scan.
db cleanup for server shutdown, 07:55:17.956
memory-to-file backup complete, 07:55:17.956.
db cleanup finished, 07:55:17.956
considering table files
considering table xattrs
considering table migrations

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2574

Can you have a look at these errors?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

[dani.barra25]

From: Daniel Andrade <dani.barra25@gmail.com>

The process of repairing the database using -B flag of pseudo sometimes causes errors because the database lock file is present.
Therefore, to fix it, first we ensure that any connection to files.db is closed using pseudo flag -S followed by the actual command to
repair it after fstab was successfully updated.

Signed-off-by: Daniel Andrade <dani.barra25@gmail.com>
---
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py | 19 ++++++++++++++-----
 2 files changed, 15 insertions(+), 19 deletions(-)

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index bf2c34d594..82d754835c 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index e29f3a4c2f..09446baef2 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -225,12 +225,21 @@ class RootfsPlugin(SourcePlugin):
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
                 # Assume that fstab should always be owned by root with fixed permissions
                 install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                exec_native_cmd(install_cmd, native_sysroot)
+ 
                 if new_pseudo:
-                    pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
-                else:
-                    pseudo = None
-                exec_native_cmd(install_cmd, native_sysroot, pseudo)
-
+                    pseudo_prefix = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
+                   
+                    # Shutdown any existing pseudo server
+                    # This ensures no other processes are using the database
+                    shutdown_cmd = "%s -S" % pseudo_prefix
+                    exec_native_cmd(shutdown_cmd, native_sysroot)
+                   
+                    # Database is not locked anymore
+                    # Repairs inodes related to fstab. This way xattrs and metadata is correctly applied to the new file
+                    repair_cmd = "%s -B" % pseudo_prefix
+                    exec_native_cmd(repair_cmd, native_sysroot)
+ 
         part.prepare_rootfs(cr_workdir, oe_builddir,
                             new_rootfs or part.rootfs_dir, native_sysroot,
                             pseudo_dir = new_pseudo or pseudo_dir)

Re: [PATCH 1/1] Fstab: Fix xattrs not being maintained on fstab file when using wic fstab update funtionalities

[Daniel Andrade]

[-- Attachment #1: Type: text/plain, Size: 4798 bytes --]

Hi Mathieu,

I did not notice that it was failing other tests. It seemed like the error
was caused due to a lock file. To fix it, before issuing the repair of the
database, I first shutdown any open connection using -S. Doing this I saw
that the 3 tests passed and fstab was correctly labeled.
The other thing I noticed was the different timestamps, but as I said a
while ago, it was caused due to using SOURCE_DATE_EPOCH,

REPRODUCIBLE_TIMESTAMP_ROOTFS, and SOURCE_DATE_EPOCH_FALLBACK. This should
also be reviewed since it might compromise build repeatability.

In any case, it was not the topic of the patch, I just wanted to state this
in case someone checked the contents of the produced image.

Best Regards,
Daniel

A domingo, 19/10/2025, 11:02, Mathieu Dubois-Briand <
mathieu.dubois-briand@bootlin.com> escreveu:

> On Fri Oct 17, 2025 at 1:04 PM CEST, dani.barra25 wrote:
> > From: Daniel Andrade <dani.barra25@gmail.com>
> >
> > The wic fstab generation capability was removing the xattrs of the
> /etc/fstab, possibly rendering the produced image unusable in some case
> (e.g.: SELinux enforced on a read-only rootfs).
> > The file appears as unlabeled no matter the fstype chosen.
> > To bypass this, some late procedures done on ext* and msdos fstype have
> been removed and all the fstab logic was passed to rootfs.py.
> > On this one, the idea is to still use the install command, but in this
> case, executed outside the pseudo environment. After the file as
> sucessfully been place, the idea was to check again for inconsistencies
> > in the database (now the fstab inode is different) and force it to
> rebuild with the correct inode, ensure fstab inode is correctly set on all
> the tables, including the xattrs one.
> >
> > Signed-off-by: Daniel Andrade <dani.barra25@gmail.com>
> > ---
>
> Hi Daniel,
>
> Thanks for the new version. It looks like this one is now breaking
> wic.Wic.test_exclude_path and wic.Wic.test_include_path tests:
>
> 2025-10-19 07:55:18,010 - oe-selftest - INFO - wic.Wic.test_exclude_path
> (subunit.RemotedTestCase)
> 2025-10-19 07:55:18,011 - oe-selftest - INFO -  ... FAIL
> ...
> AssertionError: Command 'wic create temp.wks -e core-image-minimal -o
> /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp'
> returned non-zero exit status 1:
> ...
> ERROR: _exec_cmd: export
> PATH=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/hosttools:$PATH;export
> PSEUDO_PREFIX=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr;export
> PSEUDO_LOCALSTATEDIR=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/pseudo1;export
> PSEUDO_PASSWD=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/rootfs1;export
> PSEUDO_NOSYMLINKEXP=1;/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/sysroots-components/x86_64/pseudo-native/usr/bin/pseudo
> -B  returned '1' instead of 0
> output: PRAGMA journal_mode = OFF;: database is locked
> PRAGMA synchronous = OFF;: database is locked
> Failed: database is locked
> Error getting 'logs.db' database.
> PRAGMA journal_mode = OFF;: database is locked
> PRAGMA synchronous = OFF;: database is locked
> Failed: database is locked
> Error getting 'files.db' database.
> error during load from disk: database is locked
> pdb_files: database error.
> Couldn't start file list, can't scan.
> db cleanup for server shutdown, 07:55:17.956
> memory-to-file backup complete, 07:55:17.956.
> db cleanup finished, 07:55:17.956
> considering table files
> considering table xattrs
> considering table migrations
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2574
>
> Can you have a look at these errors?
>
> Thanks,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>

[-- Attachment #2: Type: text/html, Size: 5861 bytes --]

Re: [OE-core] [PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

[Alexander Kanavin]

On Sun, 19 Oct 2025 at 23:45, dani.barra25 via lists.openembedded.org
<dani.barra25=gmail.com@lists.openembedded.org> wrote:

> +                    pseudo_prefix = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
> +
> +                    # Shutdown any existing pseudo server
> +                    # This ensures no other processes are using the database
> +                    shutdown_cmd = "%s -S" % pseudo_prefix
> +                    exec_native_cmd(shutdown_cmd, native_sysroot)
> +
> +                    # Database is not locked anymore
> +                    # Repairs inodes related to fstab. This way xattrs and metadata is correctly applied to the new file
> +                    repair_cmd = "%s -B" % pseudo_prefix
> +                    exec_native_cmd(repair_cmd, native_sysroot)
> +

Is it possible to describe how to reproduce and observe the issue that
this patch aims to fix? This snippet makes me *very* nervous as pseudo
is used by mulitple bitbake tasks in parallel, and doing shutdowns and
repairs in the middle of it can blow up spectacularly.

Alex

Re: [OE-core] [PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

[Daniel Andrade]

[-- Attachment #1: Type: text/plain, Size: 5062 bytes --]

Hello Alex,
Yes I can. Maybe I am doing something wrong since the patch is an
alternative to past threads but somehow it gets separated without context.

Starting from the beginning.
My original patch aimed to fix a simple thing. No matter what fstype you
were using, no xattrs from the build process were assigned to /etc/fstab if
you had multiple partitions. And this was particularly problematic when you
have read-only rootfs like squashfs and you need to have all the metadata
available during build, like the SELinux file labels, since they cannot be
set during boot time.
This is because it creates a temp folder and then creates a new fstab that
is later replaced in the image.

The replacement is made via 2 options:
1. If it was ext* or msdos, it will fall under specific methods for those
types that will open the already built image and remove the fstab and
writing the new one, losing all the metadata of that file.
2. It falls under the "install" cmd of rootfs.py and this one was being
executed inside the pseudo environment. This means the original entry of
fstab on files db is completely replace with this new file and all the
xattrs are gone

To avoid this, my first patch removed the debugfs specifics of ext* and
msdos, since those did not leveragy any "write only content" mechanism
without replacing the actual file. Then, all the fstab modifications would
lie under rootfs.py
which seemed to be the common point for everything. On this I simply
replaced "install" with "cp" since I only wanted to modify the contents.
https://lists.openembedded.org/g/openembedded-core/topic/115043355
Later in the discussion, Mathieu correctly pointed out that it could not be
used because it fails a test due to the usage of copyhardlinktree() for
this temporary rootfs folder. This meant that I was not only modifying the
temporary fstab, but also the original one,
leading to a crash of wic.Wic.test_no_fstab_update.
https://lists.openembedded.org/g/openembedded-core/topic/115043356

The second approach I took was looking at how pseudo behaved. This time, I
decided to maintain the original install cmd, but instead of running it
under fakeroot, I ran it on the host system. This meant that, if a later
check of the db occurred, it would correctly detect
an inode change on fstab and therefore it could fix it using the -B option.
And that is what I did :
https://lists.openembedded.org/g/openembedded-core/topic/115805391

I only neglected the fact that, in reality, other tests could have failed
because of this. And, in fact, it happened. Because of this, tests
regarding including and excluding paths were failing because the db was
locked.
That's when I reached the current patch alternative.
I knew it could cause problems but honestly I never considered it would and
when I run wic tests, I didn't see any failure. In reality, I was trying to
replicate what was being done a few lines above with the options -B -m and
-M. The difference
is that pseudo doesn't have a flag to repair a single file, therefore this
was the only solution found.

A good alternative to this would be pseudo actually having a functionality
to repair single file inodes, avoiding to completely check the whole
database and therefore better management on this side.
In the end this bug rendered all my images that enforced SELinux with read
only rootfs (squashfs) useless and I'd assume I might not be the only one
with this issue.

Another thing I notice is that, for repeatability purposes, there are
multiple variables being used to timestamp files. This is just a
recommendation/question. Shouldn't all those variable timestamps be aligned
also?

In any case, if you have some other idea on how to approach this, let me
know!

Daniel

Alexander Kanavin <alex.kanavin@gmail.com> escreveu (segunda, 20/10/2025
à(s) 13:09):

> On Sun, 19 Oct 2025 at 23:45, dani.barra25 via lists.openembedded.org
> <dani.barra25=gmail.com@lists.openembedded.org> wrote:
>
> > +                    pseudo_prefix = cls.__get_pseudo(native_sysroot,
> new_rootfs, new_pseudo)
> > +
> > +                    # Shutdown any existing pseudo server
> > +                    # This ensures no other processes are using the
> database
> > +                    shutdown_cmd = "%s -S" % pseudo_prefix
> > +                    exec_native_cmd(shutdown_cmd, native_sysroot)
> > +
> > +                    # Database is not locked anymore
> > +                    # Repairs inodes related to fstab. This way xattrs
> and metadata is correctly applied to the new file
> > +                    repair_cmd = "%s -B" % pseudo_prefix
> > +                    exec_native_cmd(repair_cmd, native_sysroot)
> > +
>
> Is it possible to describe how to reproduce and observe the issue that
> this patch aims to fix? This snippet makes me *very* nervous as pseudo
> is used by mulitple bitbake tasks in parallel, and doing shutdowns and
> repairs in the middle of it can blow up spectacularly.
>
> Alex
>

[-- Attachment #2: Type: text/html, Size: 6277 bytes --]

Re: [OE-core] [PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

[Ross Burton]

On 20 Oct 2025, at 13:09, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> 
> Is it possible to describe how to reproduce and observe the issue that
> this patch aims to fix? This snippet makes me *very* nervous as pseudo
> is used by mulitple bitbake tasks in parallel, and doing shutdowns and
> repairs in the middle of it can blow up spectacularly.

Agreed.  If you’re building a single image then yes, the wic tasks will typically happen at the end with nothing else in parallel. But this is not a guarantee and killing pseudo whilst other recipes might be mid-build is not acceptable.

Ross