Btrfs root filesystem permission issue

Btrfs root filesystem permission issue

[Lander0606]

[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]

Hi guys,

For my project, I changed the filesystem from ext4 to btrfs. The image builds and flashes fine, but when booting the system some mount problems occur because the root filesystem has UID 1000 all over it. So, when creating the filesystem, the directories are not created with root:root as owner. Some important notes:

* Currently I'm on the master branch to use the newest kernel 6.12, but when switching back to the scarthgap branch, the issue with permissions doesn't exist for a btrfs image. The filesystem mounts fine.
* I use Fedora 41 with a btrfs filesystem as build host, but I checked on an Ubuntu 24.04 machine with an ext4 filesystem as well and the issue still persists.
* I tried adding a fix with do_image[postfuncs] to run chown -R root:root on all directories, but the issue still persists...
* To test, I just mount the generated btrfs filesystem to my Fedora host and check the permissions with ls -lah

Has anyone experienced similar problems with btrfs? Or does anyone have an idea how to patch this?

Any help is appreciated! Thanks in advance!

[-- Attachment #2: Type: text/html, Size: 1287 bytes --]

Re: [yocto] Btrfs root filesystem permission issue

[Gyorgy Sarvari]

The reason seems to be this[1] update of btrfs-tools, which included
this[2] commit. The important change is that previously mkfs.btrfs was
using lstat syscall to get file stats, but then it switched to nftw libc
function to do the same. Pseudo[3] however doesn't support this call -
it "knows" that it exists (at least nftw64), but there seems to be no
implementation.

Possibilities at the first sight:
1 - Downgrade btrfs-tools in your custom layer (unless you need any of
the new functions)
2 - Revert the change with a patch in a bbappend
3 - Make pseudo support nftw

#1 and #2 would be a fast change.
#3 would be the best of course (it seems to be a nice extension, I might
try to look at it over the weekend - though chances are that it won't
happen or I will give up/fail. So feel free, anyone.).

[1]:
https://git.yoctoproject.org/poky/commit/?id=2ff9d40ccb493967151bfed6aed889de45bdd464
[2]:
https://web.git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=c6464d3f99ed1dabceff1168eabb207492c37624
[3]: https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/pseudo

On 27.02.25 19:57, Lander0606 via lists.yoctoproject.org wrote:
> Hi guys,
>  
> For my project, I changed the filesystem from ext4 to btrfs. The image
> builds and flashes fine, but when booting the system some mount
> problems occur because the root filesystem has UID 1000 all over it.
> So, when creating the filesystem, the directories are not created with
> /root:root/ as owner. Some important notes:
>  
>
>   * Currently I'm on the master branch to use the newest kernel 6.12,
>     but when switching back to the /scarthgap/ branch, the issue with
>     permissions doesn't exist for a btrfs image. The filesystem mounts
>     fine.
>   * I use Fedora 41 with a btrfs filesystem as build host, but I
>     checked on an Ubuntu 24.04 machine with an ext4 filesystem as well
>     and the issue still persists.
>   * I tried adding a fix with /do_image[postfuncs] /to run /chown -R
>     root:root/ on all directories, but the issue still persists...
>   * To test, I just mount the generated btrfs filesystem to my Fedora
>     host and check the permissions with /ls -lah/
>
>  
> Has anyone experienced similar problems with btrfs? Or does anyone
> have an idea how to patch this?
>  
> Any help is appreciated! Thanks in advance!
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#64889): https://lists.yoctoproject.org/g/yocto/message/64889
> Mute This Topic: https://lists.yoctoproject.org/mt/111420988/6084445
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [skandigraun@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [yocto] Btrfs root filesystem permission issue

[Lander0606]

[-- Attachment #1: Type: text/plain, Size: 604 bytes --]

Thanks for your response!

I did have a look at the pseudo and btrfs-progs repository and noticed that the pseudo repository contains the following code in ./ports/unix/guts/ for the nftw() system call:

rc = real_nftw(path, fn, nopenfd, flag);

So, it seems that the wrapper for the nftw() system call just forwards it to the underlying system (the host, right?). Is that the reason why the btrfs filesystem creation does work, but files are owned by the build host user?
I'm guessing there need to be some code around it to support the "fake root environment" that pseudo is trying to achieve?

[-- Attachment #2: Type: text/html, Size: 710 bytes --]

Re: [yocto] Btrfs root filesystem permission issue

[Gyorgy Sarvari]

On 01.03.25 16:56, Lander0606 via Lists.Yoctoproject.Org wrote:
> Thanks for your response!
>  
> I did have a look at the pseudo and btrfs-progs repository and noticed
> that the pseudo repository contains the following code in
> ./ports/unix/guts/ for the nftw() system call:
>  
> rc = real_nftw(path, fn, nopenfd, flag);
>  
> So, it seems that the wrapper for the nftw() system call just forwards
> it to the underlying system (the host, right?). Is that the reason why
> the btrfs filesystem creation does work, but files are owned by the
> build host user?
> I'm guessing there need to be some code around it to support the "fake
> root environment" that pseudo is trying to achieve?
Yes, pretty much. I don't know from the top of my head if the
ports/unix/* files are used in Linux env at all, but regardless, your
understanding looks correct to me and I think you summarized the issue
pretty well.

Re: [yocto] Btrfs root filesystem permission issue

[Lander0606]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

Nice! I did some research, but the only "improvement" I see would be to change the stat-struct in the callback to force UID/GID to 0. But for that, I think we need another callback function to wrap around the given callback to change the UID/GID. Unless I'm not seeing something or misunderstand where the problem really is?

I did a grep on the whole pseudo source code to search for the existance of "nftw" and found that ./ports/unix/* is the only place. If that one is not used in Linux, I would guess btrfs-tools would give an error upon creating the filesystem, right?

[-- Attachment #2: Type: text/html, Size: 617 bytes --]

Re: [yocto] Btrfs root filesystem permission issue

[Gyorgy Sarvari]

On 01.03.25 19:32, Lander0606 via Lists.Yoctoproject.Org wrote:
> Nice! I did some research, but the only "improvement" I see would be
> to change the stat-struct in the callback to force UID/GID to 0. But
> for that, I think we need another callback function to wrap around the
> given callback to change the UID/GID. Unless I'm not seeing something
> or misunderstand where the problem really is?
>  
I have created a draft patch[1] - simplified (and not a full
implementation), but it seems to do the job at least for btrfs-tools. It
works for me(TM), though it doesn't always mean an awful lot - I only
checked the generated FS with core-image-minimal, which looked what I
expected. Would you mind giving it a try, if you have time (or anyone else)?

If it works for you too, then I will write a couple of tests, and
attempt submitting it for the real thing.

Of course, everyone is free to try/comment... I intend to submit
something very similar to the current patch - the sooner comments (or
rejection) come, the better it is.

(If no feedback comes that would discourage me, I will try to submit a
patch around Wednesday-ish in any case, for further scrutiny...)


Thanks

[1]:
https://gist.github.com/OldManYellsAtCloud/f6b6ecf37ab65db8d61ac7bed583a7f4

Re: [yocto] Btrfs root filesystem permission issue

[Lander0606]

[-- Attachment #1: Type: text/plain, Size: 2100 bytes --]

> 
> I have created a draft patch[1] - simplified (and not a full
> implementation), but it seems to do the job at least for btrfs-tools. It
> works for me(TM), though it doesn't always mean an awful lot - I only
> checked the generated FS with core-image-minimal, which looked what I
> expected. Would you mind giving it a try, if you have time (or anyone
> else)?
> 
> If it works for you too, then I will write a couple of tests, and
> attempt submitting it for the real thing.
> 
> Of course, everyone is free to try/comment... I intend to submit
> something very similar to the current patch - the sooner comments (or
> rejection) come, the better it is.
> 
> (If no feedback comes that would discourage me, I will try to submit a
> patch around Wednesday-ish in any case, for further scrutiny...)
> 
> 
> Thanks
> 
> [1]:
> https://gist.github.com/OldManYellsAtCloud/f6b6ecf37ab65db8d61ac7bed583a7f4
> 
> 

Thanks for the patch! I did have a look at the code and added it to pseudo, but when creating the image, I did encounter the following error:

| DEBUG: Executing shell function do_image_btrfs
| 0+0 records in
| 0+0 records out
| 0 bytes copied, 9.021e-06 s, 0.0 kB/s
| ERROR: failed to add link for inode 262 ('/home/landervl/yocto-tegra-master/build/tmp/work/jetson_orin_nano_devkit-oe4t-linux/tegra-espimage/1.0/rootfs/./EFI/BOOT/bootaa64.efi'): File exists
| ERROR: unable to traverse directory /home/landervl/yocto-tegra-master/build/tmp/work/jetson_orin_nano_devkit-oe4t-linux/tegra-espimage/1.0/rootfs: -17
| ERROR: error while filling filesystem: -17

I use btrfs-tools 6.12, included in the master branch of Poky. When removing the patch and cleaning the objects for pseudo, the error does not occur and the image builds fine. It's important to note that I did not clean the full build and started all over again, just removed the current sstate for pseudo because the full build can run for a few hours :D. I will start a clean build for now to see if the issue may be gone. My machine is a NVIDIA Jetson Orin Nano, I use meta-tegra as BSP.

[-- Attachment #2: Type: text/html, Size: 2526 bytes --]

Re: [yocto] Btrfs root filesystem permission issue

[Gyorgy Sarvari]

On 3/3/25 22:52, Lander0606 via Lists.Yoctoproject.Org wrote:
>     I have created a draft patch[1] - simplified (and not a full
>     implementation), but it seems to do the job at least for btrfs-tools. It
>     works for me(TM), though it doesn't always mean an awful lot - I only
>     checked the generated FS with core-image-minimal, which looked what I
>     expected. Would you mind giving it a try, if you have time (or
>     anyone else)?
> 
>     If it works for you too, then I will write a couple of tests, and
>     attempt submitting it for the real thing.
> 
>     Of course, everyone is free to try/comment... I intend to submit
>     something very similar to the current patch - the sooner comments (or
>     rejection) come, the better it is.
> 
>     (If no feedback comes that would discourage me, I will try to submit a
>     patch around Wednesday-ish in any case, for further scrutiny...)
> 
> 
>     Thanks
> 
>     [1]:
>     https://gist.github.com/OldManYellsAtCloud/
>     f6b6ecf37ab65db8d61ac7bed583a7f4 <https://gist.github.com/
>     OldManYellsAtCloud/f6b6ecf37ab65db8d61ac7bed583a7f4>
> 
> Thanks for the patch! I did have a look at the code and added it to 
> pseudo, but when creating the image, I did encounter the following error:
> | DEBUG: Executing shell function do_image_btrfs
> | 0+0 records in
> | 0+0 records out
> | 0 bytes copied, 9.021e-06 s, 0.0 kB/s
> | ERROR: failed to add link for inode 262 ('/home/landervl/yocto-tegra- 
> master/build/tmp/work/jetson_orin_nano_devkit-oe4t-linux/tegra- 
> espimage/1.0/rootfs/./EFI/BOOT/bootaa64.efi'): File exists
> | ERROR: unable to traverse directory /home/landervl/yocto-tegra-master/ 
> build/tmp/work/jetson_orin_nano_devkit-oe4t-linux/tegra-espimage/1.0/ 
> rootfs: -17
> | ERROR: error while filling filesystem: -17
> I use btrfs-tools 6.12, included in the master branch of Poky. When 
> removing the patch and cleaning the objects for pseudo, the error does 
> not occur and the image builds fine. It's important to note that I did 
> not clean the full build and started all over again, just removed the 
> current sstate for pseudo because the full build can run for a few 
> hours :D. I will start a clean build for now to see if the issue may be 
> gone. My machine is a NVIDIA Jetson Orin Nano, I use meta-tegra as BSP.

Most probably it won't - I don't think a new build will change. Thanks a 
lot for trying - I will look more into the patch, and get back once/if I 
get to the bottom of it.

Re: [yocto] Btrfs root filesystem permission issue

[Lander0606]

[-- Attachment #1: Type: text/plain, Size: 212 bytes --]

No problem, I will also look into it to see if I can fix something. I will add some comments to the shared patch file on GitHub if I found something, maybe that's easier to communicate when debugging the patch.

[-- Attachment #2: Type: text/html, Size: 223 bytes --]

Re: [yocto] Btrfs root filesystem permission issue

[Lander0606]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

I did improve the patch a bit by adding a list to track which inodes already have been processed so the callback isn't executed twice for the same inode (as happened in the error above, for some reason). Maybe the current solution is not the optimal one, but the build now completes with no errors.

I also added some small changes to prevent a memory leak (closing the directory) and crashes when an error occurred while opening a directory. The updated patch can be found here, in the comments below I added the line numbers for the changes: https://gist.github.com/Lander0606/7ca20f75408b1d759a1e54b2384626d1

Please feel free to test and improve this implementation.

[-- Attachment #2: Type: text/html, Size: 857 bytes --]

Re: [yocto] Btrfs root filesystem permission issue

[Ferry Toth]

Hi,

Op 01-03-2025 om 15:35 schreef Gyorgy Sarvari:
> The reason seems to be this[1] update of btrfs-tools, which included
> this[2] commit. The important change is that previously mkfs.btrfs was
> using lstat syscall to get file stats, but then it switched to nftw libc
> function to do the same. Pseudo[3] however doesn't support this call -
> it "knows" that it exists (at least nftw64), but there seems to be no
> implementation.
> 
> Possibilities at the first sight:
> 1 - Downgrade btrfs-tools in your custom layer (unless you need any of
> the new functions)
> 2 - Revert the change with a patch in a bbappend
> 3 - Make pseudo support nftw
> 
> #1 and #2 would be a fast change.

There is a problem with mkfs.btrfs that creates a file system with 
errors if there are hardlinks out of the the rootfs (as we have). You 
can this easily by running btrfs check on the image file.

The change if fixed after btrfs-tools v6.12

See this thread (Errors on newly created file system):
https://lore.kernel.org/linux-btrfs/d98ffb69-195b-4c07-ac56-8ae1f811af32@gmail.com/T/#m30ef7259fe280660c55733a7ba37c356d292fe10

> #3 would be the best of course (it seems to be a nice extension, I might
> try to look at it over the weekend - though chances are that it won't
> happen or I will give up/fail. So feel free, anyone.).
> 
> [1]:
> https://git.yoctoproject.org/poky/commit/?id=2ff9d40ccb493967151bfed6aed889de45bdd464
> [2]:
> https://web.git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=c6464d3f99ed1dabceff1168eabb207492c37624
> [3]: https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/pseudo
> 
> On 27.02.25 19:57, Lander0606 via lists.yoctoproject.org wrote:
>> Hi guys,
>>   
>> For my project, I changed the filesystem from ext4 to btrfs. The image
>> builds and flashes fine, but when booting the system some mount
>> problems occur because the root filesystem has UID 1000 all over it.
>> So, when creating the filesystem, the directories are not created with
>> /root:root/ as owner. Some important notes:
>>   
>>
>>    * Currently I'm on the master branch to use the newest kernel 6.12,
>>      but when switching back to the /scarthgap/ branch, the issue with
>>      permissions doesn't exist for a btrfs image. The filesystem mounts
>>      fine.
>>    * I use Fedora 41 with a btrfs filesystem as build host, but I
>>      checked on an Ubuntu 24.04 machine with an ext4 filesystem as well
>>      and the issue still persists.
>>    * I tried adding a fix with /do_image[postfuncs] /to run /chown -R
>>      root:root/ on all directories, but the issue still persists...
>>    * To test, I just mount the generated btrfs filesystem to my Fedora
>>      host and check the permissions with /ls -lah/
>>
>>   
>> Has anyone experienced similar problems with btrfs? Or does anyone
>> have an idea how to patch this?
>>   
>> Any help is appreciated! Thanks in advance!
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#64889): https://lists.yoctoproject.org/g/yocto/message/64889
>> Mute This Topic: https://lists.yoctoproject.org/mt/111420988/6084445
>> Group Owner: yocto+owner@lists.yoctoproject.org
>> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [skandigraun@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
> 

Re: [yocto] Btrfs root filesystem permission issue

[Gyorgy Sarvari]

On 4/24/25 22:18, Ferry Toth wrote:
> Hi,
>
> Op 01-03-2025 om 15:35 schreef Gyorgy Sarvari:
>> The reason seems to be this[1] update of btrfs-tools, which included
>> this[2] commit. The important change is that previously mkfs.btrfs was
>> using lstat syscall to get file stats, but then it switched to nftw libc
>> function to do the same. Pseudo[3] however doesn't support this call -
>> it "knows" that it exists (at least nftw64), but there seems to be no
>> implementation.
>>
>> Possibilities at the first sight:
>> 1 - Downgrade btrfs-tools in your custom layer (unless you need any of
>> the new functions)
>> 2 - Revert the change with a patch in a bbappend
>> 3 - Make pseudo support nftw
>>
>> #1 and #2 would be a fast change.
> There is a problem with mkfs.btrfs that creates a file system with 
> errors if there are hardlinks out of the the rootfs (as we have). You 
> can this easily by running btrfs check on the image file.
>
> The change if fixed after btrfs-tools v6.12
>
> See this thread (Errors on newly created file system):
> https://lore.kernel.org/linux-btrfs/d98ffb69-195b-4c07-ac56-8ae1f811af32@gmail.com/T/#m30ef7259fe280660c55733a7ba37c356d292fe10

So old btrfs generates bad FS, a new one has pseudo problems... Trouble
never comes alone, does it? :)

I have submitted a pseudo patch[1] some time ago, though there isn't a
lot of enthusiasm about it. It works on my machine (TM), and if you want
you are free to try it of course, though I don't think it will be merged.

[1]:
https://lore.kernel.org/yocto-patches/20250407191414.2992785-2-skandigraun@gmail.com/T/#mfd5a2338ea6fefc5ed734c0beb85ac26806b8684


>
>> #3 would be the best of course (it seems to be a nice extension, I might
>> try to look at it over the weekend - though chances are that it won't
>> happen or I will give up/fail. So feel free, anyone.).
>>
>> [1]:
>> https://git.yoctoproject.org/poky/commit/?id=2ff9d40ccb493967151bfed6aed889de45bdd464
>> [2]:
>> https://web.git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=c6464d3f99ed1dabceff1168eabb207492c37624
>> [3]: https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/pseudo
>>
>> On 27.02.25 19:57, Lander0606 via lists.yoctoproject.org wrote:
>>> Hi guys,
>>>   
>>> For my project, I changed the filesystem from ext4 to btrfs. The image
>>> builds and flashes fine, but when booting the system some mount
>>> problems occur because the root filesystem has UID 1000 all over it.
>>> So, when creating the filesystem, the directories are not created with
>>> /root:root/ as owner. Some important notes:
>>>   
>>>
>>>    * Currently I'm on the master branch to use the newest kernel 6.12,
>>>      but when switching back to the /scarthgap/ branch, the issue with
>>>      permissions doesn't exist for a btrfs image. The filesystem mounts
>>>      fine.
>>>    * I use Fedora 41 with a btrfs filesystem as build host, but I
>>>      checked on an Ubuntu 24.04 machine with an ext4 filesystem as well
>>>      and the issue still persists.
>>>    * I tried adding a fix with /do_image[postfuncs] /to run /chown -R
>>>      root:root/ on all directories, but the issue still persists...
>>>    * To test, I just mount the generated btrfs filesystem to my Fedora
>>>      host and check the permissions with /ls -lah/
>>>
>>>   
>>> Has anyone experienced similar problems with btrfs? Or does anyone
>>> have an idea how to patch this?
>>>   
>>> Any help is appreciated! Thanks in advance!
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#64889): https://lists.yoctoproject.org/g/yocto/message/64889
>>> Mute This Topic: https://lists.yoctoproject.org/mt/111420988/6084445
>>> Group Owner: yocto+owner@lists.yoctoproject.org
>>> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [skandigraun@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>