* mkfs.erofs fails (failed to build shared xattrs, err 61) @ 2024-03-12 9:57 Gael Donval 2024-03-12 10:44 ` Gao Xiang 0 siblings, 1 reply; 13+ messages in thread From: Gael Donval @ 2024-03-12 9:57 UTC (permalink / raw) To: linux-erofs@lists.ozlabs.org Dear list, $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img foo mkfs.erofs 1.7.1 <E> erofs: failed to build shared xattrs: [Error 61] No data available <E> erofs: Could not format the device : [Error 61] No data available That is at a location backed by BTRFS (btrfs-progs v6.7.1) on kernel 6.8.0. If I use a TMPFS-supported folder instead all goes well. Of course (NB "-x-1"), $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 foo.erofs.img foo also works but is not how mkfs.erofs is meant to work in the general case. Kind regards, Gaël ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 9:57 mkfs.erofs fails (failed to build shared xattrs, err 61) Gael Donval @ 2024-03-12 10:44 ` Gao Xiang 2024-03-12 10:45 ` Gao Xiang 0 siblings, 1 reply; 13+ messages in thread From: Gao Xiang @ 2024-03-12 10:44 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org Hi Gael, On 2024/3/12 17:57, Gael Donval wrote: > Dear list, > > $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img foo > > mkfs.erofs 1.7.1 > <E> erofs: failed to build shared xattrs: [Error 61] No data available > <E> erofs: Could not format the device : [Error 61] No data available > > That is at a location backed by BTRFS (btrfs-progs v6.7.1) on kernel 6.8.0. > > If I use a TMPFS-supported folder instead all goes well. > > > Of course (NB "-x-1"), > > $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 foo.erofs.img foo > > also works but is not how mkfs.erofs is meant to work in the general case. Thanks for your feedback. Currently I don't have some BTRFS environment, I could set up one later. Yet in parallel could you provide a full message of "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? Thanks, Gao Xiang > > Kind regards, > Gaël ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 10:44 ` Gao Xiang @ 2024-03-12 10:45 ` Gao Xiang 2024-03-12 11:07 ` Gael Donval 0 siblings, 1 reply; 13+ messages in thread From: Gao Xiang @ 2024-03-12 10:45 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org On 2024/3/12 18:44, Gao Xiang wrote: > Hi Gael, > > On 2024/3/12 17:57, Gael Donval wrote: >> Dear list, >> >> $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img foo >> mkfs.erofs 1.7.1 >> <E> erofs: failed to build shared xattrs: [Error 61] No data available >> <E> erofs: Could not format the device : [Error 61] No data available >> That is at a location backed by BTRFS (btrfs-progs v6.7.1) on kernel 6.8.0. >> >> If I use a TMPFS-supported folder instead all goes well. >> >> >> Of course (NB "-x-1"), >> >> $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 foo.erofs.img foo >> >> also works but is not how mkfs.erofs is meant to work in the general case. > > Thanks for your feedback. > > Currently I don't have some BTRFS environment, I could set up one later. > Yet in parallel could you provide a full message of > "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? sorry, I meant "strace mkfs.erofs foo.erofs.img foo" Thanks, Gao Xiang > > Thanks, > Gao Xiang > >> >> Kind regards, >> Gaël ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 10:45 ` Gao Xiang @ 2024-03-12 11:07 ` Gael Donval 2024-03-12 11:37 ` Gao Xiang 0 siblings, 1 reply; 13+ messages in thread From: Gael Donval @ 2024-03-12 11:07 UTC (permalink / raw) To: hsiangkao@linux.alibaba.com, linux-erofs@lists.ozlabs.org [-- Attachment #1: Type: text/plain, Size: 1306 bytes --] > On 2024/3/12 18:44, Gao Xiang wrote: > > Hi Gael, > > > > On 2024/3/12 17:57, Gael Donval wrote: > > > Dear list, > > > > > > $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img foo > > > mkfs.erofs 1.7.1 > > > <E> erofs: failed to build shared xattrs: [Error 61] No data > > > available > > > <E> erofs: Could not format the device : [Error 61] No > > > data available > > > That is at a location backed by BTRFS (btrfs-progs v6.7.1) on > > > kernel 6.8.0. > > > > > > If I use a TMPFS-supported folder instead all goes well. > > > > > > > > > Of course (NB "-x-1"), > > > > > > $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 foo.erofs.img > > > foo > > > > > > also works but is not how mkfs.erofs is meant to work in the > > > general case. > > > > Thanks for your feedback. > > > > Currently I don't have some BTRFS environment, I could set up one > > later. > > Yet in parallel could you provide a full message of > > "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? > > sorry, I meant "strace mkfs.erofs foo.erofs.img foo" Hi, Here it is. Thanks for your help, Gaël > > Thanks, > Gao Xiang > > > > > Thanks, > > Gao Xiang > > > > > > > > Kind regards, > > > Gaël [-- Attachment #2: strace_mkfs_erofs --] [-- Type: text/plain, Size: 7192 bytes --] execve("/usr/bin/mkfs.erofs", ["mkfs.erofs", "foo.erofs.img", "foo"], 0x7ffd4e2e1d20 /* 49 vars */) = 0 brk(NULL) = 0x5a8b85317000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=143655, ...}) = 0 mmap(NULL, 143655, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7293c7817000 close(3) = 0 openat(AT_FDCWD, "/usr/lib/libuuid.so.1", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=30808, ...}) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7293c7815000 mmap(NULL, 32832, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7293c780c000 mmap(0x7293c780e000, 16384, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7293c780e000 mmap(0x7293c7812000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7293c7812000 mmap(0x7293c7813000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7293c7813000 close(3) = 0 openat(AT_FDCWD, "/usr/lib/liblz4.so.1", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=145280, ...}) = 0 mmap(NULL, 147472, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7293c77e7000 mmap(0x7293c77ea000, 118784, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7293c77ea000 mmap(0x7293c7807000, 12288, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0x7293c7807000 mmap(0x7293c780a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x7293c780a000 close(3) = 0 openat(AT_FDCWD, "/usr/lib/liblzma.so.5", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=206872, ...}) = 0 mmap(NULL, 204816, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7293c77b4000 mmap(0x7293c77b7000, 139264, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7293c77b7000 mmap(0x7293c77d9000, 49152, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x25000) = 0x7293c77d9000 mmap(0x7293c77e5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x31000) = 0x7293c77e5000 close(3) = 0 openat(AT_FDCWD, "/usr/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=100296, ...}) = 0 mmap(NULL, 102416, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7293c779a000 mmap(0x7293c779d000, 57344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7293c779d000 mmap(0x7293c77ab000, 28672, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x11000) = 0x7293c77ab000 mmap(0x7293c77b2000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7293c77b2000 close(3) = 0 openat(AT_FDCWD, "/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220^\2\0\0\0\0\0"..., 832) = 832 pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784 fstat(3, {st_mode=S_IFREG|0755, st_size=1948952, ...}) = 0 pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784 mmap(NULL, 1973104, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7293c75b8000 mmap(0x7293c75dc000, 1421312, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x24000) = 0x7293c75dc000 mmap(0x7293c7737000, 348160, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17f000) = 0x7293c7737000 mmap(0x7293c778c000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1d3000) = 0x7293c778c000 mmap(0x7293c7792000, 31600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7293c7792000 close(3) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7293c75b6000 arch_prctl(ARCH_SET_FS, 0x7293c75b7440) = 0 set_tid_address(0x7293c75b7710) = 29056 set_robust_list(0x7293c75b7720, 24) = 0 rseq(0x7293c75b7d60, 0x20, 0, 0x53053053) = 0 mprotect(0x7293c778c000, 16384, PROT_READ) = 0 mprotect(0x7293c77b2000, 4096, PROT_READ) = 0 mprotect(0x7293c77e5000, 4096, PROT_READ) = 0 mprotect(0x7293c780a000, 4096, PROT_READ) = 0 mprotect(0x7293c7813000, 4096, PROT_READ) = 0 mprotect(0x5a8b8316a000, 4096, PROT_READ) = 0 mprotect(0x7293c786e000, 8192, PROT_READ) = 0 prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0 munmap(0x7293c7817000, 143655) = 0 getrandom("\x78\x04\xd6\xf8\xe2\xaf\xae\xe0\xe6\xca\xe9\xd1\xb1\x45\x9a\x00", 16, GRND_NONBLOCK) = 16 getpid() = 29056 getuid() = 1000 getpid() = 29056 getppid() = 29053 gettid() = 29056 getrandom("\xee\xf6\x7d\x1f\x6d\x95\xb2\x8d", 8, GRND_NONBLOCK) = 8 brk(NULL) = 0x5a8b85317000 brk(0x5a8b85338000) = 0x5a8b85338000 getcwd("/home/gdonval/Documents/Work/RVMI/CustomOS/plop", 1024) = 48 readlink("/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo", 0x7ffee3478000, 1023) = -1 EINVAL (Invalid argument) newfstatat(AT_FDCWD, "/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo", {st_mode=S_IFDIR|0755, st_size=6, ...}, AT_SYMLINK_NOFOLLOW) = 0 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x3), ...}) = 0 write(1, "mkfs.erofs 1.7.1\n", 17) = 17 openat(AT_FDCWD, "foo.erofs.img", O_RDWR|O_CREAT, 0644) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 openat(AT_FDCWD, "/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4 fstat(4, {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0 getdents64(4, 0x5a8b85317800 /* 3 entries */, 32768) = 72 newfstatat(AT_FDCWD, "/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo/bar", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0 llistxattr("/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo/bar", NULL, 0) = 18 llistxattr("/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo/bar", "btrfs.compression\0", 18) = 18 lgetxattr("/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo/bar", "btrfs.compression", NULL, 0) = 4 lgetxattr("/home/gdonval/Documents/Work/RVMI/CustomOS/plop/foo/bar", "btrfs.compression", "zstd", 4) = 4 close(4) = 0 write(2, "<E> erofs: failed to build share"..., 71<E> erofs: failed to build shared xattrs: [Error 61] No data available ) = 71 close(3) = 0 write(2, "<E> erofs: \tCould not format the"..., 72<E> erofs: Could not format the device : [Error 61] No data available ) = 72 exit_group(1) = ? +++ exited with 1 +++ ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 11:07 ` Gael Donval @ 2024-03-12 11:37 ` Gao Xiang 2024-03-12 11:57 ` Gao Xiang 2024-03-12 12:01 ` Gael Donval 0 siblings, 2 replies; 13+ messages in thread From: Gao Xiang @ 2024-03-12 11:37 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org On 2024/3/12 19:07, Gael Donval wrote: >> On 2024/3/12 18:44, Gao Xiang wrote: >>> Hi Gael, >>> >>> On 2024/3/12 17:57, Gael Donval wrote: >>>> Dear list, >>>> >>>> $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img foo >>>> mkfs.erofs 1.7.1 >>>> <E> erofs: failed to build shared xattrs: [Error 61] No data >>>> available >>>> <E> erofs: Could not format the device : [Error 61] No >>>> data available >>>> That is at a location backed by BTRFS (btrfs-progs v6.7.1) on >>>> kernel 6.8.0. >>>> >>>> If I use a TMPFS-supported folder instead all goes well. >>>> >>>> >>>> Of course (NB "-x-1"), >>>> >>>> $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 foo.erofs.img >>>> foo >>>> >>>> also works but is not how mkfs.erofs is meant to work in the >>>> general case. >>> >>> Thanks for your feedback. >>> >>> Currently I don't have some BTRFS environment, I could set up one >>> later. >>> Yet in parallel could you provide a full message of >>> "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? >> >> sorry, I meant "strace mkfs.erofs foo.erofs.img foo" > > Hi, > > Here it is. > The problem here is that BTRFS could extract their internal xattrs ("inode properties") in their own namespace which don't belong to the original file itself like: XATTR_BTRFS_PREFIX "compression" -> btrfs.compression = zstd I think I need to introduce something to formally ignore these xattrs like: `--xattrs-exclude=pattern`: https://www.gnu.org/software/tar/manual/html_node/Extended-File-Attributes.html Thanks, Gao Xiang > Thanks for your help, > Gaël > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 11:37 ` Gao Xiang @ 2024-03-12 11:57 ` Gao Xiang 2024-03-12 12:01 ` Gael Donval 1 sibling, 0 replies; 13+ messages in thread From: Gao Xiang @ 2024-03-12 11:57 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org On 2024/3/12 19:37, Gao Xiang wrote: > > > On 2024/3/12 19:07, Gael Donval wrote: >>> On 2024/3/12 18:44, Gao Xiang wrote: >>>> Hi Gael, >>>> >>>> On 2024/3/12 17:57, Gael Donval wrote: >>>>> Dear list, >>>>> >>>>> $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img foo >>>>> mkfs.erofs 1.7.1 >>>>> <E> erofs: failed to build shared xattrs: [Error 61] No data >>>>> available >>>>> <E> erofs: Could not format the device : [Error 61] No >>>>> data available >>>>> That is at a location backed by BTRFS (btrfs-progs v6.7.1) on >>>>> kernel 6.8.0. >>>>> >>>>> If I use a TMPFS-supported folder instead all goes well. >>>>> >>>>> >>>>> Of course (NB "-x-1"), >>>>> >>>>> $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 foo.erofs.img >>>>> foo >>>>> >>>>> also works but is not how mkfs.erofs is meant to work in the >>>>> general case. >>>> >>>> Thanks for your feedback. >>>> >>>> Currently I don't have some BTRFS environment, I could set up one >>>> later. >>>> Yet in parallel could you provide a full message of >>>> "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? >>> >>> sorry, I meant "strace mkfs.erofs foo.erofs.img foo" >> >> Hi, >> >> Here it is. >> > > The problem here is that BTRFS could extract their internal > xattrs ("inode properties") in their own namespace which > don't belong to the original file itself like: > > XATTR_BTRFS_PREFIX "compression" -> btrfs.compression = zstd > > I think I need to introduce something to formally ignore > these xattrs like: `--xattrs-exclude=pattern`: > https://www.gnu.org/software/tar/manual/html_node/Extended-File-Attributes.html BTW, as for erofs-utils 1.7.1, you may use tar c --xattrs-exclude="btrfs.*" foo | mkfs.erofs --tar=f 111.erofs as a workaround. I think it will work. Thanks, Gao Xiang ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 11:37 ` Gao Xiang 2024-03-12 11:57 ` Gao Xiang @ 2024-03-12 12:01 ` Gael Donval 2024-03-12 12:43 ` Gao Xiang 1 sibling, 1 reply; 13+ messages in thread From: Gael Donval @ 2024-03-12 12:01 UTC (permalink / raw) To: hsiangkao@linux.alibaba.com, linux-erofs@lists.ozlabs.org On Tue, 2024-03-12 at 19:37 +0800, Gao Xiang wrote: > > > On 2024/3/12 19:07, Gael Donval wrote: > > > On 2024/3/12 18:44, Gao Xiang wrote: > > > > Hi Gael, > > > > > > > > On 2024/3/12 17:57, Gael Donval wrote: > > > > > Dear list, > > > > > > > > > > $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img > > > > > foo > > > > > mkfs.erofs 1.7.1 > > > > > <E> erofs: failed to build shared xattrs: [Error 61] No > > > > > data > > > > > available > > > > > <E> erofs: Could not format the device : [Error 61] > > > > > No > > > > > data available > > > > > That is at a location backed by BTRFS (btrfs-progs v6.7.1) on > > > > > kernel 6.8.0. > > > > > > > > > > If I use a TMPFS-supported folder instead all goes well. > > > > > > > > > > > > > > > Of course (NB "-x-1"), > > > > > > > > > > $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 > > > > > foo.erofs.img > > > > > foo > > > > > > > > > > also works but is not how mkfs.erofs is meant to work in the > > > > > general case. > > > > > > > > Thanks for your feedback. > > > > > > > > Currently I don't have some BTRFS environment, I could set up > > > > one > > > > later. > > > > Yet in parallel could you provide a full message of > > > > "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? > > > > > > sorry, I meant "strace mkfs.erofs foo.erofs.img foo" > > > > Hi, > > > > Here it is. > > > > The problem here is that BTRFS could extract their internal > xattrs ("inode properties") in their own namespace which > don't belong to the original file itself like: > > XATTR_BTRFS_PREFIX "compression" -> btrfs.compression = zstd > > I think I need to introduce something to formally ignore > these xattrs like: `--xattrs-exclude=pattern`: > https://urldefense.com/v3/__https://www.gnu.org/software/tar/manual/html_node/Extended-File-Attributes.html__;!!PDiH4ENfjr2_Jw!Dqu_tH2zT_TKEhHNi2CZToYlmYFeZayWE1BQVQjZBZq4egjYS1bAVS7BasQJ9BETJoVINPEB-p06y7izIUmWk-2VnofOQZwLrw$ > [gnu[.]org] I think that could work but a mode where EROFS keep the XATTRS it recognises and ignore the ones it doesn't would probably prove useful in the general case. I would suggest a --strict-xattrs mode behaving like it does today, requiring explicit --xattrs-exclude to whitelist XATTRS. The default would be a relaxed mode where ignored XATTRS are reported on stderr at INFO level. Would that make sense and be alright? Is there anything the BTRFS people could do to make their FS easier to work with? Gaël > > Thanks, > Gao Xiang > > > Thanks for your help, > > Gaël > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 12:01 ` Gael Donval @ 2024-03-12 12:43 ` Gao Xiang 2024-03-12 13:14 ` Gael Donval 0 siblings, 1 reply; 13+ messages in thread From: Gao Xiang @ 2024-03-12 12:43 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org On 2024/3/12 20:01, Gael Donval wrote: > On Tue, 2024-03-12 at 19:37 +0800, Gao Xiang wrote: >> >> >> On 2024/3/12 19:07, Gael Donval wrote: >>>> On 2024/3/12 18:44, Gao Xiang wrote: >>>>> Hi Gael, >>>>> >>>>> On 2024/3/12 17:57, Gael Donval wrote: >>>>>> Dear list, >>>>>> >>>>>> $ mkdir foo && touch foo/bar && mkfs.erofs foo.erofs.img >>>>>> foo >>>>>> mkfs.erofs 1.7.1 >>>>>> <E> erofs: failed to build shared xattrs: [Error 61] No >>>>>> data >>>>>> available >>>>>> <E> erofs: Could not format the device : [Error 61] >>>>>> No >>>>>> data available >>>>>> That is at a location backed by BTRFS (btrfs-progs v6.7.1) on >>>>>> kernel 6.8.0. >>>>>> >>>>>> If I use a TMPFS-supported folder instead all goes well. >>>>>> >>>>>> >>>>>> Of course (NB "-x-1"), >>>>>> >>>>>> $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 >>>>>> foo.erofs.img >>>>>> foo >>>>>> >>>>>> also works but is not how mkfs.erofs is meant to work in the >>>>>> general case. >>>>> >>>>> Thanks for your feedback. >>>>> >>>>> Currently I don't have some BTRFS environment, I could set up >>>>> one >>>>> later. >>>>> Yet in parallel could you provide a full message of >>>>> "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? >>>> >>>> sorry, I meant "strace mkfs.erofs foo.erofs.img foo" >>> >>> Hi, >>> >>> Here it is. >>> >> >> The problem here is that BTRFS could extract their internal >> xattrs ("inode properties") in their own namespace which >> don't belong to the original file itself like: >> >> XATTR_BTRFS_PREFIX "compression" -> btrfs.compression = zstd >> >> I think I need to introduce something to formally ignore >> these xattrs like: `--xattrs-exclude=pattern`: >> https://urldefense.com/v3/__https://www.gnu.org/software/tar/manual/html_node/Extended-File-Attributes.html__;!!PDiH4ENfjr2_Jw!Dqu_tH2zT_TKEhHNi2CZToYlmYFeZayWE1BQVQjZBZq4egjYS1bAVS7BasQJ9BETJoVINPEB-p06y7izIUmWk-2VnofOQZwLrw$ >> [gnu[.]org] > > I think that could work but a mode where EROFS keep the XATTRS it > recognises and ignore the ones it doesn't would probably prove useful > in the general case. > > I would suggest a --strict-xattrs mode behaving like it does today, > requiring explicit --xattrs-exclude to whitelist XATTRS. > > The default would be a relaxed mode where ignored XATTRS are reported > on stderr at INFO level. Would that make sense and be alright? Yes, but that needs a new version (maybe erofs-utils 1.8) though. Also I'm not sure how tar --xattrs works for "btrfs.compression", does it also work in a relaxed mode? Could you give more inputs & tries with "tar --xattrs"? > > Is there anything the BTRFS people could do to make their FS easier to > work with? Nope, I think it's unrelated to BTRFS but such xattrs are almost meaningless for EROFS to keep (since they are their own xattrs.) Thanks, Gao Xiang > > Gaël > >> >> Thanks, >> Gao Xiang >> >>> Thanks for your help, >>> Gaël >>> >>> > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 12:43 ` Gao Xiang @ 2024-03-12 13:14 ` Gael Donval 2024-03-12 14:15 ` Gao Xiang 0 siblings, 1 reply; 13+ messages in thread From: Gael Donval @ 2024-03-12 13:14 UTC (permalink / raw) To: hsiangkao@linux.alibaba.com, linux-erofs@lists.ozlabs.org On Tue, 2024-03-12 at 20:43 +0800, Gao Xiang wrote: > > > On 2024/3/12 20:01, Gael Donval wrote: > > On Tue, 2024-03-12 at 19:37 +0800, Gao Xiang wrote: > > > > > > > > > On 2024/3/12 19:07, Gael Donval wrote: > > > > > On 2024/3/12 18:44, Gao Xiang wrote: > > > > > > Hi Gael, > > > > > > > > > > > > On 2024/3/12 17:57, Gael Donval wrote: > > > > > > > Dear list, > > > > > > > > > > > > > > $ mkdir foo && touch foo/bar && mkfs.erofs > > > > > > > foo.erofs.img > > > > > > > foo > > > > > > > mkfs.erofs 1.7.1 > > > > > > > <E> erofs: failed to build shared xattrs: [Error > > > > > > > 61] No > > > > > > > data > > > > > > > available > > > > > > > <E> erofs: Could not format the device : [Error > > > > > > > 61] > > > > > > > No > > > > > > > data available > > > > > > > That is at a location backed by BTRFS (btrfs-progs > > > > > > > v6.7.1) on > > > > > > > kernel 6.8.0. > > > > > > > > > > > > > > If I use a TMPFS-supported folder instead all goes well. > > > > > > > > > > > > > > > > > > > > > Of course (NB "-x-1"), > > > > > > > > > > > > > > $ mkdir foo && touch foo/bar && mkfs.erofs -x-1 > > > > > > > foo.erofs.img > > > > > > > foo > > > > > > > > > > > > > > also works but is not how mkfs.erofs is meant to work in > > > > > > > the > > > > > > > general case. > > > > > > > > > > > > Thanks for your feedback. > > > > > > > > > > > > Currently I don't have some BTRFS environment, I could set > > > > > > up > > > > > > one > > > > > > later. > > > > > > Yet in parallel could you provide a full message of > > > > > > "strace mkfs.erofs -x-1 foo.erofs.img foo" on BTRFS too? > > > > > > > > > > sorry, I meant "strace mkfs.erofs foo.erofs.img foo" > > > > > > > > Hi, > > > > > > > > Here it is. > > > > > > > > > > The problem here is that BTRFS could extract their internal > > > xattrs ("inode properties") in their own namespace which > > > don't belong to the original file itself like: > > > > > > XATTR_BTRFS_PREFIX "compression" -> btrfs.compression = zstd > > > > > > I think I need to introduce something to formally ignore > > > these xattrs like: `--xattrs-exclude=pattern`: > > > https://urldefense.com/v3/__https://www.gnu.org/software/tar/manual/html_node/Extended-File-Attributes.html__;!!PDiH4ENfjr2_Jw!Dqu_tH2zT_TKEhHNi2CZToYlmYFeZayWE1BQVQjZBZq4egjYS1bAVS7BasQJ9BETJoVINPEB-p06y7izIUmWk-2VnofOQZwLrw$ > > > [gnu[.]org] > > > > I think that could work but a mode where EROFS keep the XATTRS it > > recognises and ignore the ones it doesn't would probably prove > > useful > > in the general case. > > > > I would suggest a --strict-xattrs mode behaving like it does today, > > requiring explicit --xattrs-exclude to whitelist XATTRS. > > > > The default would be a relaxed mode where ignored XATTRS are > > reported > > on stderr at INFO level. Would that make sense and be alright? > > Yes, but that needs a new version (maybe erofs-utils 1.8) though. Of course! > > Also I'm not sure how tar --xattrs works for "btrfs.compression", > does it also work in a relaxed mode? Could you give more inputs > & tries with "tar --xattrs"? Tar just stores XATTRS as their own field. When I $ tar --xattrs -cf foo.tar foo and inspect the tar file it appears XATTRS are stored as arbitrary SCHILY attributes: ./PaxHeaders/foo0000644000000000000000000000020114574033222010776 xustar0030 mtime=1710241426.458097911 30 atime=1710241426.458097911 30 ctime=1710241426.458097911 39 SCHILY.xattr.btrfs.compression=zstd foo/0000755000175000017500000000000014574033222011551 5ustar00gdonvalgdonval [...] So tar blindly serializes everything. > > > > > Is there anything the BTRFS people could do to make their FS easier > > to > > work with? > > Nope, I think it's unrelated to BTRFS but such xattrs are almost > meaningless for EROFS to keep (since they are their own xattrs.) Ah that's interesting. I suspect that in this instance the xattrs would not be useful at all but other components like SELinux uses xattrs to perform MAC and who knows what other xattrs-based schemes exist out there... Would it make sense then to do what tar does and blindly serialize (and expose) everything and provide a --xattrs-exclude=pattern in case people want something more fine-grained? The only main problem currently is that erofs tries to be smart about xattrs and errors out when it does not know about something. It should really be more graceful. Thanks, Gaël > > Thanks, > Gao Xiang > > > > > Gaël > > > > > > > > Thanks, > > > Gao Xiang > > > > > > > Thanks for your help, > > > > Gaël > > > > > > > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 13:14 ` Gael Donval @ 2024-03-12 14:15 ` Gao Xiang 2024-03-12 15:29 ` Gael Donval 0 siblings, 1 reply; 13+ messages in thread From: Gao Xiang @ 2024-03-12 14:15 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org On 2024/3/12 21:14, Gael Donval wrote: > On Tue, 2024-03-12 at 20:43 +0800, Gao Xiang wrote: >> ... >> Yes, but that needs a new version (maybe erofs-utils 1.8) though. > > Of course! > >> >> Also I'm not sure how tar --xattrs works for "btrfs.compression", >> does it also work in a relaxed mode? Could you give more inputs >> & tries with "tar --xattrs"? > > Tar just stores XATTRS as their own field. When I > > $ tar --xattrs -cf foo.tar foo > > and inspect the tar file it appears XATTRS are stored as arbitrary > SCHILY attributes: > > ./PaxHeaders/foo0000644000000000000000000000020114574033222010776 > xustar0030 mtime=1710241426.458097911 > 30 atime=1710241426.458097911 > 30 ctime=1710241426.458097911 > 39 SCHILY.xattr.btrfs.compression=zstd > foo/0000755000175000017500000000000014574033222011551 > 5ustar00gdonvalgdonval > [...] > > So tar blindly serializes everything. > > >> >>> >>> Is there anything the BTRFS people could do to make their FS easier >>> to >>> work with? >> >> Nope, I think it's unrelated to BTRFS but such xattrs are almost >> meaningless for EROFS to keep (since they are their own xattrs.) > > Ah that's interesting. > > I suspect that in this instance the xattrs would not be useful at all > but other components like SELinux uses xattrs to perform MAC and who > knows what other xattrs-based schemes exist out there... EROFS will keep common xattrs like selinux, but EROFS doesn't keep btrfs. namespace (or other arbitrary xattr namespaces) like other filesystems like ext4, xfs, f2fs, squashfs, etc. EROFS only supports "user.", "trusted." and some "system." domains. In other words, If EROFS has its own xattr namespace, I think other fses won't support "erofs." too, also see: https://lore.kernel.org/r/20230420092739.75464-1-o451686892@gmail.com Because that is simple, you don't even know if the content in some arbitrary namespace is static or dynamic. Or if it has some security issue to keep them. > > Would it make sense then to do what tar does and blindly serialize (and > expose) everything and provide a --xattrs-exclude=pattern in case > people want something more fine-grained? As I said above. Thanks, Gao Xiang > > The only main problem currently is that erofs tries to be smart about > xattrs and errors out when it does not know about something. It should > really be more graceful.> > Thanks, > Gaël > > >> >> Thanks, >> Gao Xiang >> >>> >>> Gaël >>> >>>> >>>> Thanks, >>>> Gao Xiang >>>> >>>>> Thanks for your help, >>>>> Gaël >>>>> >>>>> >>> > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 14:15 ` Gao Xiang @ 2024-03-12 15:29 ` Gael Donval 2024-03-12 15:39 ` Gao Xiang 0 siblings, 1 reply; 13+ messages in thread From: Gael Donval @ 2024-03-12 15:29 UTC (permalink / raw) To: hsiangkao@linux.alibaba.com, linux-erofs@lists.ozlabs.org On Tue, 2024-03-12 at 22:15 +0800, Gao Xiang wrote: > > > On 2024/3/12 21:14, Gael Donval wrote: > > On Tue, 2024-03-12 at 20:43 +0800, Gao Xiang wrote: > > > > > ... > > > > Yes, but that needs a new version (maybe erofs-utils 1.8) though. > > > > Of course! > > > > > > > > Also I'm not sure how tar --xattrs works for "btrfs.compression", > > > does it also work in a relaxed mode? Could you give more inputs > > > & tries with "tar --xattrs"? > > > > Tar just stores XATTRS as their own field. When I > > > > $ tar --xattrs -cf foo.tar foo > > > > and inspect the tar file it appears XATTRS are stored as arbitrary > > SCHILY attributes: > > > > > > ./PaxHeaders/foo0000644000000000000000000000020114574033222010776 > > xustar0030 mtime=1710241426.458097911 > > 30 atime=1710241426.458097911 > > 30 ctime=1710241426.458097911 > > 39 SCHILY.xattr.btrfs.compression=zstd > > foo/0000755000175000017500000000000014574033222011551 > > 5ustar00gdonvalgdonval > > [...] > > > > So tar blindly serializes everything. > > > > > > > > > > > > > > > Is there anything the BTRFS people could do to make their FS > > > > easier > > > > to > > > > work with? > > > > > > Nope, I think it's unrelated to BTRFS but such xattrs are almost > > > meaningless for EROFS to keep (since they are their own xattrs.) > > > > Ah that's interesting. > > > > I suspect that in this instance the xattrs would not be useful at > > all > > but other components like SELinux uses xattrs to perform MAC and > > who > > knows what other xattrs-based schemes exist out there... > > EROFS will keep common xattrs like selinux, but EROFS doesn't keep > btrfs. namespace (or other arbitrary xattr namespaces) like other > filesystems like ext4, xfs, f2fs, squashfs, etc. EROFS only supports > "user.", "trusted." and some "system." domains. In other words, If > EROFS has its own xattr namespace, I think other fses won't support > "erofs." too, also see: > > https://urldefense.com/v3/__https://lore.kernel.org/r/20230420092739.75464-1-o451686892@gmail.com__;!!PDiH4ENfjr2_Jw!BWdmKT7HAVU-ujqneKbEcblL0CJC9FhQ-MzikSN038KjlQ7HQqdbv1GpOtpZsBRAoW538RPHVpH3-kL7C5DrJV1lzJbQ_uScdQ$ > [lore[.]kernel[.]org] > > Because that is simple, you don't even know if the content in some > arbitrary namespace is static or dynamic. Or if it has some security > issue to keep them. I don't think supporting 3rd-party FS is the question here at all. If `btrfs.compression=zstd` is set, I don't expect erofs to do anything about it. No support. No understanding of what it is about even. Just that there is metadata there erofs ignores. The real question is more: does it matter, security-wise, if `btrfs.compression=zstd` is added as metadata (just like it is added in tar)? If there is no harm, there could be an argument to keep them as pure metadata (that erofs does NOT act on! no support). If there is harm, mkfs.erofs should strip anything that it does not support without producing an error. Either way, there should be no error. :) Gaël > > > > > Would it make sense then to do what tar does and blindly serialize > > (and > > expose) everything and provide a --xattrs-exclude=pattern in case > > people want something more fine-grained? > > As I said above. > > Thanks, > Gao Xiang > > > > > The only main problem currently is that erofs tries to be smart > > about > > xattrs and errors out when it does not know about something. It > > should > > really be more graceful.> > > Thanks, > > Gaël > > > > > > > > > > Thanks, > > > Gao Xiang > > > > > > > > > > > Gaël > > > > > > > > > > > > > > Thanks, > > > > > Gao Xiang > > > > > > > > > > > Thanks for your help, > > > > > > Gaël > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 15:29 ` Gael Donval @ 2024-03-12 15:39 ` Gao Xiang 2024-03-12 16:15 ` Gael Donval 0 siblings, 1 reply; 13+ messages in thread From: Gao Xiang @ 2024-03-12 15:39 UTC (permalink / raw) To: Gael Donval, linux-erofs@lists.ozlabs.org On 2024/3/12 23:29, Gael Donval wrote: > On Tue, 2024-03-12 at 22:15 +0800, Gao Xiang wrote: >> >> >> On 2024/3/12 21:14, Gael Donval wrote: >>> On Tue, 2024-03-12 at 20:43 +0800, Gao Xiang wrote: >>>> >> >> ... >> >>>> Yes, but that needs a new version (maybe erofs-utils 1.8) though. >>> >>> Of course! >>> >>>> >>>> Also I'm not sure how tar --xattrs works for "btrfs.compression", >>>> does it also work in a relaxed mode? Could you give more inputs >>>> & tries with "tar --xattrs"? >>> >>> Tar just stores XATTRS as their own field. When I >>> >>> $ tar --xattrs -cf foo.tar foo >>> >>> and inspect the tar file it appears XATTRS are stored as arbitrary >>> SCHILY attributes: >>> >>> >>> ./PaxHeaders/foo0000644000000000000000000000020114574033222010776 >>> xustar0030 mtime=1710241426.458097911 >>> 30 atime=1710241426.458097911 >>> 30 ctime=1710241426.458097911 >>> 39 SCHILY.xattr.btrfs.compression=zstd >>> foo/0000755000175000017500000000000014574033222011551 >>> 5ustar00gdonvalgdonval >>> [...] >>> >>> So tar blindly serializes everything. >>> >>> >>>> >>>>> >>>>> Is there anything the BTRFS people could do to make their FS >>>>> easier >>>>> to >>>>> work with? >>>> >>>> Nope, I think it's unrelated to BTRFS but such xattrs are almost >>>> meaningless for EROFS to keep (since they are their own xattrs.) >>> >>> Ah that's interesting. >>> >>> I suspect that in this instance the xattrs would not be useful at >>> all >>> but other components like SELinux uses xattrs to perform MAC and >>> who >>> knows what other xattrs-based schemes exist out there... >> >> EROFS will keep common xattrs like selinux, but EROFS doesn't keep >> btrfs. namespace (or other arbitrary xattr namespaces) like other >> filesystems like ext4, xfs, f2fs, squashfs, etc. EROFS only supports >> "user.", "trusted." and some "system." domains. In other words, If >> EROFS has its own xattr namespace, I think other fses won't support >> "erofs." too, also see: >> >> https://urldefense.com/v3/__https://lore.kernel.org/r/20230420092739.75464-1-o451686892@gmail.com__;!!PDiH4ENfjr2_Jw!BWdmKT7HAVU-ujqneKbEcblL0CJC9FhQ-MzikSN038KjlQ7HQqdbv1GpOtpZsBRAoW538RPHVpH3-kL7C5DrJV1lzJbQ_uScdQ$ >> [lore[.]kernel[.]org] >> >> Because that is simple, you don't even know if the content in some >> arbitrary namespace is static or dynamic. Or if it has some security >> issue to keep them. > > I don't think supporting 3rd-party FS is the question here at all. If > `btrfs.compression=zstd` is set, I don't expect erofs to do anything > about it. No support. No understanding of what it is about even. Just > that there is metadata there erofs ignores. For `btrfs.compression=zstd`, yes, but for many many other unknown xattrs, that is no. And since EROFS can be directly mounted, it's not a good idea to show special xattrs owned by other filesystems. > > The real question is more: does it matter, security-wise, if > `btrfs.compression=zstd` is added as metadata (just like it is added in > tar)? > > If there is no harm, there could be an argument to keep them as pure > metadata (that erofs does NOT act on! no support). You could try to extract this to ext4/xfs. I guess `tar` may print something, but I don't know. > > If there is harm, mkfs.erofs should strip anything that it does not > support without producing an error. > > Either way, there should be no error. :) Fair enough, I think I will do this for erofs-utils 1.8, but I tend to consider unsupported xattr namespaces as warn messages, users could silence these with proper '-d' message levels. Thanks, Gao Xiang ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: mkfs.erofs fails (failed to build shared xattrs, err 61) 2024-03-12 15:39 ` Gao Xiang @ 2024-03-12 16:15 ` Gael Donval 0 siblings, 0 replies; 13+ messages in thread From: Gael Donval @ 2024-03-12 16:15 UTC (permalink / raw) To: hsiangkao@linux.alibaba.com, linux-erofs@lists.ozlabs.org > > If there is harm, mkfs.erofs should strip anything that it does not > > support without producing an error. > > > > Either way, there should be no error. :) > > Fair enough, I think I will do this for erofs-utils 1.8, but I tend > to consider unsupported xattr namespaces as warn messages, users > could silence these with proper '-d' message levels. I'm absolutely fine with that as long as mkfs.erofs goes through with generating the FS (it does not generate anything right now) and returns success to the shell. Thanks a lot for your help, Gaël > > Thanks, > Gao Xiang ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2024-03-12 16:16 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-03-12 9:57 mkfs.erofs fails (failed to build shared xattrs, err 61) Gael Donval 2024-03-12 10:44 ` Gao Xiang 2024-03-12 10:45 ` Gao Xiang 2024-03-12 11:07 ` Gael Donval 2024-03-12 11:37 ` Gao Xiang 2024-03-12 11:57 ` Gao Xiang 2024-03-12 12:01 ` Gael Donval 2024-03-12 12:43 ` Gao Xiang 2024-03-12 13:14 ` Gael Donval 2024-03-12 14:15 ` Gao Xiang 2024-03-12 15:29 ` Gael Donval 2024-03-12 15:39 ` Gao Xiang 2024-03-12 16:15 ` Gael Donval
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.