[BUG] fuse/virtiofs: kernel module build fail

[BUG] fuse/virtiofs: kernel module build fail

[Brian Foster]

Hi folks,

I run kernel compiles quite a bit over virtiofs in some of my local test
setups and recently ran into an issue building xfs.ko once I had a
v6.16-rc kernel installed in my guest. The test case is a simple:

  make -j N M=fs/xfs clean; make -j N M=fs/xfs

... and ends up spitting out link time errors like this as of commit
63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):

...
  CC [M]  xfs.mod.o
  CC [M]  .module-common.o
  LD [M]  xfs.ko  
  BTF [M] xfs.ko  
die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
error decoding cu i_mmap_rwsem
error decoding cu 
...
error decoding cu 
pahole: xfs.ko: Invalid argument
make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
make[3]: *** Deleting file 'xfs.ko'
make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
make[1]: Leaving directory '/root/repos/linux/fs/xfs'
make: *** [Makefile:248: __sub-make] Error 2

... or this on latest master:

...
  LD [M]  fs/xfs/xfs.o
fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
make[4]: *** Deleting file 'fs/xfs/xfs.o'
make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
make[2]: *** [scripts/Makefile.build:554: fs] Error 2
make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
make: *** [Makefile:248: __sub-make] Error 2

The latter failure is what I saw through most of a bisect so I suspect
one of the related followon commits alters the failure characteristic
from the former, but I've not confirmed that. Also note out of
convenience my test was to just recompile xfs.ko out of the same tree I
was bisecting from because the failures were consistent and seemed to be
a runtime kernel issue and not a source tree issue.

I haven't had a chance to dig any further than this (and JFYI I'm
probably not going to be responsive through the rest of today). I just
completed the bisect and wanted to get it on list sooner rather than
later..

Brian

Re: [BUG] fuse/virtiofs: kernel module build fail

[Joanne Koong]

On Thu, Jun 12, 2025 at 4:19 AM Brian Foster <bfoster@redhat.com> wrote:
>
> Hi folks,
>
> I run kernel compiles quite a bit over virtiofs in some of my local test
> setups and recently ran into an issue building xfs.ko once I had a
> v6.16-rc kernel installed in my guest. The test case is a simple:
>
>   make -j N M=fs/xfs clean; make -j N M=fs/xfs

Hi Brian,

If I'm understanding your setup correctly, basically you have the
v6.16-rc kernel running on a VM, on that VM you mounted a virtiofs
directory that references a linux repo that's on your host OS, and
then from your VM you are compiling the fs/xfs module in that shared
linux repo?

I tried this on my local setup but I'm seeing some other issues:

make[1]: Entering directory '/home/vmuser/linux/linux/fs/xfs'
  LD [M]  xfs.o
xfs.o: warning: objtool: __traceiter_xfs_attr_list_sf+0x23:
unannotated intra-function call
make[3]: *** [/home/vmuser/linux/linux/scripts/Makefile.build:501:
xfs.o] Error 255
make[3]: *** Deleting file 'xfs.o'
make[2]: *** [/home/vmuser/linux/linux/Makefile:2006: .] Error 2
make[1]: *** [/home/vmuser/linux/linux/Makefile:248: __sub-make] Error 2
make[1]: Leaving directory '/home/vmuser/linux/linux/fs/xfs'
make: *** [Makefile:248: __sub-make] Error 2

Did you also run into these issues when you were compiling?

Taking a look at what 63c69ad3d18a ("fuse: refactor
fuse_fill_write_pages()") does, it seems odd to me that the changes in
that commit would lead to the issues you're seeing - that commit
doesn't alter structs or memory layouts in any way. I'll keep trying
to repro the issue you're seeing.

>
> ... and ends up spitting out link time errors like this as of commit
> 63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):
>
> ...
>   CC [M]  xfs.mod.o
>   CC [M]  .module-common.o
>   LD [M]  xfs.ko
>   BTF [M] xfs.ko
> die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
> error decoding cu i_mmap_rwsem
> error decoding cu
> ...
> error decoding cu
> pahole: xfs.ko: Invalid argument
> make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
> make[3]: *** Deleting file 'xfs.ko'
> make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
> make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/root/repos/linux/fs/xfs'
> make: *** [Makefile:248: __sub-make] Error 2
>
> ... or this on latest master:
>
> ...
>   LD [M]  fs/xfs/xfs.o
> fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
> make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
> make[4]: *** Deleting file 'fs/xfs/xfs.o'
> make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
> make[2]: *** [scripts/Makefile.build:554: fs] Error 2
> make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
> make: *** [Makefile:248: __sub-make] Error 2
>
> The latter failure is what I saw through most of a bisect so I suspect
> one of the related followon commits alters the failure characteristic
> from the former, but I've not confirmed that. Also note out of
> convenience my test was to just recompile xfs.ko out of the same tree I
> was bisecting from because the failures were consistent and seemed to be
> a runtime kernel issue and not a source tree issue.
>
> I haven't had a chance to dig any further than this (and JFYI I'm
> probably not going to be responsive through the rest of today). I just
> completed the bisect and wanted to get it on list sooner rather than
> later..
>
> Brian
>

Re: [BUG] fuse/virtiofs: kernel module build fail

[Brian Foster]

On Thu, Jun 12, 2025 at 02:56:56PM -0700, Joanne Koong wrote:
> On Thu, Jun 12, 2025 at 4:19 AM Brian Foster <bfoster@redhat.com> wrote:
> >
> > Hi folks,
> >
> > I run kernel compiles quite a bit over virtiofs in some of my local test
> > setups and recently ran into an issue building xfs.ko once I had a
> > v6.16-rc kernel installed in my guest. The test case is a simple:
> >
> >   make -j N M=fs/xfs clean; make -j N M=fs/xfs
> 
> Hi Brian,
> 

Hi Joanne,

> If I'm understanding your setup correctly, basically you have the
> v6.16-rc kernel running on a VM, on that VM you mounted a virtiofs
> directory that references a linux repo that's on your host OS, and
> then from your VM you are compiling the fs/xfs module in that shared
> linux repo?
> 

Yep. Note again that I happen to be using the same repo that I was
bisecting, so technically the test case of recompiling the kernel module
is targeting different code each time, but the failure was so consistent
that I believe it to be a runtime issue.

> I tried this on my local setup but I'm seeing some other issues:
> 
> make[1]: Entering directory '/home/vmuser/linux/linux/fs/xfs'
>   LD [M]  xfs.o
> xfs.o: warning: objtool: __traceiter_xfs_attr_list_sf+0x23:
> unannotated intra-function call
> make[3]: *** [/home/vmuser/linux/linux/scripts/Makefile.build:501:
> xfs.o] Error 255
> make[3]: *** Deleting file 'xfs.o'
> make[2]: *** [/home/vmuser/linux/linux/Makefile:2006: .] Error 2
> make[1]: *** [/home/vmuser/linux/linux/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/home/vmuser/linux/linux/fs/xfs'
> make: *** [Makefile:248: __sub-make] Error 2
> 
> Did you also run into these issues when you were compiling?
> 

I don't recall seeing that specific error, but TBH this kind of looks in
line with what I'm seeing in general. My suspicion is that this is not
actually a source code error, but something is corrupted somehow or
another via virtiofs.

As a quick additional test, I just built my same repo from my host
system (running a distro v6.14 kernel) without any issue. Then if I do
the kernel module rebuild test of the same exact repo over virtiofs from
the guest, the error occurs.

What is interesting is that if I try the same thing with another module
(i.e. fs/fuse, fs/nfs), the build seems to work fine, so maybe there is
something unique to XFS going on here. As a followup to that, I set my
repo back to v6.15 but still reproduce the same phenomenon: build
failure when compiling over virtiofs and not from the host.. :/

> Taking a look at what 63c69ad3d18a ("fuse: refactor
> fuse_fill_write_pages()") does, it seems odd to me that the changes in
> that commit would lead to the issues you're seeing - that commit
> doesn't alter structs or memory layouts in any way. I'll keep trying
> to repro the issue you're seeing.
> 

Apologies I haven't really had a chance to look at the code.. It's
certainly possible I botched something in the bisect or misunderstood
some tooling differences or whatever, but I'm pretty sure I at least
tried the target commit and the immediate preceding commit after the
bisect to double check where the failing starts to happen.

That said, I suspect you are actually reproducing the same general
problem with your test above, regardless of whether the analysis is
wrong. If you're able, have you tried whether that same compile works
from a host (or non-virtiofs) env? Thanks.

Brian

> >
> > ... and ends up spitting out link time errors like this as of commit
> > 63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):
> >
> > ...
> >   CC [M]  xfs.mod.o
> >   CC [M]  .module-common.o
> >   LD [M]  xfs.ko
> >   BTF [M] xfs.ko
> > die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
> > error decoding cu i_mmap_rwsem
> > error decoding cu
> > ...
> > error decoding cu
> > pahole: xfs.ko: Invalid argument
> > make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
> > make[3]: *** Deleting file 'xfs.ko'
> > make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
> > make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
> > make[1]: Leaving directory '/root/repos/linux/fs/xfs'
> > make: *** [Makefile:248: __sub-make] Error 2
> >
> > ... or this on latest master:
> >
> > ...
> >   LD [M]  fs/xfs/xfs.o
> > fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
> > make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
> > make[4]: *** Deleting file 'fs/xfs/xfs.o'
> > make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
> > make[2]: *** [scripts/Makefile.build:554: fs] Error 2
> > make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
> > make: *** [Makefile:248: __sub-make] Error 2
> >
> > The latter failure is what I saw through most of a bisect so I suspect
> > one of the related followon commits alters the failure characteristic
> > from the former, but I've not confirmed that. Also note out of
> > convenience my test was to just recompile xfs.ko out of the same tree I
> > was bisecting from because the failures were consistent and seemed to be
> > a runtime kernel issue and not a source tree issue.
> >
> > I haven't had a chance to dig any further than this (and JFYI I'm
> > probably not going to be responsive through the rest of today). I just
> > completed the bisect and wanted to get it on list sooner rather than
> > later..
> >
> > Brian
> >
> 

Re: [BUG] fuse/virtiofs: kernel module build fail

[Luis Henriques]

On Thu, Jun 12 2025, Joanne Koong wrote:

> On Thu, Jun 12, 2025 at 4:19 AM Brian Foster <bfoster@redhat.com> wrote:
>>
>> Hi folks,
>>
>> I run kernel compiles quite a bit over virtiofs in some of my local test
>> setups and recently ran into an issue building xfs.ko once I had a
>> v6.16-rc kernel installed in my guest. The test case is a simple:
>>
>>   make -j N M=fs/xfs clean; make -j N M=fs/xfs
>
> Hi Brian,
>
> If I'm understanding your setup correctly, basically you have the
> v6.16-rc kernel running on a VM, on that VM you mounted a virtiofs
> directory that references a linux repo that's on your host OS, and
> then from your VM you are compiling the fs/xfs module in that shared
> linux repo?
>
> I tried this on my local setup but I'm seeing some other issues:
>
> make[1]: Entering directory '/home/vmuser/linux/linux/fs/xfs'
>   LD [M]  xfs.o
> xfs.o: warning: objtool: __traceiter_xfs_attr_list_sf+0x23:
> unannotated intra-function call
> make[3]: *** [/home/vmuser/linux/linux/scripts/Makefile.build:501:
> xfs.o] Error 255
> make[3]: *** Deleting file 'xfs.o'
> make[2]: *** [/home/vmuser/linux/linux/Makefile:2006: .] Error 2
> make[1]: *** [/home/vmuser/linux/linux/Makefile:248: __sub-make] Error 2
> make[1]: Leaving directory '/home/vmuser/linux/linux/fs/xfs'
> make: *** [Makefile:248: __sub-make] Error 2
>
> Did you also run into these issues when you were compiling?

This is probably just a shot in the dark, but I remember seeing similar
build failures long time ago due to virtiofs caching.  I don't remember
the details, but maybe it's worth checking that.  I *think* that what
fixed it for me was to use '--cache auto'.

Cheers,
-- 
Luís


> Taking a look at what 63c69ad3d18a ("fuse: refactor
> fuse_fill_write_pages()") does, it seems odd to me that the changes in
> that commit would lead to the issues you're seeing - that commit
> doesn't alter structs or memory layouts in any way. I'll keep trying
> to repro the issue you're seeing.
>
>>
>> ... and ends up spitting out link time errors like this as of commit
>> 63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):
>>
>> ...
>>   CC [M]  xfs.mod.o
>>   CC [M]  .module-common.o
>>   LD [M]  xfs.ko
>>   BTF [M] xfs.ko
>> die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
>> error decoding cu i_mmap_rwsem
>> error decoding cu
>> ...
>> error decoding cu
>> pahole: xfs.ko: Invalid argument
>> make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
>> make[3]: *** Deleting file 'xfs.ko'
>> make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
>> make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
>> make[1]: Leaving directory '/root/repos/linux/fs/xfs'
>> make: *** [Makefile:248: __sub-make] Error 2
>>
>> ... or this on latest master:
>>
>> ...
>>   LD [M]  fs/xfs/xfs.o
>> fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
>> make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
>> make[4]: *** Deleting file 'fs/xfs/xfs.o'
>> make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
>> make[2]: *** [scripts/Makefile.build:554: fs] Error 2
>> make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
>> make: *** [Makefile:248: __sub-make] Error 2
>>
>> The latter failure is what I saw through most of a bisect so I suspect
>> one of the related followon commits alters the failure characteristic
>> from the former, but I've not confirmed that. Also note out of
>> convenience my test was to just recompile xfs.ko out of the same tree I
>> was bisecting from because the failures were consistent and seemed to be
>> a runtime kernel issue and not a source tree issue.
>>
>> I haven't had a chance to dig any further than this (and JFYI I'm
>> probably not going to be responsive through the rest of today). I just
>> completed the bisect and wanted to get it on list sooner rather than
>> later..
>>
>> Brian
>>
>

Re: [BUG] fuse/virtiofs: kernel module build fail

[Joanne Koong]

On Fri, Jun 13, 2025 at 7:06 AM Luis Henriques <luis@igalia.com> wrote:
>
> On Thu, Jun 12 2025, Joanne Koong wrote:
>
> > On Thu, Jun 12, 2025 at 4:19 AM Brian Foster <bfoster@redhat.com> wrote:
> >>
> >> Hi folks,
> >>
> >> I run kernel compiles quite a bit over virtiofs in some of my local test
> >> setups and recently ran into an issue building xfs.ko once I had a
> >> v6.16-rc kernel installed in my guest. The test case is a simple:
> >>
> >>   make -j N M=fs/xfs clean; make -j N M=fs/xfs
> >
> > Hi Brian,
> >
> > If I'm understanding your setup correctly, basically you have the
> > v6.16-rc kernel running on a VM, on that VM you mounted a virtiofs
> > directory that references a linux repo that's on your host OS, and
> > then from your VM you are compiling the fs/xfs module in that shared
> > linux repo?
> >
> > I tried this on my local setup but I'm seeing some other issues:
> >
> > make[1]: Entering directory '/home/vmuser/linux/linux/fs/xfs'
> >   LD [M]  xfs.o
> > xfs.o: warning: objtool: __traceiter_xfs_attr_list_sf+0x23:
> > unannotated intra-function call
> > make[3]: *** [/home/vmuser/linux/linux/scripts/Makefile.build:501:
> > xfs.o] Error 255
> > make[3]: *** Deleting file 'xfs.o'
> > make[2]: *** [/home/vmuser/linux/linux/Makefile:2006: .] Error 2
> > make[1]: *** [/home/vmuser/linux/linux/Makefile:248: __sub-make] Error 2
> > make[1]: Leaving directory '/home/vmuser/linux/linux/fs/xfs'
> > make: *** [Makefile:248: __sub-make] Error 2
> >
> > Did you also run into these issues when you were compiling?
>
> This is probably just a shot in the dark, but I remember seeing similar
> build failures long time ago due to virtiofs caching.  I don't remember
> the details, but maybe it's worth checking that.  I *think* that what
> fixed it for me was to use '--cache auto'.

Thanks for the tip. I just tried it again with --cache=auto but I'm
still seeing the same issue.

>
> Cheers,
> --
> Luís
>
>
> > Taking a look at what 63c69ad3d18a ("fuse: refactor
> > fuse_fill_write_pages()") does, it seems odd to me that the changes in
> > that commit would lead to the issues you're seeing - that commit
> > doesn't alter structs or memory layouts in any way. I'll keep trying
> > to repro the issue you're seeing.
> >
> >>
> >> ... and ends up spitting out link time errors like this as of commit
> >> 63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):
> >>
> >> ...
> >>   CC [M]  xfs.mod.o
> >>   CC [M]  .module-common.o
> >>   LD [M]  xfs.ko
> >>   BTF [M] xfs.ko
> >> die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
> >> error decoding cu i_mmap_rwsem
> >> error decoding cu
> >> ...
> >> error decoding cu
> >> pahole: xfs.ko: Invalid argument
> >> make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
> >> make[3]: *** Deleting file 'xfs.ko'
> >> make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
> >> make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
> >> make[1]: Leaving directory '/root/repos/linux/fs/xfs'
> >> make: *** [Makefile:248: __sub-make] Error 2
> >>
> >> ... or this on latest master:
> >>
> >> ...
> >>   LD [M]  fs/xfs/xfs.o
> >> fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
> >> make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
> >> make[4]: *** Deleting file 'fs/xfs/xfs.o'
> >> make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
> >> make[2]: *** [scripts/Makefile.build:554: fs] Error 2
> >> make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
> >> make: *** [Makefile:248: __sub-make] Error 2
> >>
> >> The latter failure is what I saw through most of a bisect so I suspect
> >> one of the related followon commits alters the failure characteristic
> >> from the former, but I've not confirmed that. Also note out of
> >> convenience my test was to just recompile xfs.ko out of the same tree I
> >> was bisecting from because the failures were consistent and seemed to be
> >> a runtime kernel issue and not a source tree issue.
> >>
> >> I haven't had a chance to dig any further than this (and JFYI I'm
> >> probably not going to be responsive through the rest of today). I just
> >> completed the bisect and wanted to get it on list sooner rather than
> >> later..
> >>
> >> Brian
> >>
> >
>

Re: [BUG] fuse/virtiofs: kernel module build fail

[Joanne Koong]

On Fri, Jun 13, 2025 at 4:41 AM Brian Foster <bfoster@redhat.com> wrote:
>
> On Thu, Jun 12, 2025 at 02:56:56PM -0700, Joanne Koong wrote:
> > On Thu, Jun 12, 2025 at 4:19 AM Brian Foster <bfoster@redhat.com> wrote:
> > >
> > > Hi folks,
> > >
> > > I run kernel compiles quite a bit over virtiofs in some of my local test
> > > setups and recently ran into an issue building xfs.ko once I had a
> > > v6.16-rc kernel installed in my guest. The test case is a simple:
> > >
> > >   make -j N M=fs/xfs clean; make -j N M=fs/xfs
> >
> > Hi Brian,
> >
>
> Hi Joanne,
>
> > If I'm understanding your setup correctly, basically you have the
> > v6.16-rc kernel running on a VM, on that VM you mounted a virtiofs
> > directory that references a linux repo that's on your host OS, and
> > then from your VM you are compiling the fs/xfs module in that shared
> > linux repo?
> >
>
> Yep. Note again that I happen to be using the same repo that I was
> bisecting, so technically the test case of recompiling the kernel module
> is targeting different code each time, but the failure was so consistent
> that I believe it to be a runtime issue.
>
> > I tried this on my local setup but I'm seeing some other issues:
> >
> > make[1]: Entering directory '/home/vmuser/linux/linux/fs/xfs'
> >   LD [M]  xfs.o
> > xfs.o: warning: objtool: __traceiter_xfs_attr_list_sf+0x23:
> > unannotated intra-function call
> > make[3]: *** [/home/vmuser/linux/linux/scripts/Makefile.build:501:
> > xfs.o] Error 255
> > make[3]: *** Deleting file 'xfs.o'
> > make[2]: *** [/home/vmuser/linux/linux/Makefile:2006: .] Error 2
> > make[1]: *** [/home/vmuser/linux/linux/Makefile:248: __sub-make] Error 2
> > make[1]: Leaving directory '/home/vmuser/linux/linux/fs/xfs'
> > make: *** [Makefile:248: __sub-make] Error 2
> >
> > Did you also run into these issues when you were compiling?
> >
>
> I don't recall seeing that specific error, but TBH this kind of looks in
> line with what I'm seeing in general. My suspicion is that this is not
> actually a source code error, but something is corrupted somehow or
> another via virtiofs.

This is indeed the same issue as the one you saw. I was able to repro
it on that commit you pointed to  (more details below)

>
> As a quick additional test, I just built my same repo from my host
> system (running a distro v6.14 kernel) without any issue. Then if I do
> the kernel module rebuild test of the same exact repo over virtiofs from
> the guest, the error occurs.
>
> What is interesting is that if I try the same thing with another module
> (i.e. fs/fuse, fs/nfs), the build seems to work fine, so maybe there is
> something unique to XFS going on here. As a followup to that, I set my
> repo back to v6.15 but still reproduce the same phenomenon: build
> failure when compiling over virtiofs and not from the host.. :/
>
> > Taking a look at what 63c69ad3d18a ("fuse: refactor
> > fuse_fill_write_pages()") does, it seems odd to me that the changes in
> > that commit would lead to the issues you're seeing - that commit
> > doesn't alter structs or memory layouts in any way. I'll keep trying
> > to repro the issue you're seeing.
> >
>
> Apologies I haven't really had a chance to look at the code.. It's
> certainly possible I botched something in the bisect or misunderstood
> some tooling differences or whatever, but I'm pretty sure I at least
> tried the target commit and the immediate preceding commit after the
> bisect to double check where the failing starts to happen.
>
> That said, I suspect you are actually reproducing the same general
> problem with your test above, regardless of whether the analysis is
> wrong. If you're able, have you tried whether that same compile works
> from a host (or non-virtiofs) env? Thanks.

You didn't mess up the bisect, I was able to verify that it is that
commit that causes the issue. I misunderstood the error message and
thought it was complaining about alignment in a struct being broken
somewhere.

This fixes the commit:
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -1147,7 +1147,7 @@ static ssize_t fuse_send_write_pages(struct
fuse_io_args *ia,
 static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
                                     struct address_space *mapping,
                                     struct iov_iter *ii, loff_t pos,
-                                    unsigned int max_pages)
+                                    unsigned int max_folios)
 {
        struct fuse_args_pages *ap = &ia->ap;
        struct fuse_conn *fc = get_fuse_conn(mapping->host);
@@ -1157,12 +1157,11 @@ static ssize_t fuse_fill_write_pages(struct
fuse_io_args *ia,
        int err = 0;

        num = min(iov_iter_count(ii), fc->max_write);
-       num = min(num, max_pages << PAGE_SHIFT);

        ap->args.in_pages = true;
        ap->descs[0].offset = offset;

-       while (num) {
+       while (num && ap->num_folios < max_folios) {
                size_t tmp;
                struct folio *folio;
                pgoff_t index = pos >> PAGE_SHIFT;


The bug is that I incorrectly assumed that I could use max_pages <<
PAGE_SHIFT as the upper limit for how many bytes to copy in, but
there's the possibility that the copy_folio_from_iter_atomic() call
that we do can copy over bytes from the iov_iter that are less than
the length of the folio, so using max_pages << PAGE_SHIFT as the bound
for max_pages is wrong.

I ran the fix locally on top of origin/master (commit 27605c8c0) as
well and verified that it fixes the issue. I'll send this fix
upstream.

Sorry for the inconvenience. Hope this bug didn't waste too much of
your time. Thanks for reporting it.

>
> Brian
>
> > >
> > > ... and ends up spitting out link time errors like this as of commit
> > > 63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):
> > >
> > > ...
> > >   CC [M]  xfs.mod.o
> > >   CC [M]  .module-common.o
> > >   LD [M]  xfs.ko
> > >   BTF [M] xfs.ko
> > > die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
> > > error decoding cu i_mmap_rwsem
> > > error decoding cu
> > > ...
> > > error decoding cu
> > > pahole: xfs.ko: Invalid argument
> > > make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
> > > make[3]: *** Deleting file 'xfs.ko'
> > > make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
> > > make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
> > > make[1]: Leaving directory '/root/repos/linux/fs/xfs'
> > > make: *** [Makefile:248: __sub-make] Error 2
> > >
> > > ... or this on latest master:
> > >
> > > ...
> > >   LD [M]  fs/xfs/xfs.o
> > > fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
> > > make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
> > > make[4]: *** Deleting file 'fs/xfs/xfs.o'
> > > make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
> > > make[2]: *** [scripts/Makefile.build:554: fs] Error 2
> > > make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
> > > make: *** [Makefile:248: __sub-make] Error 2
> > >
> > > The latter failure is what I saw through most of a bisect so I suspect
> > > one of the related followon commits alters the failure characteristic
> > > from the former, but I've not confirmed that. Also note out of
> > > convenience my test was to just recompile xfs.ko out of the same tree I
> > > was bisecting from because the failures were consistent and seemed to be
> > > a runtime kernel issue and not a source tree issue.
> > >
> > > I haven't had a chance to dig any further than this (and JFYI I'm
> > > probably not going to be responsive through the rest of today). I just
> > > completed the bisect and wanted to get it on list sooner rather than
> > > later..
> > >
> > > Brian
> > >
> >
>

Re: [BUG] fuse/virtiofs: kernel module build fail

[Brian Foster]

On Fri, Jun 13, 2025 at 04:42:00PM -0700, Joanne Koong wrote:
> On Fri, Jun 13, 2025 at 4:41 AM Brian Foster <bfoster@redhat.com> wrote:
> >
> > On Thu, Jun 12, 2025 at 02:56:56PM -0700, Joanne Koong wrote:
> > > On Thu, Jun 12, 2025 at 4:19 AM Brian Foster <bfoster@redhat.com> wrote:
...
> 
> You didn't mess up the bisect, I was able to verify that it is that
> commit that causes the issue. I misunderstood the error message and
> thought it was complaining about alignment in a struct being broken
> somewhere.
> 
> This fixes the commit:
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -1147,7 +1147,7 @@ static ssize_t fuse_send_write_pages(struct
> fuse_io_args *ia,
>  static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
>                                      struct address_space *mapping,
>                                      struct iov_iter *ii, loff_t pos,
> -                                    unsigned int max_pages)
> +                                    unsigned int max_folios)
>  {
>         struct fuse_args_pages *ap = &ia->ap;
>         struct fuse_conn *fc = get_fuse_conn(mapping->host);
> @@ -1157,12 +1157,11 @@ static ssize_t fuse_fill_write_pages(struct
> fuse_io_args *ia,
>         int err = 0;
> 
>         num = min(iov_iter_count(ii), fc->max_write);
> -       num = min(num, max_pages << PAGE_SHIFT);
> 
>         ap->args.in_pages = true;
>         ap->descs[0].offset = offset;
> 
> -       while (num) {
> +       while (num && ap->num_folios < max_folios) {
>                 size_t tmp;
>                 struct folio *folio;
>                 pgoff_t index = pos >> PAGE_SHIFT;
> 
> 
> The bug is that I incorrectly assumed that I could use max_pages <<
> PAGE_SHIFT as the upper limit for how many bytes to copy in, but
> there's the possibility that the copy_folio_from_iter_atomic() call
> that we do can copy over bytes from the iov_iter that are less than
> the length of the folio, so using max_pages << PAGE_SHIFT as the bound
> for max_pages is wrong.
> 
> I ran the fix locally on top of origin/master (commit 27605c8c0) as
> well and verified that it fixes the issue. I'll send this fix
> upstream.
> 

Ah, great thanks. That makes sense.. I'll give it a try.

> Sorry for the inconvenience. Hope this bug didn't waste too much of
> your time. Thanks for reporting it.
> 

No worries at all, thanks for tracking it down!

Brian

> >
> > Brian
> >
> > > >
> > > > ... and ends up spitting out link time errors like this as of commit
> > > > 63c69ad3d18a ("fuse: refactor fuse_fill_write_pages()"):
> > > >
> > > > ...
> > > >   CC [M]  xfs.mod.o
> > > >   CC [M]  .module-common.o
> > > >   LD [M]  xfs.ko
> > > >   BTF [M] xfs.ko
> > > > die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got subprogram (0x2e) @ ed957!
> > > > error decoding cu i_mmap_rwsem
> > > > error decoding cu
> > > > ...
> > > > error decoding cu
> > > > pahole: xfs.ko: Invalid argument
> > > > make[3]: *** [/root/repos/linux/scripts/Makefile.modfinal:57: xfs.ko] Error 1
> > > > make[3]: *** Deleting file 'xfs.ko'
> > > > make[2]: *** [/root/repos/linux/Makefile:1937: modules] Error 2
> > > > make[1]: *** [/root/repos/linux/Makefile:248: __sub-make] Error 2
> > > > make[1]: Leaving directory '/root/repos/linux/fs/xfs'
> > > > make: *** [Makefile:248: __sub-make] Error 2
> > > >
> > > > ... or this on latest master:
> > > >
> > > > ...
> > > >   LD [M]  fs/xfs/xfs.o
> > > > fs/xfs/xfs.o: error: objtool: can't find reloc entry symbol 2145964924 for .rela.text
> > > > make[4]: *** [scripts/Makefile.build:501: fs/xfs/xfs.o] Error 1
> > > > make[4]: *** Deleting file 'fs/xfs/xfs.o'
> > > > make[3]: *** [scripts/Makefile.build:554: fs/xfs] Error 2
> > > > make[2]: *** [scripts/Makefile.build:554: fs] Error 2
> > > > make[1]: *** [/root/repos/linux/Makefile:2006: .] Error 2
> > > > make: *** [Makefile:248: __sub-make] Error 2
> > > >
> > > > The latter failure is what I saw through most of a bisect so I suspect
> > > > one of the related followon commits alters the failure characteristic
> > > > from the former, but I've not confirmed that. Also note out of
> > > > convenience my test was to just recompile xfs.ko out of the same tree I
> > > > was bisecting from because the failures were consistent and seemed to be
> > > > a runtime kernel issue and not a source tree issue.
> > > >
> > > > I haven't had a chance to dig any further than this (and JFYI I'm
> > > > probably not going to be responsive through the rest of today). I just
> > > > completed the bisect and wanted to get it on list sooner rather than
> > > > later..
> > > >
> > > > Brian
> > > >
> > >
> >
>