Re: BTFIDS: FAILED unresolved symbol udp6_sock

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Qais Yousef]

Hi Jiri

On 12/29/20 18:34, Jiri Olsa wrote:
> On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > Hi
> > 
> > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > stage
> > 
> > 	FAILED unresolved symbol udp6_sock
> > 
> > I cross compile for arm64. My .config is attached.
> > 
> > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > 
> > Have you seen this before? I couldn't find a specific report about this
> > problem.
> > 
> > Let me know if you need more info.
> 
> hi,
> this looks like symptom of the gcc DWARF bug we were
> dealing with recently:
> 
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
>   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> 
> what pahole/gcc version are you using?

I'm on gcc 9.3.0

	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

I was on pahole v1.17. I moved to v1.19 but I still see the same problem.

Thanks

--
Qais Yousef

> 
> we fixed pahole (v1.19) to workaround the issue and AFAIK
> there's already gcc fix as well

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Jiri Olsa]

On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> Hi Jiri
> 
> On 12/29/20 18:34, Jiri Olsa wrote:
> > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > Hi
> > > 
> > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > stage
> > > 
> > > 	FAILED unresolved symbol udp6_sock
> > > 
> > > I cross compile for arm64. My .config is attached.
> > > 
> > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > 
> > > Have you seen this before? I couldn't find a specific report about this
> > > problem.
> > > 
> > > Let me know if you need more info.
> > 
> > hi,
> > this looks like symptom of the gcc DWARF bug we were
> > dealing with recently:
> > 
> >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > 
> > what pahole/gcc version are you using?
> 
> I'm on gcc 9.3.0
> 
> 	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> 
> I was on pahole v1.17. I moved to v1.19 but I still see the same problem.

I can reproduce with your .config, but make 'defconfig' works,
so I guess it's some config option issue, I'll check later today

jirka

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Jiri Olsa]

On Wed, Dec 30, 2020 at 10:03:37AM +0100, Jiri Olsa wrote:
> On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> > Hi Jiri
> > 
> > On 12/29/20 18:34, Jiri Olsa wrote:
> > > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > > Hi
> > > > 
> > > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > > stage
> > > > 
> > > > 	FAILED unresolved symbol udp6_sock
> > > > 
> > > > I cross compile for arm64. My .config is attached.
> > > > 
> > > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > > 
> > > > Have you seen this before? I couldn't find a specific report about this
> > > > problem.
> > > > 
> > > > Let me know if you need more info.
> > > 
> > > hi,
> > > this looks like symptom of the gcc DWARF bug we were
> > > dealing with recently:
> > > 
> > >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> > >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > > 
> > > what pahole/gcc version are you using?
> > 
> > I'm on gcc 9.3.0
> > 
> > 	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > 
> > I was on pahole v1.17. I moved to v1.19 but I still see the same problem.
> 
> I can reproduce with your .config, but make 'defconfig' works,
> so I guess it's some config option issue, I'll check later today

so your .config has
  CONFIG_CRYPTO_DEV_BCM_SPU=y

and that defines 'struct device_private' which
clashes with the same struct defined in drivers/base/base.h

so several networking structs will be doubled, like net_device:

	$ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
	[2731] STRUCT 'net_device' size=2240 vlen=133
	[113981] STRUCT 'net_device' size=2240 vlen=133

each is using different 'struct device_private' when it's unwinded

and that will confuse BTFIDS logic, becase we have multiple structs
with the same name, and we can't be sure which one to pick

perhaps we should check on this in pahole and warn earlier with
better error message.. I'll check, but I'm not sure if pahole can
survive another hastab ;-)

Andrii, any ideas on this? ;-)

easy fix is the patch below that renames the bcm's structs,
it makes the kernel to compile.. but of course the new name
is probably wrong and we should push this through that code
authors

jirka


---
diff --git a/drivers/crypto/bcm/cipher.c b/drivers/crypto/bcm/cipher.c
index 30390a7324b2..0e5537838ef3 100644
--- a/drivers/crypto/bcm/cipher.c
+++ b/drivers/crypto/bcm/cipher.c
@@ -42,7 +42,7 @@
 
 /* ================= Device Structure ================== */
 
-struct device_private iproc_priv;
+struct bcm_device_private iproc_priv;
 
 /* ==================== Parameters ===================== */
 
diff --git a/drivers/crypto/bcm/cipher.h b/drivers/crypto/bcm/cipher.h
index 0ad5892b445d..71281a3bdbdc 100644
--- a/drivers/crypto/bcm/cipher.h
+++ b/drivers/crypto/bcm/cipher.h
@@ -420,7 +420,7 @@ struct spu_hw {
 	u32 num_chan;
 };
 
-struct device_private {
+struct bcm_device_private {
 	struct platform_device *pdev;
 
 	struct spu_hw spu;
@@ -467,6 +467,6 @@ struct device_private {
 	struct mbox_chan **mbox;
 };
 
-extern struct device_private iproc_priv;
+extern struct bcm_device_private iproc_priv;
 
 #endif
diff --git a/drivers/crypto/bcm/util.c b/drivers/crypto/bcm/util.c
index 2b304fc78059..77aeedb84055 100644
--- a/drivers/crypto/bcm/util.c
+++ b/drivers/crypto/bcm/util.c
@@ -348,7 +348,7 @@ char *spu_alg_name(enum spu_cipher_alg alg, enum spu_cipher_mode mode)
 static ssize_t spu_debugfs_read(struct file *filp, char __user *ubuf,
 				size_t count, loff_t *offp)
 {
-	struct device_private *ipriv;
+	struct bcm_device_private *ipriv;
 	char *buf;
 	ssize_t ret, out_offset, out_count;
 	int i;

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Jiri Olsa]

On Wed, Dec 30, 2020 at 02:28:02PM +0100, Jiri Olsa wrote:
> On Wed, Dec 30, 2020 at 10:03:37AM +0100, Jiri Olsa wrote:
> > On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> > > Hi Jiri
> > > 
> > > On 12/29/20 18:34, Jiri Olsa wrote:
> > > > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > > > Hi
> > > > > 
> > > > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > > > stage
> > > > > 
> > > > > 	FAILED unresolved symbol udp6_sock
> > > > > 
> > > > > I cross compile for arm64. My .config is attached.
> > > > > 
> > > > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > > > 
> > > > > Have you seen this before? I couldn't find a specific report about this
> > > > > problem.
> > > > > 
> > > > > Let me know if you need more info.
> > > > 
> > > > hi,
> > > > this looks like symptom of the gcc DWARF bug we were
> > > > dealing with recently:
> > > > 
> > > >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> > > >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > > > 
> > > > what pahole/gcc version are you using?
> > > 
> > > I'm on gcc 9.3.0
> > > 
> > > 	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > > 
> > > I was on pahole v1.17. I moved to v1.19 but I still see the same problem.
> > 
> > I can reproduce with your .config, but make 'defconfig' works,
> > so I guess it's some config option issue, I'll check later today
> 
> so your .config has
>   CONFIG_CRYPTO_DEV_BCM_SPU=y
> 
> and that defines 'struct device_private' which
> clashes with the same struct defined in drivers/base/base.h
> 
> so several networking structs will be doubled, like net_device:
> 
> 	$ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
> 	[2731] STRUCT 'net_device' size=2240 vlen=133
> 	[113981] STRUCT 'net_device' size=2240 vlen=133
> 
> each is using different 'struct device_private' when it's unwinded
> 
> and that will confuse BTFIDS logic, becase we have multiple structs
> with the same name, and we can't be sure which one to pick
> 
> perhaps we should check on this in pahole and warn earlier with
> better error message.. I'll check, but I'm not sure if pahole can
> survive another hastab ;-)
> 
> Andrii, any ideas on this? ;-)
> 
> easy fix is the patch below that renames the bcm's structs,
> it makes the kernel to compile.. but of course the new name
> is probably wrong and we should push this through that code
> authors

also another quick fix is to switch it to module

jirka

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Arnaldo Carvalho de Melo]

Em Wed, Dec 30, 2020 at 02:28:52PM +0100, Jiri Olsa escreveu:
> On Wed, Dec 30, 2020 at 02:28:02PM +0100, Jiri Olsa wrote:
> > On Wed, Dec 30, 2020 at 10:03:37AM +0100, Jiri Olsa wrote:
> > > On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> > > > Hi Jiri
> > > > 
> > > > On 12/29/20 18:34, Jiri Olsa wrote:
> > > > > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > > > > Hi
> > > > > > 
> > > > > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > > > > stage
> > > > > > 
> > > > > > 	FAILED unresolved symbol udp6_sock
> > > > > > 
> > > > > > I cross compile for arm64. My .config is attached.
> > > > > > 
> > > > > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > > > > 
> > > > > > Have you seen this before? I couldn't find a specific report about this
> > > > > > problem.
> > > > > > 
> > > > > > Let me know if you need more info.
> > > > > 
> > > > > hi,
> > > > > this looks like symptom of the gcc DWARF bug we were
> > > > > dealing with recently:
> > > > > 
> > > > >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> > > > >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > > > > 
> > > > > what pahole/gcc version are you using?
> > > > 
> > > > I'm on gcc 9.3.0
> > > > 
> > > > 	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > > > 
> > > > I was on pahole v1.17. I moved to v1.19 but I still see the same problem.

There are some changes post v1.19 in the git repo:

[acme@five pahole]$ git log --oneline v1.19..
b688e35970600c15 (HEAD -> master) btf_encoder: fix skipping per-CPU variables at offset 0
8c009d6ce762dfc9 btf_encoder: fix BTF variable generation for kernel modules
b94e97e015a94e6b dwarves: Fix compilation on 32-bit architectures
17df51c700248f02 btf_encoder: Detect kernel module ftrace addresses
06ca639505fc56c6 btf_encoder: Use address size based on ELF's class
aff60970d16b909e btf_encoder: Factor filter_functions function
1e6a3fed6e52d365 (quaco/master) rpm: Fix changelog date
[acme@five pahole]$

But I think these won't matter in this case :-\

- Arnaldo

> > > I can reproduce with your .config, but make 'defconfig' works,
> > > so I guess it's some config option issue, I'll check later today
> > 
> > so your .config has
> >   CONFIG_CRYPTO_DEV_BCM_SPU=y
> > 
> > and that defines 'struct device_private' which
> > clashes with the same struct defined in drivers/base/base.h
> > 
> > so several networking structs will be doubled, like net_device:
> > 
> > 	$ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
> > 	[2731] STRUCT 'net_device' size=2240 vlen=133
> > 	[113981] STRUCT 'net_device' size=2240 vlen=133
> > 
> > each is using different 'struct device_private' when it's unwinded
> > 
> > and that will confuse BTFIDS logic, becase we have multiple structs
> > with the same name, and we can't be sure which one to pick
> > 
> > perhaps we should check on this in pahole and warn earlier with
> > better error message.. I'll check, but I'm not sure if pahole can
> > survive another hastab ;-)
> > 
> > Andrii, any ideas on this? ;-)
> > 
> > easy fix is the patch below that renames the bcm's structs,
> > it makes the kernel to compile.. but of course the new name
> > is probably wrong and we should push this through that code
> > authors
> 
> also another quick fix is to switch it to module
> 
> jirka
> 

-- 

- Arnaldo

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Jiri Olsa]

On Wed, Dec 30, 2020 at 11:19:36AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Dec 30, 2020 at 02:28:52PM +0100, Jiri Olsa escreveu:
> > On Wed, Dec 30, 2020 at 02:28:02PM +0100, Jiri Olsa wrote:
> > > On Wed, Dec 30, 2020 at 10:03:37AM +0100, Jiri Olsa wrote:
> > > > On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> > > > > Hi Jiri
> > > > > 
> > > > > On 12/29/20 18:34, Jiri Olsa wrote:
> > > > > > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > > > > > Hi
> > > > > > > 
> > > > > > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > > > > > stage
> > > > > > > 
> > > > > > > 	FAILED unresolved symbol udp6_sock
> > > > > > > 
> > > > > > > I cross compile for arm64. My .config is attached.
> > > > > > > 
> > > > > > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > > > > > 
> > > > > > > Have you seen this before? I couldn't find a specific report about this
> > > > > > > problem.
> > > > > > > 
> > > > > > > Let me know if you need more info.
> > > > > > 
> > > > > > hi,
> > > > > > this looks like symptom of the gcc DWARF bug we were
> > > > > > dealing with recently:
> > > > > > 
> > > > > >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> > > > > >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > > > > > 
> > > > > > what pahole/gcc version are you using?
> > > > > 
> > > > > I'm on gcc 9.3.0
> > > > > 
> > > > > 	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > > > > 
> > > > > I was on pahole v1.17. I moved to v1.19 but I still see the same problem.
> 
> There are some changes post v1.19 in the git repo:
> 
> [acme@five pahole]$ git log --oneline v1.19..
> b688e35970600c15 (HEAD -> master) btf_encoder: fix skipping per-CPU variables at offset 0
> 8c009d6ce762dfc9 btf_encoder: fix BTF variable generation for kernel modules
> b94e97e015a94e6b dwarves: Fix compilation on 32-bit architectures
> 17df51c700248f02 btf_encoder: Detect kernel module ftrace addresses
> 06ca639505fc56c6 btf_encoder: Use address size based on ELF's class
> aff60970d16b909e btf_encoder: Factor filter_functions function
> 1e6a3fed6e52d365 (quaco/master) rpm: Fix changelog date
> [acme@five pahole]$
> 
> But I think these won't matter in this case :-\

yep, it did not.. I used the latest dwarves code

jirka

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Qais Yousef]

On 12/30/20 14:28, Jiri Olsa wrote:
> On Wed, Dec 30, 2020 at 02:28:02PM +0100, Jiri Olsa wrote:
> > On Wed, Dec 30, 2020 at 10:03:37AM +0100, Jiri Olsa wrote:
> > > On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> > > > Hi Jiri
> > > > 
> > > > On 12/29/20 18:34, Jiri Olsa wrote:
> > > > > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > > > > Hi
> > > > > > 
> > > > > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > > > > stage
> > > > > > 
> > > > > > 	FAILED unresolved symbol udp6_sock
> > > > > > 
> > > > > > I cross compile for arm64. My .config is attached.
> > > > > > 
> > > > > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > > > > 
> > > > > > Have you seen this before? I couldn't find a specific report about this
> > > > > > problem.
> > > > > > 
> > > > > > Let me know if you need more info.
> > > > > 
> > > > > hi,
> > > > > this looks like symptom of the gcc DWARF bug we were
> > > > > dealing with recently:
> > > > > 
> > > > >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> > > > >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > > > > 
> > > > > what pahole/gcc version are you using?
> > > > 
> > > > I'm on gcc 9.3.0
> > > > 
> > > > 	aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > > > 
> > > > I was on pahole v1.17. I moved to v1.19 but I still see the same problem.
> > > 
> > > I can reproduce with your .config, but make 'defconfig' works,
> > > so I guess it's some config option issue, I'll check later today

My .config was a defconfig + CONFIG_DEBUG_INFO_BTF + convert all modules to
built-in.

> > 
> > so your .config has
> >   CONFIG_CRYPTO_DEV_BCM_SPU=y

Ah, how random. I removed this config and indeed it does work then, thanks!

> > 
> > and that defines 'struct device_private' which
> > clashes with the same struct defined in drivers/base/base.h
> > 
> > so several networking structs will be doubled, like net_device:
> > 
> > 	$ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
> > 	[2731] STRUCT 'net_device' size=2240 vlen=133
> > 	[113981] STRUCT 'net_device' size=2240 vlen=133
> > 
> > each is using different 'struct device_private' when it's unwinded
> > 
> > and that will confuse BTFIDS logic, becase we have multiple structs
> > with the same name, and we can't be sure which one to pick

We can't tell which object/subsystem the struct come from?

Or maybe we can introduce some annotation to help BTFIDS to pick the right one?

> > 
> > perhaps we should check on this in pahole and warn earlier with
> > better error message.. I'll check, but I'm not sure if pahole can
> > survive another hastab ;-)
> > 
> > Andrii, any ideas on this? ;-)
> > 
> > easy fix is the patch below that renames the bcm's structs,
> > it makes the kernel to compile.. but of course the new name
> > is probably wrong and we should push this through that code
> > authors
> 
> also another quick fix is to switch it to module

This works too.

Thanks for your speedy response.

Cheers

--
Qais Yousef

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Andrii Nakryiko]

On Wed, Dec 30, 2020 at 5:28 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Wed, Dec 30, 2020 at 10:03:37AM +0100, Jiri Olsa wrote:
> > On Tue, Dec 29, 2020 at 11:28:35PM +0000, Qais Yousef wrote:
> > > Hi Jiri
> > >
> > > On 12/29/20 18:34, Jiri Olsa wrote:
> > > > On Tue, Dec 29, 2020 at 03:13:52PM +0000, Qais Yousef wrote:
> > > > > Hi
> > > > >
> > > > > When I enable CONFIG_DEBUG_INFO_BTF I get the following error in the BTFIDS
> > > > > stage
> > > > >
> > > > >         FAILED unresolved symbol udp6_sock
> > > > >
> > > > > I cross compile for arm64. My .config is attached.
> > > > >
> > > > > I managed to reproduce the problem on v5.9 and v5.10. Plus 5.11-rc1.
> > > > >
> > > > > Have you seen this before? I couldn't find a specific report about this
> > > > > problem.
> > > > >
> > > > > Let me know if you need more info.
> > > >
> > > > hi,
> > > > this looks like symptom of the gcc DWARF bug we were
> > > > dealing with recently:
> > > >
> > > >   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060
> > > >   https://lore.kernel.org/lkml/CAE1WUT75gu9G62Q9uAALGN6vLX=o7vZ9uhqtVWnbUV81DgmFPw@mail.gmail.com/#r
> > > >
> > > > what pahole/gcc version are you using?
> > >
> > > I'm on gcc 9.3.0
> > >
> > >     aarch64-linux-gnu-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> > >
> > > I was on pahole v1.17. I moved to v1.19 but I still see the same problem.
> >
> > I can reproduce with your .config, but make 'defconfig' works,
> > so I guess it's some config option issue, I'll check later today
>
> so your .config has
>   CONFIG_CRYPTO_DEV_BCM_SPU=y
>
> and that defines 'struct device_private' which
> clashes with the same struct defined in drivers/base/base.h
>
> so several networking structs will be doubled, like net_device:
>
>         $ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
>         [2731] STRUCT 'net_device' size=2240 vlen=133
>         [113981] STRUCT 'net_device' size=2240 vlen=133
>
> each is using different 'struct device_private' when it's unwinded
>
> and that will confuse BTFIDS logic, becase we have multiple structs
> with the same name, and we can't be sure which one to pick
>
> perhaps we should check on this in pahole and warn earlier with
> better error message.. I'll check, but I'm not sure if pahole can
> survive another hastab ;-)
>
> Andrii, any ideas on this? ;-)

It's both unavoidable and correct from the C type system's
perspective, so there is nothing for pahole to warn about. We used to
have (for a long time) a similar clash with two completely different
ring_buffer structs. Eventually they just got renamed to avoid
duplication of related structs (task_struct and tons of other). But
both BTF dedup and CO-RE relocation algorithms are designed to handle
this correctly, so perhaps BTFIDS should be able to handle this as
well?

>
> easy fix is the patch below that renames the bcm's structs,
> it makes the kernel to compile.. but of course the new name
> is probably wrong and we should push this through that code
> authors

In this case, I think renaming generic device_private name is a good
thing regardless.

>
> jirka
>
>
> ---

[...]

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Jiri Olsa]

On Sat, Jan 02, 2021 at 02:25:34PM -0800, Andrii Nakryiko wrote:

SNIP

> >
> > so your .config has
> >   CONFIG_CRYPTO_DEV_BCM_SPU=y
> >
> > and that defines 'struct device_private' which
> > clashes with the same struct defined in drivers/base/base.h
> >
> > so several networking structs will be doubled, like net_device:
> >
> >         $ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
> >         [2731] STRUCT 'net_device' size=2240 vlen=133
> >         [113981] STRUCT 'net_device' size=2240 vlen=133
> >
> > each is using different 'struct device_private' when it's unwinded
> >
> > and that will confuse BTFIDS logic, becase we have multiple structs
> > with the same name, and we can't be sure which one to pick
> >
> > perhaps we should check on this in pahole and warn earlier with
> > better error message.. I'll check, but I'm not sure if pahole can
> > survive another hastab ;-)
> >
> > Andrii, any ideas on this? ;-)
> 
> It's both unavoidable and correct from the C type system's
> perspective, so there is nothing for pahole to warn about. We used to
> have (for a long time) a similar clash with two completely different
> ring_buffer structs. Eventually they just got renamed to avoid
> duplication of related structs (task_struct and tons of other). But
> both BTF dedup and CO-RE relocation algorithms are designed to handle
> this correctly, ...

AFAIU it's all correctly dedulicated, but still all structs that
contain (at some point) 'struct device_private' will appear twice
in BTF data.. each with different 'struct device_private'

> ... so perhaps BTFIDS should be able to handle this as
> well?

hm, BTFIDS sees BTF data with two same struct names and has no
way to tell which one to use

unless we have some annotation data for BTF types I don't
see a way to handle this correctly.. but I think we can
detect this directly in BTFIDS and print more accurate error
message

as long as we dont see this on daily basis, I think that better
error message + following struct rename is good solution

> 
> >
> > easy fix is the patch below that renames the bcm's structs,
> > it makes the kernel to compile.. but of course the new name
> > is probably wrong and we should push this through that code
> > authors
> 
> In this case, I think renaming generic device_private name is a good
> thing regardless.

ok, I'll send the change

jirka

Re: BTFIDS: FAILED unresolved symbol udp6_sock

[Andrii Nakryiko]

On Sat, Jan 2, 2021 at 3:07 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Sat, Jan 02, 2021 at 02:25:34PM -0800, Andrii Nakryiko wrote:
>
> SNIP
>
> > >
> > > so your .config has
> > >   CONFIG_CRYPTO_DEV_BCM_SPU=y
> > >
> > > and that defines 'struct device_private' which
> > > clashes with the same struct defined in drivers/base/base.h
> > >
> > > so several networking structs will be doubled, like net_device:
> > >
> > >         $ bpftool btf dump file ../vmlinux.config | grep net_device\' | grep STRUCT
> > >         [2731] STRUCT 'net_device' size=2240 vlen=133
> > >         [113981] STRUCT 'net_device' size=2240 vlen=133
> > >
> > > each is using different 'struct device_private' when it's unwinded
> > >
> > > and that will confuse BTFIDS logic, becase we have multiple structs
> > > with the same name, and we can't be sure which one to pick
> > >
> > > perhaps we should check on this in pahole and warn earlier with
> > > better error message.. I'll check, but I'm not sure if pahole can
> > > survive another hastab ;-)
> > >
> > > Andrii, any ideas on this? ;-)
> >
> > It's both unavoidable and correct from the C type system's
> > perspective, so there is nothing for pahole to warn about. We used to
> > have (for a long time) a similar clash with two completely different
> > ring_buffer structs. Eventually they just got renamed to avoid
> > duplication of related structs (task_struct and tons of other). But
> > both BTF dedup and CO-RE relocation algorithms are designed to handle
> > this correctly, ...
>
> AFAIU it's all correctly dedulicated, but still all structs that
> contain (at some point) 'struct device_private' will appear twice
> in BTF data.. each with different 'struct device_private'

it's correct from the type system perspective, right. Those two
duplicates of struct device_private are parts of two different
hierarchies of types. However inconvenient it is, C allows it,
unfortunately :(

>
> > ... so perhaps BTFIDS should be able to handle this as
> > well?
>
> hm, BTFIDS sees BTF data with two same struct names and has no
> way to tell which one to use
>
> unless we have some annotation data for BTF types I don't
> see a way to handle this correctly.. but I think we can
> detect this directly in BTFIDS and print more accurate error
> message
>
> as long as we dont see this on daily basis, I think that better
> error message + following struct rename is good solution

Perhaps warning and handling this gracefully is a bit better way to
handle this. Renaming is definitely good, but shouldn't block the
kernel build process. I don't remember the exact details for why
duplicate struct would cause troubles for resolve_btfids, but maybe
just picking the struct with minimal ID (out of 2+ duplicates) would
be ok in practice most of the time. In any case, that's what most
users (including libbpf) will do, when searching for the type by name.

>
> >
> > >
> > > easy fix is the patch below that renames the bcm's structs,
> > > it makes the kernel to compile.. but of course the new name
> > > is probably wrong and we should push this through that code
> > > authors
> >
> > In this case, I think renaming generic device_private name is a good
> > thing regardless.
>
> ok, I'll send the change
>

great, thanks

> jirka
>