netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [BUG] kernel oops during bridge creation
@ 2017-04-06 18:19 Peter V. Saveliev
  2017-04-06 18:58 ` Ido Schimmel
  0 siblings, 1 reply; 5+ messages in thread
From: Peter V. Saveliev @ 2017-04-06 18:19 UTC (permalink / raw)
  To: netdev

Operation:

# ip link add name test type bridge vlan_default_pvid 1

Result:

Kernel oops. Minimal required netlink packet structure:

ifinfmsg:
	- IFLA_IFNAME: "test"
	- IFLA_LINKINFO:
		= IFLA_INFO_KIND: "bridge"
		= IFLA_INFO_DATA:
			- IFLA_BR_VLAN_DEFAULT_PVID: 1

Reproducible: always

Versions affected: all the latest kernels, including net-next

Possibly related to 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b6677449dff674cf5b81429b11d5c7f358852ef9

Trace:

[13634.939408] BUG: unable to handle kernel NULL pointer dereference at 
0000000000000190
[13634.939436] IP: __vlan_add+0x73/0x5f0
[13634.939445] PGD 3dfe0067
[13634.939445] PUD 35807067
[13634.939451] PMD 0

[13634.939467] Oops: 0002 [#1] SMP
[13634.939474] Modules linked in: crct10dif_pclmul crc32_pclmul 
ghash_clmulni_intel pcbc qxl ttm drm_kms_helper drm aesni_intel 
aes_x86_64 crypto_simd glue_helper cryptd virtio_balloon evdev 
input_leds pcspkr led_class serio_raw virtio_console i2c_piix4 
acpi_cpufreq tpm_tis tpm_tis_core tpm button ext4 crc16 jbd2 mbcache 
virtio_net virtio_blk crc32c_intel psmouse 8139cp mii virtio_pci 
virtio_ring virtio
[13634.939561] CPU: 0 PID: 1535 Comm: ip Not tainted 4.11.0-rc3+ #6
[13634.939574] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), 
BIOS 1.9.3-1.fc25 04/01/2014
[13634.939593] task: ffff9b4838f5a000 task.stack: ffffc1b74046c000
[13634.939607] RIP: 0010:__vlan_add+0x73/0x5f0
[13634.939617] RSP: 0018:ffffc1b74046f668 EFLAGS: 00010246
[13634.939629] RAX: 0000000000000000 RBX: ffff9b483522d680 RCX: 
0000000000000000
[13634.939644] RDX: ffffffffbe8f2978 RSI: 0000000000000200 RDI: 
ffffffffbe2c120f
[13634.939659] RBP: 0000000000000000 R08: 0000000000000000 R09: 
0000000000000000
[13634.939674] R10: 0000000000000000 R11: 000000000000c609 R12: 
0000000000000000
[13634.939689] R13: ffff9b48350ae8c0 R14: ffff9b48350ae000 R15: 
0000000000000000
[13634.939705] FS:  00007fe5e4f13700(0000) GS:ffff9b483fc00000(0000) 
knlGS:0000000000000000
[13634.939722] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13634.939734] CR2: 0000000000000190 CR3: 0000000035805000 CR4: 
00000000000406f0
[13634.939752] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[13634.939767] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
[13634.939783] Call Trace:
[13634.939791]  ? pcpu_next_unpop+0x3b/0x50
[13634.939801]  ? pcpu_alloc+0x3d2/0x680
[13634.939810]  ? br_vlan_add+0x135/0x1b0
[13634.939820]  ? __br_vlan_set_default_pvid.part.28+0x204/0x2b0
[13634.939834]  ? br_changelink+0x120/0x4e0
[13634.939844]  ? br_dev_newlink+0x50/0x70
[13634.939854]  ? rtnl_newlink+0x5f5/0x8a0
[13634.939864]  ? rtnl_newlink+0x176/0x8a0
[13634.939874]  ? mem_cgroup_commit_charge+0x7c/0x4e0
[13634.939886]  ? rtnetlink_rcv_msg+0xe1/0x220
[13634.939896]  ? lookup_fast+0x52/0x370
[13634.939905]  ? rtnl_newlink+0x8a0/0x8a0
[13634.939915]  ? netlink_rcv_skb+0xa1/0xc0
[13634.939925]  ? rtnetlink_rcv+0x24/0x30
[13634.939934]  ? netlink_unicast+0x177/0x220
[13634.939944]  ? netlink_sendmsg+0x2fe/0x3b0
[13634.939954]  ? _copy_from_user+0x39/0x40
[13634.939964]  ? sock_sendmsg+0x30/0x40
[13634.940159]  ? ___sys_sendmsg+0x29d/0x2b0
[13634.940326]  ? __alloc_pages_nodemask+0xdf/0x230
[13634.940478]  ? mem_cgroup_commit_charge+0x7c/0x4e0
[13634.940592]  ? mem_cgroup_try_charge+0x76/0x1a0
[13634.940701]  ? __handle_mm_fault+0xdb9/0x10b0
[13634.940809]  ? __sys_sendmsg+0x51/0x90
[13634.940917]  ? entry_SYSCALL_64_fastpath+0x1e/0xad
[13634.941027] Code: 75 18 49 8b 6d 30 a8 20 74 29 0f b7 4b 10 49 8b 96 
28 03 00 00 4c 89 e6 4c 89 ef e8 88 eb fe ff 85 c0 41 89 c7 0f 85 58 05 
00 00 <66> 83 85 90 01 00 00 01 48 8b 45 30 48 89 c2 48 f7 da 48 83 7d
[13634.941301] RIP: __vlan_add+0x73/0x5f0 RSP: ffffc1b74046f668
[13634.941423] CR2: 0000000000000190
[13634.943641] ---[ end trace bb793e957e3e263c ]---

…

Thanks.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [BUG] kernel oops during bridge creation
  2017-04-06 18:19 [BUG] kernel oops during bridge creation Peter V. Saveliev
@ 2017-04-06 18:58 ` Ido Schimmel
  2017-04-06 20:26   ` Nikolay Aleksandrov
  0 siblings, 1 reply; 5+ messages in thread
From: Ido Schimmel @ 2017-04-06 18:58 UTC (permalink / raw)
  To: Peter V. Saveliev, nikolay; +Cc: netdev

+Nik

On Thu, Apr 06, 2017 at 08:19:35PM +0200, Peter V. Saveliev wrote:
> Operation:
> 
> # ip link add name test type bridge vlan_default_pvid 1
> 
> Result:
> 
> Kernel oops. Minimal required netlink packet structure:
> 
> ifinfmsg:
> 	- IFLA_IFNAME: "test"
> 	- IFLA_LINKINFO:
> 		= IFLA_INFO_KIND: "bridge"
> 		= IFLA_INFO_DATA:
> 			- IFLA_BR_VLAN_DEFAULT_PVID: 1
> 
> Reproducible: always
> 
> Versions affected: all the latest kernels, including net-next
> 
> Possibly related to https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b6677449dff674cf5b81429b11d5c7f358852ef9

I believe this is correct.

> 
> Trace:
> 
> [13634.939408] BUG: unable to handle kernel NULL pointer dereference at
> 0000000000000190
> [13634.939436] IP: __vlan_add+0x73/0x5f0
> [13634.939445] PGD 3dfe0067
> [13634.939445] PUD 35807067
> [13634.939451] PMD 0
> 
> [13634.939467] Oops: 0002 [#1] SMP
> [13634.939474] Modules linked in: crct10dif_pclmul crc32_pclmul
> ghash_clmulni_intel pcbc qxl ttm drm_kms_helper drm aesni_intel aes_x86_64
> crypto_simd glue_helper cryptd virtio_balloon evdev input_leds pcspkr
> led_class serio_raw virtio_console i2c_piix4 acpi_cpufreq tpm_tis
> tpm_tis_core tpm button ext4 crc16 jbd2 mbcache virtio_net virtio_blk
> crc32c_intel psmouse 8139cp mii virtio_pci virtio_ring virtio
> [13634.939561] CPU: 0 PID: 1535 Comm: ip Not tainted 4.11.0-rc3+ #6
> [13634.939574] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> 1.9.3-1.fc25 04/01/2014
> [13634.939593] task: ffff9b4838f5a000 task.stack: ffffc1b74046c000
> [13634.939607] RIP: 0010:__vlan_add+0x73/0x5f0
> [13634.939617] RSP: 0018:ffffc1b74046f668 EFLAGS: 00010246
> [13634.939629] RAX: 0000000000000000 RBX: ffff9b483522d680 RCX:
> 0000000000000000
> [13634.939644] RDX: ffffffffbe8f2978 RSI: 0000000000000200 RDI:
> ffffffffbe2c120f
> [13634.939659] RBP: 0000000000000000 R08: 0000000000000000 R09:
> 0000000000000000
> [13634.939674] R10: 0000000000000000 R11: 000000000000c609 R12:
> 0000000000000000
> [13634.939689] R13: ffff9b48350ae8c0 R14: ffff9b48350ae000 R15:
> 0000000000000000
> [13634.939705] FS:  00007fe5e4f13700(0000) GS:ffff9b483fc00000(0000)
> knlGS:0000000000000000
> [13634.939722] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [13634.939734] CR2: 0000000000000190 CR3: 0000000035805000 CR4:
> 00000000000406f0
> [13634.939752] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [13634.939767] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
> 0000000000000400
> [13634.939783] Call Trace:
> [13634.939791]  ? pcpu_next_unpop+0x3b/0x50
> [13634.939801]  ? pcpu_alloc+0x3d2/0x680
> [13634.939810]  ? br_vlan_add+0x135/0x1b0

Nik, I looked at this earlier today since Peter mentioned this over IRC.
I think the problem is that the bridge's vlan group is accessed before
it's allocated in br_vlan_init().

Not sure what's the correct way to solve this though, maybe moving
br_vlan_init() to the bridge's newlink()? Do you see any problem with
that?

> [13634.939820]  ? __br_vlan_set_default_pvid.part.28+0x204/0x2b0
> [13634.939834]  ? br_changelink+0x120/0x4e0
> [13634.939844]  ? br_dev_newlink+0x50/0x70
> [13634.939854]  ? rtnl_newlink+0x5f5/0x8a0
> [13634.939864]  ? rtnl_newlink+0x176/0x8a0
> [13634.939874]  ? mem_cgroup_commit_charge+0x7c/0x4e0
> [13634.939886]  ? rtnetlink_rcv_msg+0xe1/0x220
> [13634.939896]  ? lookup_fast+0x52/0x370
> [13634.939905]  ? rtnl_newlink+0x8a0/0x8a0
> [13634.939915]  ? netlink_rcv_skb+0xa1/0xc0
> [13634.939925]  ? rtnetlink_rcv+0x24/0x30
> [13634.939934]  ? netlink_unicast+0x177/0x220
> [13634.939944]  ? netlink_sendmsg+0x2fe/0x3b0
> [13634.939954]  ? _copy_from_user+0x39/0x40
> [13634.939964]  ? sock_sendmsg+0x30/0x40
> [13634.940159]  ? ___sys_sendmsg+0x29d/0x2b0
> [13634.940326]  ? __alloc_pages_nodemask+0xdf/0x230
> [13634.940478]  ? mem_cgroup_commit_charge+0x7c/0x4e0
> [13634.940592]  ? mem_cgroup_try_charge+0x76/0x1a0
> [13634.940701]  ? __handle_mm_fault+0xdb9/0x10b0
> [13634.940809]  ? __sys_sendmsg+0x51/0x90
> [13634.940917]  ? entry_SYSCALL_64_fastpath+0x1e/0xad
> [13634.941027] Code: 75 18 49 8b 6d 30 a8 20 74 29 0f b7 4b 10 49 8b 96 28
> 03 00 00 4c 89 e6 4c 89 ef e8 88 eb fe ff 85 c0 41 89 c7 0f 85 58 05 00 00
> <66> 83 85 90 01 00 00 01 48 8b 45 30 48 89 c2 48 f7 da 48 83 7d
> [13634.941301] RIP: __vlan_add+0x73/0x5f0 RSP: ffffc1b74046f668
> [13634.941423] CR2: 0000000000000190
> [13634.943641] ---[ end trace bb793e957e3e263c ]---
> 
> …
> 
> Thanks.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [BUG] kernel oops during bridge creation
  2017-04-06 18:58 ` Ido Schimmel
@ 2017-04-06 20:26   ` Nikolay Aleksandrov
  2017-04-06 20:44     ` Ido Schimmel
  0 siblings, 1 reply; 5+ messages in thread
From: Nikolay Aleksandrov @ 2017-04-06 20:26 UTC (permalink / raw)
  To: Ido Schimmel, Peter V. Saveliev; +Cc: netdev, Ivan Vecera

On 06/04/17 21:58, Ido Schimmel wrote:
> +Nik
> 

Thanks!

> On Thu, Apr 06, 2017 at 08:19:35PM +0200, Peter V. Saveliev wrote:
>> Operation:
>>
>> # ip link add name test type bridge vlan_default_pvid 1
>>
>> Result:
>>
>> Kernel oops. Minimal required netlink packet structure:
>>
>> ifinfmsg:
>> 	- IFLA_IFNAME: "test"
>> 	- IFLA_LINKINFO:
>> 		= IFLA_INFO_KIND: "bridge"
>> 		= IFLA_INFO_DATA:
>> 			- IFLA_BR_VLAN_DEFAULT_PVID: 1
>>
>> Reproducible: always
>>
>> Versions affected: all the latest kernels, including net-next
>>
>> Possibly related to https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b6677449dff674cf5b81429b11d5c7f358852ef9
> 
> I believe this is correct.
> 

Yep.
+CC Ivan

>>
>> Trace:
>>
>> [13634.939408] BUG: unable to handle kernel NULL pointer dereference at
>> 0000000000000190
>> [13634.939436] IP: __vlan_add+0x73/0x5f0
>> [13634.939445] PGD 3dfe0067
>> [13634.939445] PUD 35807067
>> [13634.939451] PMD 0
>>
>> [13634.939467] Oops: 0002 [#1] SMP
>> [13634.939474] Modules linked in: crct10dif_pclmul crc32_pclmul
>> ghash_clmulni_intel pcbc qxl ttm drm_kms_helper drm aesni_intel aes_x86_64
>> crypto_simd glue_helper cryptd virtio_balloon evdev input_leds pcspkr
>> led_class serio_raw virtio_console i2c_piix4 acpi_cpufreq tpm_tis
>> tpm_tis_core tpm button ext4 crc16 jbd2 mbcache virtio_net virtio_blk
>> crc32c_intel psmouse 8139cp mii virtio_pci virtio_ring virtio
>> [13634.939561] CPU: 0 PID: 1535 Comm: ip Not tainted 4.11.0-rc3+ #6
>> [13634.939574] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
>> 1.9.3-1.fc25 04/01/2014
>> [13634.939593] task: ffff9b4838f5a000 task.stack: ffffc1b74046c000
>> [13634.939607] RIP: 0010:__vlan_add+0x73/0x5f0
>> [13634.939617] RSP: 0018:ffffc1b74046f668 EFLAGS: 00010246
>> [13634.939629] RAX: 0000000000000000 RBX: ffff9b483522d680 RCX:
>> 0000000000000000
>> [13634.939644] RDX: ffffffffbe8f2978 RSI: 0000000000000200 RDI:
>> ffffffffbe2c120f
>> [13634.939659] RBP: 0000000000000000 R08: 0000000000000000 R09:
>> 0000000000000000
>> [13634.939674] R10: 0000000000000000 R11: 000000000000c609 R12:
>> 0000000000000000
>> [13634.939689] R13: ffff9b48350ae8c0 R14: ffff9b48350ae000 R15:
>> 0000000000000000
>> [13634.939705] FS:  00007fe5e4f13700(0000) GS:ffff9b483fc00000(0000)
>> knlGS:0000000000000000
>> [13634.939722] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [13634.939734] CR2: 0000000000000190 CR3: 0000000035805000 CR4:
>> 00000000000406f0
>> [13634.939752] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [13634.939767] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
>> 0000000000000400
>> [13634.939783] Call Trace:
>> [13634.939791]  ? pcpu_next_unpop+0x3b/0x50
>> [13634.939801]  ? pcpu_alloc+0x3d2/0x680
>> [13634.939810]  ? br_vlan_add+0x135/0x1b0
> 
> Nik, I looked at this earlier today since Peter mentioned this over IRC.
> I think the problem is that the bridge's vlan group is accessed before
> it's allocated in br_vlan_init().
> 

Yes, that's the problem.

> Not sure what's the correct way to solve this though, maybe moving
> br_vlan_init() to the bridge's newlink()? Do you see any problem with
> that?
> 

I think we also have to support the old way of adding bridges via the ioctl, and moving
the init to newlink() would break that, making br_vlan_init() idempotent wouldn't help
either as we need to destroy the vg if it's allocated upon failure. So we'll have to
consider all error paths in both newlink() and the old ioctl and handle the vlan if it
was created before, ugh..

Right now I don't see a better option that would keep the current state though, so if you'd
like go ahead and cook a patch, I won't be able to do it until tomorrow my time.

Actually making br_vlan_init() idempotent might work, keep the code as-is just init the
the vlans before the changelink() in newlink(), then the second vlan_init() inside would
be a no-op, but it will work for the old ioctl and wouldn't require too much change.
On the downside it will be ugly.

Cheers,
 Nik

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [BUG] kernel oops during bridge creation
  2017-04-06 20:26   ` Nikolay Aleksandrov
@ 2017-04-06 20:44     ` Ido Schimmel
  2017-04-07  8:36       ` Nikolay Aleksandrov
  0 siblings, 1 reply; 5+ messages in thread
From: Ido Schimmel @ 2017-04-06 20:44 UTC (permalink / raw)
  To: Nikolay Aleksandrov; +Cc: Peter V. Saveliev, netdev, Ivan Vecera

On Thu, Apr 06, 2017 at 11:26:06PM +0300, Nikolay Aleksandrov wrote:
> Actually making br_vlan_init() idempotent might work, keep the code as-is just init the
> the vlans before the changelink() in newlink(), then the second vlan_init() inside would
> be a no-op, but it will work for the old ioctl and wouldn't require too much change.
> On the downside it will be ugly.

Yep, that can work. I'll try that tomorrow morning and let you know.

Thanks Nik!

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [BUG] kernel oops during bridge creation
  2017-04-06 20:44     ` Ido Schimmel
@ 2017-04-07  8:36       ` Nikolay Aleksandrov
  0 siblings, 0 replies; 5+ messages in thread
From: Nikolay Aleksandrov @ 2017-04-07  8:36 UTC (permalink / raw)
  To: Ido Schimmel; +Cc: Peter V. Saveliev, netdev, Ivan Vecera

On 06/04/17 23:44, Ido Schimmel wrote:
> On Thu, Apr 06, 2017 at 11:26:06PM +0300, Nikolay Aleksandrov wrote:
>> Actually making br_vlan_init() idempotent might work, keep the code as-is just init the
>> the vlans before the changelink() in newlink(), then the second vlan_init() inside would
>> be a no-op, but it will work for the old ioctl and wouldn't require too much change.
>> On the downside it will be ugly.
> 
> Yep, that can work. I'll try that tomorrow morning and let you know.
> 
> Thanks Nik!
> 

Great, thanks, just one more thing to keep in mind. The vlan add would try
to add the bridge mac address which in turn will generate a notification
with ifindex = 0 since it will get assigned during register_netdevice() later.

The only way to get correct behaviour seems to make the changelink() opportunistic
and move it after the netdevice_register(), though that may cause a device which
is half configured if something's wrong with the config, but the good thing with
the bridge is that everything can be set during runtime so no need to destroy/make
it again.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2017-04-07  8:36 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-04-06 18:19 [BUG] kernel oops during bridge creation Peter V. Saveliev
2017-04-06 18:58 ` Ido Schimmel
2017-04-06 20:26   ` Nikolay Aleksandrov
2017-04-06 20:44     ` Ido Schimmel
2017-04-07  8:36       ` Nikolay Aleksandrov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).