* [net 2/3] ixgbe: fix use after free adapter->state test in ixgbe_remove/ixgbe_probe
2014-11-22 7:52 [net 1/3] ixgbe: Correctly disable VLAN filter in promiscuous mode Jeff Kirsher
@ 2014-11-22 7:52 ` Jeff Kirsher
2014-11-23 19:27 ` David Miller
2014-11-22 7:52 ` [net 3/3] igb: Fixes needed for surprise removal support Jeff Kirsher
2014-11-23 19:26 ` [net 1/3] ixgbe: Correctly disable VLAN filter in promiscuous mode David Miller
2 siblings, 1 reply; 6+ messages in thread
From: Jeff Kirsher @ 2014-11-22 7:52 UTC (permalink / raw)
To: davem
Cc: Daniel Borkmann, netdev, nhorman, sassmann, jogreene, stable,
Mark Rustad, Jeff Kirsher
From: Daniel Borkmann <dborkman@redhat.com>
While working on a different issue, I noticed an annoying use
after free bug on my machine when unloading the ixgbe driver:
[ 8642.318797] ixgbe 0000:02:00.1: removed PHC on p2p2
[ 8642.742716] ixgbe 0000:02:00.1: complete
[ 8642.743784] BUG: unable to handle kernel paging request at ffff8807d3740a90
[ 8642.744828] IP: [<ffffffffa01c77dc>] ixgbe_remove+0xfc/0x1b0 [ixgbe]
[ 8642.745886] PGD 20c6067 PUD 81c1f6067 PMD 81c15a067 PTE 80000007d3740060
[ 8642.746956] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[ 8642.748039] Modules linked in: [...]
[ 8642.752929] CPU: 1 PID: 1225 Comm: rmmod Not tainted 3.18.0-rc2+ #49
[ 8642.754203] Hardware name: Supermicro X10SLM-F/X10SLM-F, BIOS 1.1b 11/01/2013
[ 8642.755505] task: ffff8807e34d3fe0 ti: ffff8807b7204000 task.ti: ffff8807b7204000
[ 8642.756831] RIP: 0010:[<ffffffffa01c77dc>] [<ffffffffa01c77dc>] ixgbe_remove+0xfc/0x1b0 [ixgbe]
[...]
[ 8642.774335] Stack:
[ 8642.775805] ffff8807ee824098 ffff8807ee824098 ffffffffa01f3000 ffff8807ee824000
[ 8642.777326] ffff8807b7207e18 ffffffff8137720f ffff8807ee824098 ffff8807ee824098
[ 8642.778848] ffffffffa01f3068 ffff8807ee8240f8 ffff8807b7207e38 ffffffff8144180f
[ 8642.780365] Call Trace:
[ 8642.781869] [<ffffffff8137720f>] pci_device_remove+0x3f/0xc0
[ 8642.783395] [<ffffffff8144180f>] __device_release_driver+0x7f/0xf0
[ 8642.784876] [<ffffffff814421f8>] driver_detach+0xb8/0xc0
[ 8642.786352] [<ffffffff814414a9>] bus_remove_driver+0x59/0xe0
[ 8642.787783] [<ffffffff814429d0>] driver_unregister+0x30/0x70
[ 8642.789202] [<ffffffff81375c65>] pci_unregister_driver+0x25/0xa0
[ 8642.790657] [<ffffffffa01eb38e>] ixgbe_exit_module+0x1c/0xc8e [ixgbe]
[ 8642.792064] [<ffffffff810f93a2>] SyS_delete_module+0x132/0x1c0
[ 8642.793450] [<ffffffff81012c61>] ? do_notify_resume+0x61/0xa0
[ 8642.794837] [<ffffffff816d2029>] system_call_fastpath+0x12/0x17
The issue is that test_and_set_bit() done on adapter->state is being
performed *after* the netdevice has been freed via free_netdev().
When netdev is being allocated on initialization time, it allocates
a private area, here struct ixgbe_adapter, that resides after the
net_device structure. In ixgbe_probe(), the device init routine,
we set up the adapter after alloc_etherdev_mq() on the private area
and add a reference for the pci_dev as well via pci_set_drvdata().
Both in the error path of ixgbe_probe(), but also on module unload
when ixgbe_remove() is being called, commit 41c62843eb6a ("ixgbe:
Fix rcu warnings induced by LER") accesses adapter after free_netdev().
The patch stores the result in a bool and thus fixes above oops on my
side.
Fixes: 41c62843eb6a ("ixgbe: Fix rcu warnings induced by LER")
Cc: stable <stable@vger.kernel.org>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 7acde46..82ffe8b 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7979,6 +7979,7 @@ static int ixgbe_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
int i, err, pci_using_dac, expected_gts;
unsigned int indices = MAX_TX_QUEUES;
u8 part_str[IXGBE_PBANUM_LENGTH];
+ bool disable_dev = false;
#ifdef IXGBE_FCOE
u16 device_caps;
#endif
@@ -8369,13 +8370,14 @@ err_sw_init:
iounmap(adapter->io_addr);
kfree(adapter->mac_table);
err_ioremap:
+ disable_dev = !test_and_set_bit(__IXGBE_DISABLED, &adapter->state);
free_netdev(netdev);
err_alloc_etherdev:
pci_release_selected_regions(pdev,
pci_select_bars(pdev, IORESOURCE_MEM));
err_pci_reg:
err_dma:
- if (!adapter || !test_and_set_bit(__IXGBE_DISABLED, &adapter->state))
+ if (!adapter || disable_dev)
pci_disable_device(pdev);
return err;
}
@@ -8393,6 +8395,7 @@ static void ixgbe_remove(struct pci_dev *pdev)
{
struct ixgbe_adapter *adapter = pci_get_drvdata(pdev);
struct net_device *netdev = adapter->netdev;
+ bool disable_dev;
ixgbe_dbg_adapter_exit(adapter);
@@ -8442,11 +8445,12 @@ static void ixgbe_remove(struct pci_dev *pdev)
e_dev_info("complete\n");
kfree(adapter->mac_table);
+ disable_dev = !test_and_set_bit(__IXGBE_DISABLED, &adapter->state);
free_netdev(netdev);
pci_disable_pcie_error_reporting(pdev);
- if (!test_and_set_bit(__IXGBE_DISABLED, &adapter->state))
+ if (disable_dev)
pci_disable_device(pdev);
}
--
1.9.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [net 2/3] ixgbe: fix use after free adapter->state test in ixgbe_remove/ixgbe_probe
2014-11-22 7:52 ` [net 2/3] ixgbe: fix use after free adapter->state test in ixgbe_remove/ixgbe_probe Jeff Kirsher
@ 2014-11-23 19:27 ` David Miller
0 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2014-11-23 19:27 UTC (permalink / raw)
To: jeffrey.t.kirsher
Cc: dborkman, netdev, nhorman, sassmann, jogreene, stable,
mark.d.rustad
From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Date: Fri, 21 Nov 2014 23:52:53 -0800
> From: Daniel Borkmann <dborkman@redhat.com>
>
> While working on a different issue, I noticed an annoying use
> after free bug on my machine when unloading the ixgbe driver:
...
> The issue is that test_and_set_bit() done on adapter->state is being
> performed *after* the netdevice has been freed via free_netdev().
>
> When netdev is being allocated on initialization time, it allocates
> a private area, here struct ixgbe_adapter, that resides after the
> net_device structure. In ixgbe_probe(), the device init routine,
> we set up the adapter after alloc_etherdev_mq() on the private area
> and add a reference for the pci_dev as well via pci_set_drvdata().
>
> Both in the error path of ixgbe_probe(), but also on module unload
> when ixgbe_remove() is being called, commit 41c62843eb6a ("ixgbe:
> Fix rcu warnings induced by LER") accesses adapter after free_netdev().
> The patch stores the result in a bool and thus fixes above oops on my
> side.
>
> Fixes: 41c62843eb6a ("ixgbe: Fix rcu warnings induced by LER")
> Cc: stable <stable@vger.kernel.org>
> Cc: Mark Rustad <mark.d.rustad@intel.com>
> Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Applied.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [net 3/3] igb: Fixes needed for surprise removal support
2014-11-22 7:52 [net 1/3] ixgbe: Correctly disable VLAN filter in promiscuous mode Jeff Kirsher
2014-11-22 7:52 ` [net 2/3] ixgbe: fix use after free adapter->state test in ixgbe_remove/ixgbe_probe Jeff Kirsher
@ 2014-11-22 7:52 ` Jeff Kirsher
2014-11-23 19:27 ` David Miller
2014-11-23 19:26 ` [net 1/3] ixgbe: Correctly disable VLAN filter in promiscuous mode David Miller
2 siblings, 1 reply; 6+ messages in thread
From: Jeff Kirsher @ 2014-11-22 7:52 UTC (permalink / raw)
To: davem
Cc: Carolyn Wyborny, netdev, nhorman, sassmann, jogreene,
Yanir Lubetkin, Jeff Kirsher
From: Carolyn Wyborny <carolyn.wyborny@intel.com>
This patch adds some checks in order to prevent panic's on surprise
removal of devices during S0, S3, S4. Without this patch, Thunderbolt
type device removal will panic the system.
Signed-off-by: Yanir Lubetkin <yanirx.lubetkin@intel.com>
Signed-off-by: Carolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: Aaron Brown <aaron.f.brown@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index a2d72a8..487cd9c 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1012,7 +1012,8 @@ static void igb_free_q_vector(struct igb_adapter *adapter, int v_idx)
/* igb_get_stats64() might access the rings on this vector,
* we must wait a grace period before freeing it.
*/
- kfree_rcu(q_vector, rcu);
+ if (q_vector)
+ kfree_rcu(q_vector, rcu);
}
/**
@@ -1792,8 +1793,10 @@ void igb_down(struct igb_adapter *adapter)
adapter->flags &= ~IGB_FLAG_NEED_LINK_UPDATE;
for (i = 0; i < adapter->num_q_vectors; i++) {
- napi_synchronize(&(adapter->q_vector[i]->napi));
- napi_disable(&(adapter->q_vector[i]->napi));
+ if (adapter->q_vector[i]) {
+ napi_synchronize(&adapter->q_vector[i]->napi);
+ napi_disable(&adapter->q_vector[i]->napi);
+ }
}
@@ -3717,7 +3720,8 @@ static void igb_free_all_tx_resources(struct igb_adapter *adapter)
int i;
for (i = 0; i < adapter->num_tx_queues; i++)
- igb_free_tx_resources(adapter->tx_ring[i]);
+ if (adapter->tx_ring[i])
+ igb_free_tx_resources(adapter->tx_ring[i]);
}
void igb_unmap_and_free_tx_resource(struct igb_ring *ring,
@@ -3782,7 +3786,8 @@ static void igb_clean_all_tx_rings(struct igb_adapter *adapter)
int i;
for (i = 0; i < adapter->num_tx_queues; i++)
- igb_clean_tx_ring(adapter->tx_ring[i]);
+ if (adapter->tx_ring[i])
+ igb_clean_tx_ring(adapter->tx_ring[i]);
}
/**
@@ -3819,7 +3824,8 @@ static void igb_free_all_rx_resources(struct igb_adapter *adapter)
int i;
for (i = 0; i < adapter->num_rx_queues; i++)
- igb_free_rx_resources(adapter->rx_ring[i]);
+ if (adapter->rx_ring[i])
+ igb_free_rx_resources(adapter->rx_ring[i]);
}
/**
@@ -3874,7 +3880,8 @@ static void igb_clean_all_rx_rings(struct igb_adapter *adapter)
int i;
for (i = 0; i < adapter->num_rx_queues; i++)
- igb_clean_rx_ring(adapter->rx_ring[i]);
+ if (adapter->rx_ring[i])
+ igb_clean_rx_ring(adapter->rx_ring[i]);
}
/**
@@ -7404,6 +7411,8 @@ static int igb_resume(struct device *dev)
pci_restore_state(pdev);
pci_save_state(pdev);
+ if (!pci_device_is_present(pdev))
+ return -ENODEV;
err = pci_enable_device_mem(pdev);
if (err) {
dev_err(&pdev->dev,
--
1.9.3
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [net 3/3] igb: Fixes needed for surprise removal support
2014-11-22 7:52 ` [net 3/3] igb: Fixes needed for surprise removal support Jeff Kirsher
@ 2014-11-23 19:27 ` David Miller
0 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2014-11-23 19:27 UTC (permalink / raw)
To: jeffrey.t.kirsher
Cc: carolyn.wyborny, netdev, nhorman, sassmann, jogreene,
yanirx.lubetkin
From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Date: Fri, 21 Nov 2014 23:52:54 -0800
> From: Carolyn Wyborny <carolyn.wyborny@intel.com>
>
> This patch adds some checks in order to prevent panic's on surprise
> removal of devices during S0, S3, S4. Without this patch, Thunderbolt
> type device removal will panic the system.
>
> Signed-off-by: Yanir Lubetkin <yanirx.lubetkin@intel.com>
> Signed-off-by: Carolyn Wyborny <carolyn.wyborny@intel.com>
> Tested-by: Aaron Brown <aaron.f.brown@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Applied.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [net 1/3] ixgbe: Correctly disable VLAN filter in promiscuous mode
2014-11-22 7:52 [net 1/3] ixgbe: Correctly disable VLAN filter in promiscuous mode Jeff Kirsher
2014-11-22 7:52 ` [net 2/3] ixgbe: fix use after free adapter->state test in ixgbe_remove/ixgbe_probe Jeff Kirsher
2014-11-22 7:52 ` [net 3/3] igb: Fixes needed for surprise removal support Jeff Kirsher
@ 2014-11-23 19:26 ` David Miller
2 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2014-11-23 19:26 UTC (permalink / raw)
To: jeffrey.t.kirsher
Cc: vyasevich, netdev, nhorman, sassmann, jogreene, stable,
jacob.e.keller, vyasevic
From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Date: Fri, 21 Nov 2014 23:52:52 -0800
> From: Vlad Yasevich <vyasevich@gmail.com>
>
> IXGBE adapter seems to require that VLAN filtering be enabled if
> VMDQ or SRIOV are enabled. When those functions are disabled,
> VLAN filtering may be disabled in promiscuous mode.
>
> Prior to commit a9b8943ee129 ("ixgbe: remove vlan_filter_disable
> and enable functions")
>
> The logic was correct. However, after the commit the logic
> got reversed and VLAN filtered in now turned on when VMDQ/SRIOV
> is disabled.
>
> This patch changes the condition to enable hw vlan filtered
> when VMDQ or SRIOV is enabled.
>
> Fixes: a9b8943ee129 ("ixgbe: remove vlan_filter_disable and enable functions")
> Cc: stable <stable@vger.kernel.org>
> CC: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Vladislav Yasevich <vyasevic@redhat.com>
> Acked-by: Emil Tantilov <emil.s.tantilov@intel.com>
> Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Applied.
^ permalink raw reply [flat|nested] 6+ messages in thread