[PATCH net 1/3] enic: unlock napi busy poll before unmasking intr

[PATCH net 1/3] enic: unlock napi busy poll before unmasking intr

[Govindarajulu Varadarajan]

There is a small window between vnic_intr_unmask() and enic_poll_unlock_napi().
In this window if an irq occurs and napi is scheduled on different cpu, it tries
to acquire enic_poll_lock_napi() and hits the following WARN_ON message.

Fix is to unlock napi_poll before unmasking the interrupt.

[  781.121746] ------------[ cut here ]------------
[  781.121789] WARNING: CPU: 1 PID: 0 at drivers/net/ethernet/cisco/enic/vnic_rq.h:228 enic_poll_msix_rq+0x36a/0x3c0 [enic]()
[  781.121834] Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss oid_registry nfsv4 dns_resolver coretemp intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel mgag200 ttm drm_kms_helper joydev aes_x86_64 lrw drm gf128mul mousedev glue_helper sb_edac ablk_helper iTCO_wdt iTCO_vendor_support evdev ipmi_si syscopyarea sysfillrect sysimgblt i2c_algo_bit i2c_core edac_core lpc_ich mac_hid cryptd pcspkr ipmi_msghandler shpchp tpm_tis acpi_power_meter tpm wmi processor hwmon button ac sch_fq_codel nfs lockd grace sunrpc fscache hid_generic usbhid hid ehci_pci ehci_hcd sd_mod megaraid_sas usbcore scsi_mod usb_common enic crc32c_generic crc32c_intel btrfs xor raid6_pq ext4 crc16 mbcache jbd2
[  781.122176] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.0-rc6-ARCH-00040-gc46a024-dirty #106
[  781.122210] Hardware name: Cisco Systems Inc UCSB-B200-M4/UCSB-B200-M4, BIOS B200M4.2.2.2.23.061220140128 06/12/2014
[  781.122252]  0000000000000000 bddbbc9d655ec96e ffff880277e43da8 ffffffff81583fe8
[  781.122286]  0000000000000000 0000000000000000 ffff880277e43de8 ffffffff8107acfa
[  781.122319]  ffff880272c01000 ffff880273f18000 ffff880273f1a100 0000000000000000
[  781.122352] Call Trace:
[  781.122364]  <IRQ>  [<ffffffff81583fe8>] dump_stack+0x4f/0x7b
[  781.122399]  [<ffffffff8107acfa>] warn_slowpath_common+0x8a/0xc0
[  781.122425]  [<ffffffff8107ae2a>] warn_slowpath_null+0x1a/0x20
[  781.122455]  [<ffffffffa01fa9ca>] enic_poll_msix_rq+0x36a/0x3c0 [enic]
[  781.122487]  [<ffffffff8148525a>] net_rx_action+0x22a/0x370
[  781.122512]  [<ffffffff8107ed3d>] __do_softirq+0xed/0x2d0
[  781.122537]  [<ffffffff8107f06e>] irq_exit+0x7e/0xa0
[  781.122560]  [<ffffffff8158c424>] do_IRQ+0x64/0x100
[  781.122582]  [<ffffffff8158a42e>] common_interrupt+0x6e/0x6e
[  781.122605]  <EOI>  [<ffffffff810bd331>] ? cpu_startup_entry+0x121/0x480
[  781.122638]  [<ffffffff810bd2fc>] ? cpu_startup_entry+0xec/0x480
[  781.122667]  [<ffffffff810f2ed3>] ? clockevents_register_device+0x113/0x1f0
[  781.122698]  [<ffffffff81050ab6>] start_secondary+0x196/0x1e0
[  781.122723] ---[ end trace cec2e9dd3af7b9db ]---

Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
---
 drivers/net/ethernet/cisco/enic/enic_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c
index 204bd182..0e5a01d 100644
--- a/drivers/net/ethernet/cisco/enic/enic_main.c
+++ b/drivers/net/ethernet/cisco/enic/enic_main.c
@@ -1407,6 +1407,7 @@ static int enic_poll_msix_rq(struct napi_struct *napi, int budget)
 		 */
 		enic_calc_int_moderation(enic, &enic->rq[rq]);
 
+	enic_poll_unlock_napi(&enic->rq[rq]);
 	if (work_done < work_to_do) {
 
 		/* Some work done, but not enough to stay in polling,
@@ -1418,7 +1419,6 @@ static int enic_poll_msix_rq(struct napi_struct *napi, int budget)
 			enic_set_int_moderation(enic, &enic->rq[rq]);
 		vnic_intr_unmask(&enic->intr[intr]);
 	}
-	enic_poll_unlock_napi(&enic->rq[rq]);
 
 	return work_done;
 }
-- 
2.4.2

[PATCH net 2/3] enic: check return value for stat dump

[Govindarajulu Varadarajan]

We do not check the return value of enic_dev_stats_dump(). If allocation
fails, we will hit NULL pointer reference.

Return only if memory allocation fails. For other failures, we return the
previously recorded values.

Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
---
 drivers/net/ethernet/cisco/enic/enic_ethtool.c | 20 +++++++++++++++++---
 drivers/net/ethernet/cisco/enic/enic_main.c    |  9 ++++++++-
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/cisco/enic/enic_ethtool.c b/drivers/net/ethernet/cisco/enic/enic_ethtool.c
index 28d9ca6..68d47b1 100644
--- a/drivers/net/ethernet/cisco/enic/enic_ethtool.c
+++ b/drivers/net/ethernet/cisco/enic/enic_ethtool.c
@@ -131,8 +131,15 @@ static void enic_get_drvinfo(struct net_device *netdev,
 {
 	struct enic *enic = netdev_priv(netdev);
 	struct vnic_devcmd_fw_info *fw_info;
+	int err;
 
-	enic_dev_fw_info(enic, &fw_info);
+	err = enic_dev_fw_info(enic, &fw_info);
+	/* return only when pci_zalloc_consistent fails in vnic_dev_fw_info
+	 * For other failures, like devcmd failure, we return previously
+	 * recorded info.
+	 */
+	if (err == -ENOMEM)
+		return;
 
 	strlcpy(drvinfo->driver, DRV_NAME, sizeof(drvinfo->driver));
 	strlcpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version));
@@ -181,8 +188,15 @@ static void enic_get_ethtool_stats(struct net_device *netdev,
 	struct enic *enic = netdev_priv(netdev);
 	struct vnic_stats *vstats;
 	unsigned int i;
-
-	enic_dev_stats_dump(enic, &vstats);
+	int err;
+
+	err = enic_dev_stats_dump(enic, &vstats);
+	/* return only when pci_zalloc_consistent fails in vnic_dev_stats_dump
+	 * For other failures, like devcmd failure, we return previously
+	 * recorded stats.
+	 */
+	if (err == -ENOMEM)
+		return;
 
 	for (i = 0; i < enic_n_tx_stats; i++)
 		*(data++) = ((u64 *)&vstats->tx)[enic_tx_stats[i].index];
diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c
index 0e5a01d..eadae1b 100644
--- a/drivers/net/ethernet/cisco/enic/enic_main.c
+++ b/drivers/net/ethernet/cisco/enic/enic_main.c
@@ -615,8 +615,15 @@ static struct rtnl_link_stats64 *enic_get_stats(struct net_device *netdev,
 {
 	struct enic *enic = netdev_priv(netdev);
 	struct vnic_stats *stats;
+	int err;
 
-	enic_dev_stats_dump(enic, &stats);
+	err = enic_dev_stats_dump(enic, &stats);
+	/* return only when pci_zalloc_consistent fails in vnic_dev_stats_dump
+	 * For other failures, like devcmd failure, we return previously
+	 * recorded stats.
+	 */
+	if (err == -ENOMEM)
+		return net_stats;
 
 	net_stats->tx_packets = stats->tx.tx_frames_ok;
 	net_stats->tx_bytes = stats->tx.tx_bytes_ok;
-- 
2.4.2

[PATCH net 3/3] enic: fix memory leak in rq_clean

[Govindarajulu Varadarajan]

When incoming packet qualifies for rx_copybreak, we copy the data to newly
allocated skb. We do not free/unmap the original buffer. At this point driver
assumes this buffer is unallocated. When enic_rq_alloc_buf() is called for
buffer allocation, it checks if buf->os_buf is NULL. If its not NULL that means
buffer can be re-used.

When vnic_rq_clean() is called for freeing all rq buffers, and if the
rx_copybreak reused buffer falls outside the used desc, we do not free the
buffer. The following trace is observer when dma-debug is enabled.

Fix is to walk through complete ring and clean if buffer is present.

[   40.555386] ------------[ cut here ]------------
[   40.555396] WARNING: CPU: 0 PID: 491 at lib/dma-debug.c:971 dma_debug_device_change+0x188/0x1f0()
[   40.555400] pci 0000:06:00.0: DMA-API: device driver has pending DMA allocations while released from device [count=4]
               One of leaked entries details: [device address=0x00000000ff4cc040] [size=9018 bytes] [mapped with DMA_FROM_DEVICE] [mapped as single]
[   40.555402] Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss oid_registry nfsv4 dns_resolver coretemp intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw joydev mousedev gf128mul hid_generic glue_helper mgag200 usbhid ttm hid drm_kms_helper drm ablk_helper syscopyarea sysfillrect sysimgblt i2c_algo_bit i2c_core iTCO_wdt cryptd mac_hid evdev pcspkr sb_edac edac_core tpm_tis iTCO_vendor_support ipmi_si wmi tpm ipmi_msghandler shpchp lpc_ich processor acpi_power_meter hwmon button ac sch_fq_codel nfs lockd grace sunrpc fscache sd_mod ehci_pci ehci_hcd megaraid_sas usbcore scsi_mod usb_common enic(-) crc32c_generic crc32c_intel btrfs xor raid6_pq ext4 crc16 mbcache jbd2
[   40.555467] CPU: 0 PID: 491 Comm: rmmod Not tainted 4.1.0-rc7-ARCH-01305-gf59b71f #118
[   40.555469] Hardware name: Cisco Systems Inc UCSB-B200-M4/UCSB-B200-M4, BIOS B200M4.2.2.2.23.061220140128 06/12/2014
[   40.555471]  0000000000000000 00000000e2f8a5b7 ffff880275f8bc48 ffffffff8158d6f0
[   40.555474]  0000000000000000 ffff880275f8bca0 ffff880275f8bc88 ffffffff8107b04a
[   40.555477]  ffff8802734e0000 0000000000000004 ffff8804763fb3c0 ffff88027600b650
[   40.555480] Call Trace:
[   40.555488]  [<ffffffff8158d6f0>] dump_stack+0x4f/0x7b
[   40.555492]  [<ffffffff8107b04a>] warn_slowpath_common+0x8a/0xc0
[   40.555494]  [<ffffffff8107b0d5>] warn_slowpath_fmt+0x55/0x70
[   40.555498]  [<ffffffff812fa408>] dma_debug_device_change+0x188/0x1f0
[   40.555503]  [<ffffffff8109aaef>] notifier_call_chain+0x4f/0x80
[   40.555506]  [<ffffffff8109aecb>] __blocking_notifier_call_chain+0x4b/0x70
[   40.555510]  [<ffffffff8109af06>] blocking_notifier_call_chain+0x16/0x20
[   40.555514]  [<ffffffff813f8066>] __device_release_driver+0xf6/0x120
[   40.555518]  [<ffffffff813f8b08>] driver_detach+0xc8/0xd0
[   40.555523]  [<ffffffff813f7c59>] bus_remove_driver+0x59/0xe0
[   40.555527]  [<ffffffff813f93a0>] driver_unregister+0x30/0x70
[   40.555534]  [<ffffffff8131532d>] pci_unregister_driver+0x2d/0xa0
[   40.555542]  [<ffffffffa0200ec2>] enic_cleanup_module+0x10/0x14e [enic]
[   40.555547]  [<ffffffff8110158f>] SyS_delete_module+0x1cf/0x280
[   40.555551]  [<ffffffff811e284e>] ? ____fput+0xe/0x10
[   40.555554]  [<ffffffff810980ec>] ? task_work_run+0xbc/0xf0
[   40.555558]  [<ffffffff815930ee>] system_call_fastpath+0x12/0x71
[   40.555561] ---[ end trace 4988cadc77c2b236 ]---
[   40.555562] Mapped at:
[   40.555563]  [<ffffffff812fa865>] debug_dma_map_page+0x95/0x150
[   40.555566]  [<ffffffffa01f4a88>] enic_rq_alloc_buf+0x1b8/0x360 [enic]
[   40.555570]  [<ffffffffa01f7658>] enic_open+0xf8/0x820 [enic]
[   40.555574]  [<ffffffff8148d50e>] __dev_open+0xce/0x150
[   40.555579]  [<ffffffff8148d851>] __dev_change_flags+0xa1/0x170

Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
---
 drivers/net/ethernet/cisco/enic/vnic_rq.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/cisco/enic/vnic_rq.c b/drivers/net/ethernet/cisco/enic/vnic_rq.c
index 36a2ed6..c4b2183 100644
--- a/drivers/net/ethernet/cisco/enic/vnic_rq.c
+++ b/drivers/net/ethernet/cisco/enic/vnic_rq.c
@@ -188,16 +188,15 @@ void vnic_rq_clean(struct vnic_rq *rq,
 	struct vnic_rq_buf *buf;
 	u32 fetch_index;
 	unsigned int count = rq->ring.desc_count;
+	int i;
 
 	buf = rq->to_clean;
 
-	while (vnic_rq_desc_used(rq) > 0) {
-
+	for (i = 0; i < rq->ring.desc_count; i++) {
 		(*buf_clean)(rq, buf);
-
-		buf = rq->to_clean = buf->next;
-		rq->ring.desc_avail++;
+		buf = buf->next;
 	}
+	rq->ring.desc_avail = rq->ring.desc_count - 1;
 
 	/* Use current fetch_index as the ring starting point */
 	fetch_index = ioread32(&rq->ctrl->fetch_index);
-- 
2.4.2

Re: [PATCH net 1/3] enic: unlock napi busy poll before unmasking intr

[David Miller]

From: Govindarajulu Varadarajan <_govind@gmx.com>
Date: Thu, 11 Jun 2015 11:52:54 +0530

> There is a small window between vnic_intr_unmask() and enic_poll_unlock_napi().
> In this window if an irq occurs and napi is scheduled on different cpu, it tries
> to acquire enic_poll_lock_napi() and hits the following WARN_ON message.
> 
> Fix is to unlock napi_poll before unmasking the interrupt.
 ...
> Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>

Applied.

Re: [PATCH net 2/3] enic: check return value for stat dump

[David Miller]

From: Govindarajulu Varadarajan <_govind@gmx.com>
Date: Thu, 11 Jun 2015 11:52:55 +0530

> We do not check the return value of enic_dev_stats_dump(). If allocation
> fails, we will hit NULL pointer reference.
> 
> Return only if memory allocation fails. For other failures, we return the
> previously recorded values.
> 
> Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>

Applied.

Re: [PATCH net 3/3] enic: fix memory leak in rq_clean

[David Miller]

From: Govindarajulu Varadarajan <_govind@gmx.com>
Date: Thu, 11 Jun 2015 11:52:56 +0530

> When incoming packet qualifies for rx_copybreak, we copy the data to newly
> allocated skb. We do not free/unmap the original buffer. At this point driver
> assumes this buffer is unallocated. When enic_rq_alloc_buf() is called for
> buffer allocation, it checks if buf->os_buf is NULL. If its not NULL that means
> buffer can be re-used.
> 
> When vnic_rq_clean() is called for freeing all rq buffers, and if the
> rx_copybreak reused buffer falls outside the used desc, we do not free the
> buffer. The following trace is observer when dma-debug is enabled.
> 
> Fix is to walk through complete ring and clean if buffer is present.
 ...
> Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>

Applied.