[net-next 00/11][pull request] Intel Wired LAN Driver Updates

[net-next 00/11][pull request] Intel Wired LAN Driver Updates

[Jeff Kirsher]

This series contains updates to ixgbe only.

My previous pull request had a mix of ixgbe and igb patches, and
while Akeem works on the changes requested on the igb patches, I
going ahead and re-submitting the originally submitted ixgbe patches
with several other ixgbe patches.

Alex provides a performance improvement with the enabling the use of
build_skb for instances where jumbo frames are disabled.  In addition,
Alex provides a fix where we were incorrectly checking the entire frag_off
field when we only wanted the fragment offset.  Lastly, he cleans up
the check for PAGE_SIZE, since the default configuration allocates 32K
for all buffers.

Emil provides a change to the calculation of eerd so that it is consistent
between the read and write functions by using | instead of +.

Jacob adds support for displaying PCIe Gen3 link speed, which was
previously missing from the ixgbe driver.  He also provides a patch
to clean up ixgbe_get_bus_info_generic to call some conversion
functions, which are used also in another patch provided by Jacob.
Jacob modifies the driver to enable certain devices (which have an
internal switch) to read from the physical slot rather than reading
data from the internal switch.  Lastly, Jacob adds a function which
enables the ixgbe driver to walk up the PCI bus for the device and
query the PCI config space for the bus width at each point.


Don provides a couple of fixes (which are more appropriate for net-next),
one of which resolves an issue where ixgbe was only turning on the laser
when the adapter was up which caused issues for those who wanted to
access the MNG firmware while the port was in a down state.  The other
fix is for WoL when currently linked at 1G.  Lastly Don bumps the driver
version keep the in-kernel driver up to date with the current functionality.

The following are changes since commit 6c6779856a294649dbb468ef46e893e80b0d72ad:
  Revert "netprio_cgroup: make local table static"
and are available in the git repository at:
  git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-next master

Alexander Duyck (3):
  ixgbe: Support using build_skb in the case that jumbo frames are
    disabled
  ixgbe: Mask off check of frag_off as we only want fragment offset
  ixgbe: Drop check for PAGE_SIZE from ixgbe_xmit_frame_ring

Don Skidmore (3):
  ixgbe: fix MNG FW support when adapter not up
  ixgbe: Fix 1G link WoL
  ixgbe: bump version number

Emil Tantilov (1):
  ixgbe: don't do arithmetic operations on bitmasks

Jacob Keller (4):
  ixgbe: Enable support for recognizing PCI-e Gen3 link speed
  ixgbe: create conversion functions from link_status to bus/speed
  ixgbe: enable devices with internal switch to read pci parent
  ixgbe: walk pci-e bus to find minimum width

 drivers/net/ethernet/intel/ixgbe/ixgbe.h        |   7 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c  |   1 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c  |  51 ++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.c |  63 +++---
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.h |   2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c   | 289 ++++++++++++++++++++----
 drivers/net/ethernet/intel/ixgbe/ixgbe_type.h   |  13 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c   |   1 +
 8 files changed, 343 insertions(+), 84 deletions(-)

-- 
1.7.11.7

[net-next 01/11] ixgbe: Support using build_skb in the case that jumbo frames are disabled

[Jeff Kirsher]

From: Alexander Duyck <alexander.h.duyck@intel.com>

This change makes it so that we can enable the use of build_skb for cases
where jumbo frames are disabled.  The advantage to this is that we do not have
to perform a memcpy to populate the header and as a result we see a
significant performance improvement.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h      |   7 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 163 ++++++++++++++++++++------
 2 files changed, 137 insertions(+), 33 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
index a8e10cf..e5bc1d7 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
@@ -207,6 +207,7 @@ enum ixgbe_ring_state_t {
 	__IXGBE_RX_RSC_ENABLED,
 	__IXGBE_RX_CSUM_UDP_ZERO_ERR,
 	__IXGBE_RX_FCOE,
+	__IXGBE_RX_BUILD_SKB_ENABLED,
 };
 
 #define check_for_tx_hang(ring) \
@@ -221,6 +222,12 @@ enum ixgbe_ring_state_t {
 	set_bit(__IXGBE_RX_RSC_ENABLED, &(ring)->state)
 #define clear_ring_rsc_enabled(ring) \
 	clear_bit(__IXGBE_RX_RSC_ENABLED, &(ring)->state)
+#define ring_uses_build_skb(ring) \
+	test_bit(__IXGBE_RX_BUILD_SKB_ENABLED, &(ring)->state)
+#define set_ring_build_skb_enabled(ring) \
+	set_bit(__IXGBE_RX_BUILD_SKB_ENABLED, &(ring)->state)
+#define clear_ring_build_skb_enabled(ring) \
+	clear_bit(__IXGBE_RX_BUILD_SKB_ENABLED, &(ring)->state)
 struct ixgbe_ring {
 	struct ixgbe_ring *next;	/* pointer to next ring in q_vector */
 	struct ixgbe_q_vector *q_vector; /* backpointer to host q_vector */
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 1339932..0c51a9f 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -1227,6 +1227,14 @@ static bool ixgbe_alloc_mapped_page(struct ixgbe_ring *rx_ring,
 	return true;
 }
 
+static inline unsigned int ixgbe_rx_offset(struct ixgbe_ring *rx_ring)
+{
+	if (ring_uses_build_skb(rx_ring))
+		return NET_SKB_PAD + NET_IP_ALIGN;
+	else
+		return 0;
+}
+
 /**
  * ixgbe_alloc_rx_buffers - Replace used receive buffers
  * @rx_ring: ring to place buffers on
@@ -1254,7 +1262,9 @@ void ixgbe_alloc_rx_buffers(struct ixgbe_ring *rx_ring, u16 cleaned_count)
 		 * Refresh the desc even if buffer_addrs didn't change
 		 * because each write-back erases this info.
 		 */
-		rx_desc->read.pkt_addr = cpu_to_le64(bi->dma + bi->page_offset);
+		rx_desc->read.pkt_addr = cpu_to_le64(bi->dma +
+						     bi->page_offset +
+						     ixgbe_rx_offset(rx_ring));
 
 		rx_desc++;
 		bi++;
@@ -1673,6 +1683,47 @@ static void ixgbe_reuse_rx_page(struct ixgbe_ring *rx_ring,
 					 DMA_FROM_DEVICE);
 }
 
+static bool ixgbe_can_reuse_rx_page(struct ixgbe_ring *rx_ring,
+				    struct ixgbe_rx_buffer *rx_buffer,
+				    struct page *page,
+				    unsigned int truesize)
+{
+#if (PAGE_SIZE >= 8192)
+	unsigned int last_offset = ixgbe_rx_pg_size(rx_ring) -
+				   ixgbe_rx_bufsz(rx_ring);
+
+#endif
+	/* avoid re-using remote pages */
+	if (unlikely(page_to_nid(page) != numa_node_id()))
+		return false;
+
+#if (PAGE_SIZE < 8192)
+	/* if we are only owner of page we can reuse it */
+	if (unlikely(page_count(page) != 1))
+		return false;
+
+	/* flip page offset to other buffer */
+	rx_buffer->page_offset ^= truesize;
+
+	/* since we are the only owner of the page and we need to
+	 * increment it, just set the value to 2 in order to avoid
+	 * an unnecessary locked operation
+	 */
+	atomic_set(&page->_count, 2);
+#else
+	/* move offset up to the next cache line */
+	rx_buffer->page_offset += truesize;
+
+	if (rx_buffer->page_offset > last_offset)
+		return false;
+
+	/* bump ref count on page before it is given to the stack */
+	get_page(page);
+#endif
+
+	return true;
+}
+
 /**
  * ixgbe_add_rx_frag - Add contents of Rx buffer to sk_buff
  * @rx_ring: rx descriptor ring to transact packets on
@@ -1699,8 +1750,6 @@ static bool ixgbe_add_rx_frag(struct ixgbe_ring *rx_ring,
 	unsigned int truesize = ixgbe_rx_bufsz(rx_ring);
 #else
 	unsigned int truesize = ALIGN(size, L1_CACHE_BYTES);
-	unsigned int last_offset = ixgbe_rx_pg_size(rx_ring) -
-				   ixgbe_rx_bufsz(rx_ring);
 #endif
 
 	if ((size <= IXGBE_RX_HDR_SIZE) && !skb_is_nonlinear(skb)) {
@@ -1720,36 +1769,82 @@ static bool ixgbe_add_rx_frag(struct ixgbe_ring *rx_ring,
 	skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
 			rx_buffer->page_offset, size, truesize);
 
-	/* avoid re-using remote pages */
-	if (unlikely(page_to_nid(page) != numa_node_id()))
-		return false;
+	return ixgbe_can_reuse_rx_page(rx_ring, rx_buffer, page, truesize);
+}
 
+static struct sk_buff *ixgbe_build_rx_buffer(struct ixgbe_ring *rx_ring,
+					     union ixgbe_adv_rx_desc *rx_desc)
+{
+	struct ixgbe_rx_buffer *rx_buffer;
+	struct sk_buff *skb;
+	struct page *page;
+	void *page_addr;
+	unsigned int size = le16_to_cpu(rx_desc->wb.upper.length);
 #if (PAGE_SIZE < 8192)
-	/* if we are only owner of page we can reuse it */
-	if (unlikely(page_count(page) != 1))
-		return false;
+	unsigned int truesize = ixgbe_rx_bufsz(rx_ring);
+#else
+	unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +
+				SKB_DATA_ALIGN(NET_SKB_PAD +
+					       NET_IP_ALIGN +
+					       size);
+#endif
 
-	/* flip page offset to other buffer */
-	rx_buffer->page_offset ^= truesize;
+	/* If we spanned a buffer we have a huge mess so test for it */
+	BUG_ON(unlikely(!ixgbe_test_staterr(rx_desc, IXGBE_RXD_STAT_EOP)));
 
-	/*
-	 * since we are the only owner of the page and we need to
-	 * increment it, just set the value to 2 in order to avoid
-	 * an unecessary locked operation
-	 */
-	atomic_set(&page->_count, 2);
-#else
-	/* move offset up to the next cache line */
-	rx_buffer->page_offset += truesize;
+	/* Guarantee this function can be used by verifying buffer sizes */
+	BUILD_BUG_ON(SKB_WITH_OVERHEAD(IXGBE_RXBUFFER_2K) < (NET_SKB_PAD +
+							     NET_IP_ALIGN +
+							     VLAN_HLEN +
+							     ETH_FRAME_LEN +
+							     ETH_FCS_LEN));
 
-	if (rx_buffer->page_offset > last_offset)
-		return false;
+	rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean];
+	page = rx_buffer->page;
+	prefetchw(page);
 
-	/* bump ref count on page before it is given to the stack */
-	get_page(page);
+	page_addr = page_address(page) + rx_buffer->page_offset;
+
+	/* prefetch first cache line of first page */
+	prefetch(page_addr + NET_SKB_PAD + NET_IP_ALIGN);
+#if L1_CACHE_BYTES < 128
+	prefetch(page_addr + L1_CACHE_BYTES + NET_SKB_PAD + NET_IP_ALIGN);
 #endif
 
-	return true;
+	/* build an skb to go around the page buffer */
+	skb = build_skb(page_addr, truesize);
+	if (unlikely(!skb)) {
+		rx_ring->rx_stats.alloc_rx_buff_failed++;
+		return NULL;
+	}
+
+	/* we are reusing so sync this buffer for CPU use */
+	dma_sync_single_range_for_cpu(rx_ring->dev,
+				      rx_buffer->dma,
+				      rx_buffer->page_offset,
+				      ixgbe_rx_bufsz(rx_ring),
+				      DMA_FROM_DEVICE);
+
+	/* update pointers within the skb to store the data */
+	skb_reserve(skb, NET_IP_ALIGN + NET_SKB_PAD);
+	__skb_put(skb, size);
+
+	if (ixgbe_can_reuse_rx_page(rx_ring, rx_buffer, page, truesize)) {
+		/* hand second half of page back to the ring */
+		ixgbe_reuse_rx_page(rx_ring, rx_buffer);
+	} else {
+		/* we are not reusing the buffer so unmap it */
+		dma_unmap_page(rx_ring->dev, rx_buffer->dma,
+			       ixgbe_rx_pg_size(rx_ring),
+			       DMA_FROM_DEVICE);
+	}
+
+	/* clear contents of buffer_info */
+	rx_buffer->skb = NULL;
+	rx_buffer->dma = 0;
+	rx_buffer->page = NULL;
+
+	return skb;
 }
 
 static struct sk_buff *ixgbe_fetch_rx_buffer(struct ixgbe_ring *rx_ring,
@@ -1883,7 +1978,10 @@ static bool ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
 		rmb();
 
 		/* retrieve a buffer from the ring */
-		skb = ixgbe_fetch_rx_buffer(rx_ring, rx_desc);
+		if (ring_uses_build_skb(rx_ring))
+			skb = ixgbe_build_rx_buffer(rx_ring, rx_desc);
+		else
+			skb = ixgbe_fetch_rx_buffer(rx_ring, rx_desc);
 
 		/* exit if we failed to retrieve a buffer */
 		if (!skb)
@@ -3312,7 +3410,6 @@ static void ixgbe_set_rx_buffer_len(struct ixgbe_adapter *adapter)
 		max_frame = IXGBE_FCOE_JUMBO_FRAME_SIZE;
 
 #endif /* IXGBE_FCOE */
-
 	/* adjust max frame to be at least the size of a standard frame */
 	if (max_frame < (ETH_FRAME_LEN + ETH_FCS_LEN))
 		max_frame = (ETH_FRAME_LEN + ETH_FCS_LEN);
@@ -3330,16 +3427,16 @@ static void ixgbe_set_rx_buffer_len(struct ixgbe_adapter *adapter)
 	hlreg0 |= IXGBE_HLREG0_JUMBOEN;
 	IXGBE_WRITE_REG(hw, IXGBE_HLREG0, hlreg0);
 
-	/*
-	 * Setup the HW Rx Head and Tail Descriptor Pointers and
-	 * the Base and Length of the Rx Descriptor Ring
-	 */
+	/* Setup the ring features based on max frame size */
 	for (i = 0; i < adapter->num_rx_queues; i++) {
 		rx_ring = adapter->rx_ring[i];
+		clear_ring_rsc_enabled(rx_ring);
+		clear_ring_build_skb_enabled(rx_ring);
+
 		if (adapter->flags2 & IXGBE_FLAG2_RSC_ENABLED)
 			set_ring_rsc_enabled(rx_ring);
-		else
-			clear_ring_rsc_enabled(rx_ring);
+		else if (max_frame <= (ETH_FRAME_LEN + ETH_FCS_LEN))
+			set_ring_build_skb_enabled(rx_ring);
 	}
 }
 
-- 
1.7.11.7

[net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Jeff Kirsher]

From: Alexander Duyck <alexander.h.duyck@intel.com>

We were incorrectly checking the entire frag_off field when we only wanted the
fragment offset.  As a result we were not pulling in TCP headers when the DNF
flag was set.

To correct that we will now check for frag off using the IP_OFFSET mask.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 0c51a9f..096a221 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -1347,7 +1347,7 @@ static unsigned int ixgbe_get_headlen(unsigned char *data,
 			return hdr.network - data;
 
 		/* record next protocol if header is present */
-		if (!hdr.ipv4->frag_off)
+		if (!(hdr.ipv4->frag_off & htons(IP_OFFSET)))
 			nexthdr = hdr.ipv4->protocol;
 	} else if (protocol == __constant_htons(ETH_P_IPV6)) {
 		if ((hdr.network - data) > (max_len - sizeof(struct ipv6hdr)))
-- 
1.7.11.7

[net-next 03/11] ixgbe: don't do arithmetic operations on bitmasks

[Jeff Kirsher]

From: Emil Tantilov <emil.s.tantilov@intel.com>

Make the calculation of eerd consistent between the read and write functions
by using | instead of + for IXGBE_EEPROM_RW_REG_START

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Emil Tantilov <emil.s.tantilov@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
index 99e472e..f8d3dec 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
@@ -1125,7 +1125,7 @@ s32 ixgbe_read_eerd_buffer_generic(struct ixgbe_hw *hw, u16 offset,
 	}
 
 	for (i = 0; i < words; i++) {
-		eerd = ((offset + i) << IXGBE_EEPROM_RW_ADDR_SHIFT) +
+		eerd = ((offset + i) << IXGBE_EEPROM_RW_ADDR_SHIFT) |
 		       IXGBE_EEPROM_RW_REG_START;
 
 		IXGBE_WRITE_REG(hw, IXGBE_EERD, eerd);
-- 
1.7.11.7

[net-next 04/11] ixgbe: Drop check for PAGE_SIZE from ixgbe_xmit_frame_ring

[Jeff Kirsher]

From: Alexander Duyck <alexander.h.duyck@intel.com>

The check for PAGE_SIZE is pointless now that the default configuration is to
allocate 32K for all buffers.  Since the Tx descriptor limit is 16K we can
just drop the check and always compare the descriptors to the maximum size
supported.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 096a221..3ae311d 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -6522,9 +6522,7 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
 	struct ixgbe_tx_buffer *first;
 	int tso;
 	u32 tx_flags = 0;
-#if PAGE_SIZE > IXGBE_MAX_DATA_PER_TXD
 	unsigned short f;
-#endif
 	u16 count = TXD_USE_COUNT(skb_headlen(skb));
 	__be16 protocol = skb->protocol;
 	u8 hdr_len = 0;
@@ -6536,12 +6534,9 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
 	 *       + 1 desc for context descriptor,
 	 * otherwise try next time
 	 */
-#if PAGE_SIZE > IXGBE_MAX_DATA_PER_TXD
 	for (f = 0; f < skb_shinfo(skb)->nr_frags; f++)
 		count += TXD_USE_COUNT(skb_shinfo(skb)->frags[f].size);
-#else
-	count += skb_shinfo(skb)->nr_frags;
-#endif
+
 	if (ixgbe_maybe_stop_tx(tx_ring, count + 3)) {
 		tx_ring->tx_stats.tx_busy++;
 		return NETDEV_TX_BUSY;
-- 
1.7.11.7

[net-next 05/11] ixgbe: Enable support for recognizing PCI-e Gen3 link speed

[Jeff Kirsher]

From: Jacob Keller <jacob.e.keller@intel.com>

This patch adds support for displaying PCIe Gen3 link speed, which was
previously missing from the driver.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.c | 3 +++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c   | 3 ++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_type.h   | 2 ++
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
index f8d3dec..3f66abc 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
@@ -635,6 +635,9 @@ s32 ixgbe_get_bus_info_generic(struct ixgbe_hw *hw)
 	case IXGBE_PCI_LINK_SPEED_5000:
 		hw->bus.speed = ixgbe_bus_speed_5000;
 		break;
+	case IXGBE_PCI_LINK_SPEED_8000:
+		hw->bus.speed = ixgbe_bus_speed_8000;
+		break;
 	default:
 		hw->bus.speed = ixgbe_bus_speed_unknown;
 		break;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 3ae311d..35727dc 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7587,7 +7587,8 @@ skip_sriov:
 
 	/* print bus type/speed/width info */
 	e_dev_info("(PCI Express:%s:%s) %pM\n",
-		   (hw->bus.speed == ixgbe_bus_speed_5000 ? "5.0GT/s" :
+		   (hw->bus.speed == ixgbe_bus_speed_8000 ? "8.0GT/s" :
+		    hw->bus.speed == ixgbe_bus_speed_5000 ? "5.0GT/s" :
 		    hw->bus.speed == ixgbe_bus_speed_2500 ? "2.5GT/s" :
 		    "Unknown"),
 		   (hw->bus.width == ixgbe_bus_width_pcie_x8 ? "Width x8" :
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
index 6652e96..05df36e 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
@@ -1827,6 +1827,7 @@ enum {
 #define IXGBE_PCI_LINK_SPEED      0xF
 #define IXGBE_PCI_LINK_SPEED_2500 0x1
 #define IXGBE_PCI_LINK_SPEED_5000 0x2
+#define IXGBE_PCI_LINK_SPEED_8000 0x3
 #define IXGBE_PCI_HEADER_TYPE_REGISTER  0x0E
 #define IXGBE_PCI_HEADER_TYPE_MULTIFUNC 0x80
 #define IXGBE_PCI_DEVICE_CONTROL2_16ms  0x0005
@@ -2650,6 +2651,7 @@ enum ixgbe_bus_speed {
 	ixgbe_bus_speed_133     = 133,
 	ixgbe_bus_speed_2500    = 2500,
 	ixgbe_bus_speed_5000    = 5000,
+	ixgbe_bus_speed_8000    = 8000,
 	ixgbe_bus_speed_reserved
 };
 
-- 
1.7.11.7

[net-next 06/11] ixgbe: create conversion functions from link_status to bus/speed

[Jeff Kirsher]

From: Jacob Keller <jacob.e.keller@intel.com>

This patch cleans up ixgbe_get_bus_info_generic to call some conversion
functions, which are used also in a follow on patch that needs to convert
between the link_status PCIe config values into ixgbe's internal enum
representations.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.c | 64 ++++++++++++-------------
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.h |  2 +
 2 files changed, 34 insertions(+), 32 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
index 3f66abc..9bcdeb8 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
@@ -592,6 +592,36 @@ s32 ixgbe_get_mac_addr_generic(struct ixgbe_hw *hw, u8 *mac_addr)
 	return 0;
 }
 
+enum ixgbe_bus_width ixgbe_convert_bus_width(u16 link_status)
+{
+	switch (link_status & IXGBE_PCI_LINK_WIDTH) {
+	case IXGBE_PCI_LINK_WIDTH_1:
+		return ixgbe_bus_width_pcie_x1;
+	case IXGBE_PCI_LINK_WIDTH_2:
+		return ixgbe_bus_width_pcie_x2;
+	case IXGBE_PCI_LINK_WIDTH_4:
+		return ixgbe_bus_width_pcie_x4;
+	case IXGBE_PCI_LINK_WIDTH_8:
+		return ixgbe_bus_width_pcie_x8;
+	default:
+		return ixgbe_bus_width_unknown;
+	}
+}
+
+enum ixgbe_bus_speed ixgbe_convert_bus_speed(u16 link_status)
+{
+	switch (link_status & IXGBE_PCI_LINK_SPEED) {
+	case IXGBE_PCI_LINK_SPEED_2500:
+		return ixgbe_bus_speed_2500;
+	case IXGBE_PCI_LINK_SPEED_5000:
+		return ixgbe_bus_speed_5000;
+	case IXGBE_PCI_LINK_SPEED_8000:
+		return ixgbe_bus_speed_8000;
+	default:
+		return ixgbe_bus_speed_unknown;
+	}
+}
+
 /**
  *  ixgbe_get_bus_info_generic - Generic set PCI bus info
  *  @hw: pointer to hardware structure
@@ -610,38 +640,8 @@ s32 ixgbe_get_bus_info_generic(struct ixgbe_hw *hw)
 	pci_read_config_word(adapter->pdev, IXGBE_PCI_LINK_STATUS,
 	                     &link_status);
 
-	switch (link_status & IXGBE_PCI_LINK_WIDTH) {
-	case IXGBE_PCI_LINK_WIDTH_1:
-		hw->bus.width = ixgbe_bus_width_pcie_x1;
-		break;
-	case IXGBE_PCI_LINK_WIDTH_2:
-		hw->bus.width = ixgbe_bus_width_pcie_x2;
-		break;
-	case IXGBE_PCI_LINK_WIDTH_4:
-		hw->bus.width = ixgbe_bus_width_pcie_x4;
-		break;
-	case IXGBE_PCI_LINK_WIDTH_8:
-		hw->bus.width = ixgbe_bus_width_pcie_x8;
-		break;
-	default:
-		hw->bus.width = ixgbe_bus_width_unknown;
-		break;
-	}
-
-	switch (link_status & IXGBE_PCI_LINK_SPEED) {
-	case IXGBE_PCI_LINK_SPEED_2500:
-		hw->bus.speed = ixgbe_bus_speed_2500;
-		break;
-	case IXGBE_PCI_LINK_SPEED_5000:
-		hw->bus.speed = ixgbe_bus_speed_5000;
-		break;
-	case IXGBE_PCI_LINK_SPEED_8000:
-		hw->bus.speed = ixgbe_bus_speed_8000;
-		break;
-	default:
-		hw->bus.speed = ixgbe_bus_speed_unknown;
-		break;
-	}
+	hw->bus.width = ixgbe_convert_bus_width(link_status);
+	hw->bus.speed = ixgbe_convert_bus_speed(link_status);
 
 	mac->ops.set_lan_id(hw);
 
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
index bc3948ea..22eee38 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
@@ -40,6 +40,8 @@ s32 ixgbe_clear_hw_cntrs_generic(struct ixgbe_hw *hw);
 s32 ixgbe_read_pba_string_generic(struct ixgbe_hw *hw, u8 *pba_num,
                                   u32 pba_num_size);
 s32 ixgbe_get_mac_addr_generic(struct ixgbe_hw *hw, u8 *mac_addr);
+enum ixgbe_bus_width ixgbe_convert_bus_width(u16 link_status);
+enum ixgbe_bus_speed ixgbe_convert_bus_speed(u16 link_status);
 s32 ixgbe_get_bus_info_generic(struct ixgbe_hw *hw);
 void ixgbe_set_lan_id_multi_port_pcie(struct ixgbe_hw *hw);
 s32 ixgbe_stop_adapter_generic(struct ixgbe_hw *hw);
-- 
1.7.11.7

[net-next 07/11] ixgbe: enable devices with internal switch to read pci parent

[Jeff Kirsher]

From: Jacob Keller <jacob.e.keller@intel.com>

This patch modifies the driver to enable certain devices, which have an internal
switch, to read data from the physical slot rather than reading data from the
internal switch. The internal switch will always report the same PCI width and
speed, which is not useful compared to knowing the width and speed of the slot
the physical card is plugged into.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 48 +++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 35727dc..e02b4e4 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -149,6 +149,52 @@ MODULE_DESCRIPTION("Intel(R) 10 Gigabit PCI Express Network Driver");
 MODULE_LICENSE("GPL");
 MODULE_VERSION(DRV_VERSION);
 
+static int ixgbe_read_pci_cfg_word_parent(struct ixgbe_adapter *adapter,
+					  u32 reg, u16 *value)
+{
+	int pos = 0;
+	struct pci_dev *parent_dev;
+	struct pci_bus *parent_bus;
+
+	parent_bus = adapter->pdev->bus->parent;
+	if (!parent_bus)
+		return -1;
+
+	parent_dev = parent_bus->self;
+	if (!parent_dev)
+		return -1;
+
+	pos = pci_find_capability(parent_dev, PCI_CAP_ID_EXP);
+	if (!pos)
+		return -1;
+
+	pci_read_config_word(parent_dev, pos + reg, value);
+	return 0;
+}
+
+static s32 ixgbe_get_parent_bus_info(struct ixgbe_adapter *adapter)
+{
+	struct ixgbe_hw *hw = &adapter->hw;
+	u16 link_status = 0;
+	int err;
+
+	hw->bus.type = ixgbe_bus_type_pci_express;
+
+	/* Get the negotiated link width and speed from PCI config space of the
+	 * parent, as this device is behind a switch
+	 */
+	err = ixgbe_read_pci_cfg_word_parent(adapter, 18, &link_status);
+
+	/* assume caller will handle error case */
+	if (err)
+		return err;
+
+	hw->bus.width = ixgbe_convert_bus_width(link_status);
+	hw->bus.speed = ixgbe_convert_bus_speed(link_status);
+
+	return 0;
+}
+
 static void ixgbe_service_event_schedule(struct ixgbe_adapter *adapter)
 {
 	if (!test_bit(__IXGBE_DOWN, &adapter->state) &&
@@ -7584,6 +7630,8 @@ skip_sriov:
 
 	/* pick up the PCI bus settings for reporting later */
 	hw->mac.ops.get_bus_info(hw);
+	if (hw->device_id == IXGBE_DEV_ID_82599_SFP_SF_QP)
+		ixgbe_get_parent_bus_info(adapter);
 
 	/* print bus type/speed/width info */
 	e_dev_info("(PCI Express:%s:%s) %pM\n",
-- 
1.7.11.7

[net-next 08/11] ixgbe: walk pci-e bus to find minimum width

[Jeff Kirsher]

From: Jacob Keller <jacob.e.keller@intel.com>

This patch adds a function which enables the ixgbe driver to walk up the PCI bus
for the device and query the PCI config space for the bus width at each
point. This enables the driver to determine what the minimum PCIe width is for
the device, and warn the user if it is not enough.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 50 +++++++++++++++++++++++----
 1 file changed, 44 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index e02b4e4..8793aae 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -172,6 +172,49 @@ static int ixgbe_read_pci_cfg_word_parent(struct ixgbe_adapter *adapter,
 	return 0;
 }
 
+static void ixgbe_check_minimum_bus_width(struct ixgbe_adapter *adapter)
+{
+	int pos = 0;
+	struct pci_dev *dev = adapter->pdev;
+	enum ixgbe_bus_width minimum_width = adapter->hw.bus.width;
+	enum ixgbe_bus_speed minimum_speed = adapter->hw.bus.speed;
+
+	do {
+		u16 link_status;
+		enum ixgbe_bus_width next_width;
+		enum ixgbe_bus_speed next_speed;
+		struct pci_bus *bus;
+
+		pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
+		if (!pos)
+			return;
+
+		pci_read_config_word(dev, pos + 18, &link_status);
+		next_width = ixgbe_convert_bus_width(link_status);
+		next_speed = ixgbe_convert_bus_speed(link_status);
+
+		if (next_width < minimum_width)
+			minimum_width = next_width;
+		if (next_speed < minimum_speed)
+			minimum_speed = next_speed;
+
+		if ((minimum_width < ixgbe_bus_width_pcie_x4) ||
+		    ((minimum_width == ixgbe_bus_width_pcie_x4) &&
+		     (minimum_speed <= ixgbe_bus_speed_5000))) {
+			e_dev_warn("PCI-Express bandwidth available for this card is not sufficient for optimal performance.\n");
+			e_dev_warn("For optimal performance a x8 PCI-Express slot is required.\n");
+			return;
+		}
+
+		bus = dev->bus->parent;
+		if (!bus)
+			return;
+
+		dev = bus->self;
+
+	} while (dev);
+}
+
 static s32 ixgbe_get_parent_bus_info(struct ixgbe_adapter *adapter)
 {
 	struct ixgbe_hw *hw = &adapter->hw;
@@ -7656,12 +7699,7 @@ skip_sriov:
 		e_dev_info("MAC: %d, PHY: %d, PBA No: %s\n",
 			   hw->mac.type, hw->phy.type, part_str);
 
-	if (hw->bus.width <= ixgbe_bus_width_pcie_x4) {
-		e_dev_warn("PCI-Express bandwidth available for this card is "
-			   "not sufficient for optimal performance.\n");
-		e_dev_warn("For optimal performance a x8 PCI-Express slot "
-			   "is required.\n");
-	}
+	ixgbe_check_minimum_bus_width(adapter);
 
 	/* reset the hardware with the new settings */
 	err = hw->mac.ops.start_hw(hw);
-- 
1.7.11.7

[net-next 09/11] ixgbe: fix MNG FW support when adapter not up

[Jeff Kirsher]

From: Don Skidmore <donald.c.skidmore@intel.com>

We were only turning the laser on when the adapter was up.  This
causes issues for those who wanted to access the MNG FW while the
port was in a down state.  This patch makes sure the laser is turned
on in probe and remain up even after the port is brought down.

Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c |  1 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c | 48 +++++++++++++++++++++++---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c  | 10 ++++++
 drivers/net/ethernet/intel/ixgbe/ixgbe_type.h  | 10 ++++++
 drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c  |  1 +
 5 files changed, 65 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
index d0113fc..4a5bfb6 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
@@ -1305,6 +1305,7 @@ static struct ixgbe_mac_operations mac_ops_82598 = {
 	.release_swfw_sync      = &ixgbe_release_swfw_sync,
 	.get_thermal_sensor_data = NULL,
 	.init_thermal_sensor_thresh = NULL,
+	.mng_fw_enabled		= NULL,
 };
 
 static struct ixgbe_eeprom_operations eeprom_ops_82598 = {
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
index 203a00c..b6289f1 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
@@ -59,12 +59,34 @@ static s32 ixgbe_setup_copper_link_82599(struct ixgbe_hw *hw,
                                          bool autoneg_wait_to_complete);
 static s32 ixgbe_verify_fw_version_82599(struct ixgbe_hw *hw);
 
+static bool ixgbe_mng_enabled(struct ixgbe_hw *hw)
+{
+	u32 fwsm, manc, factps;
+
+	fwsm = IXGBE_READ_REG(hw, IXGBE_FWSM);
+	if ((fwsm & IXGBE_FWSM_MODE_MASK) != IXGBE_FWSM_FW_MODE_PT)
+		return false;
+
+	manc = IXGBE_READ_REG(hw, IXGBE_MANC);
+	if (!(manc & IXGBE_MANC_RCV_TCO_EN))
+		return false;
+
+	factps = IXGBE_READ_REG(hw, IXGBE_FACTPS);
+	if (factps & IXGBE_FACTPS_MNGCG)
+		return false;
+
+	return true;
+}
+
 static void ixgbe_init_mac_link_ops_82599(struct ixgbe_hw *hw)
 {
 	struct ixgbe_mac_info *mac = &hw->mac;
 
-	/* enable the laser control functions for SFP+ fiber */
-	if (mac->ops.get_media_type(hw) == ixgbe_media_type_fiber) {
+	/* enable the laser control functions for SFP+ fiber
+	 * and MNG not enabled
+	 */
+	if ((mac->ops.get_media_type(hw) == ixgbe_media_type_fiber) &&
+	    !hw->mng_fw_enabled) {
 		mac->ops.disable_tx_laser =
 		                       &ixgbe_disable_tx_laser_multispeed_fiber;
 		mac->ops.enable_tx_laser =
@@ -563,7 +585,8 @@ static s32 ixgbe_setup_mac_link_multispeed_fiber(struct ixgbe_hw *hw,
 			return status;
 
 		/* Flap the tx laser if it has not already been done */
-		hw->mac.ops.flap_tx_laser(hw);
+		if (hw->mac.ops.flap_tx_laser)
+			hw->mac.ops.flap_tx_laser(hw);
 
 		/*
 		 * Wait for the controller to acquire link.  Per IEEE 802.3ap,
@@ -615,7 +638,8 @@ static s32 ixgbe_setup_mac_link_multispeed_fiber(struct ixgbe_hw *hw,
 			return status;
 
 		/* Flap the tx laser if it has not already been done */
-		hw->mac.ops.flap_tx_laser(hw);
+		if (hw->mac.ops.flap_tx_laser)
+			hw->mac.ops.flap_tx_laser(hw);
 
 		/* Wait for the link partner to also set speed */
 		msleep(100);
@@ -933,6 +957,7 @@ static s32 ixgbe_reset_hw_82599(struct ixgbe_hw *hw)
 	ixgbe_link_speed link_speed;
 	s32 status;
 	u32 ctrl, i, autoc, autoc2;
+	u32 curr_lms;
 	bool link_up = false;
 
 	/* Call adapter stop to disable tx/rx and clear interrupts */
@@ -964,6 +989,9 @@ static s32 ixgbe_reset_hw_82599(struct ixgbe_hw *hw)
 	if (hw->phy.reset_disable == false && hw->phy.ops.reset != NULL)
 		hw->phy.ops.reset(hw);
 
+	/* remember AUTOC LMS from before we reset */
+	curr_lms = IXGBE_READ_REG(hw, IXGBE_AUTOC) & IXGBE_AUTOC_LMS_MASK;
+
 mac_reset_top:
 	/*
 	 * Issue global reset to the MAC. Needs to be SW reset if link is up.
@@ -1019,6 +1047,16 @@ mac_reset_top:
 		hw->mac.orig_autoc2 = autoc2;
 		hw->mac.orig_link_settings_stored = true;
 	} else {
+
+		/* If MNG FW is running on a multi-speed device that
+		 * doesn't autoneg with out driver support we need to
+		 * leave LMS in the state it was before we MAC reset.
+		 */
+		if (hw->phy.multispeed_fiber && hw->mng_fw_enabled)
+			hw->mac.orig_autoc =
+				(hw->mac.orig_autoc & ~IXGBE_AUTOC_LMS_MASK) |
+				curr_lms;
+
 		if (autoc != hw->mac.orig_autoc) {
 			/* Need SW/FW semaphore around AUTOC writes if LESM is
 			 * on, likewise reset_pipeline requires us to hold
@@ -2216,7 +2254,7 @@ static struct ixgbe_mac_operations mac_ops_82599 = {
 	.release_swfw_sync      = &ixgbe_release_swfw_sync,
 	.get_thermal_sensor_data = &ixgbe_get_thermal_sensor_data_generic,
 	.init_thermal_sensor_thresh = &ixgbe_init_thermal_sensor_thresh_generic,
-
+	.mng_fw_enabled		= &ixgbe_mng_enabled,
 };
 
 static struct ixgbe_eeprom_operations eeprom_ops_82599 = {
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 8793aae..ed0a6b6 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7510,6 +7510,10 @@ static int ixgbe_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	if (err)
 		goto err_sw_init;
 
+	/* Cache if MNG FW is up so we don't have to read the REG later */
+	if (hw->mac.ops.mng_fw_enabled)
+		hw->mng_fw_enabled = hw->mac.ops.mng_fw_enabled(hw);
+
 	/* Make it possible the adapter to be woken up via WOL */
 	switch (adapter->hw.mac.type) {
 	case ixgbe_mac_82599EB:
@@ -7758,6 +7762,12 @@ skip_sriov:
 	ixgbe_dbg_adapter_init(adapter);
 #endif /* CONFIG_DEBUG_FS */
 
+	/* Need link setup for MNG FW, else wait for IXGBE_UP */
+	if (hw->mng_fw_enabled && hw->mac.ops.setup_link)
+		hw->mac.ops.setup_link(hw,
+			IXGBE_LINK_SPEED_10GB_FULL | IXGBE_LINK_SPEED_1GB_FULL,
+			true);
+
 	return 0;
 
 err_register:
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
index 05df36e..200b1a8 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
@@ -729,6 +729,13 @@ struct ixgbe_thermal_sensor_data {
 #define IXGBE_MDEF_EXT(_i) (0x05160 + ((_i) * 4)) /* 8 of these (0-7) */
 #define IXGBE_LSWFW     0x15014
 
+/* Management Bit Fields and Masks */
+#define IXGBE_MANC_RCV_TCO_EN	0x00020000 /* Rcv TCO packet enable */
+
+/* Firmware Semaphore Register */
+#define IXGBE_FWSM_MODE_MASK	0xE
+#define IXGBE_FWSM_FW_MODE_PT	0x4
+
 /* ARC Subsystem registers */
 #define IXGBE_HICR      0x15F00
 #define IXGBE_FWSTS     0x15F0C
@@ -1019,6 +1026,7 @@ struct ixgbe_thermal_sensor_data {
 #define IXGBE_CTRL_RST_MASK     (IXGBE_CTRL_LNK_RST | IXGBE_CTRL_RST)
 
 /* FACTPS */
+#define IXGBE_FACTPS_MNGCG      0x20000000 /* Manageblility Clock Gated */
 #define IXGBE_FACTPS_LFS        0x40000000 /* LAN Function Select */
 
 /* MHADD Bit Masks */
@@ -2861,6 +2869,7 @@ struct ixgbe_mac_operations {
 	s32 (*set_fw_drv_ver)(struct ixgbe_hw *, u8, u8, u8, u8);
 	s32 (*get_thermal_sensor_data)(struct ixgbe_hw *);
 	s32 (*init_thermal_sensor_thresh)(struct ixgbe_hw *hw);
+	bool (*mng_fw_enabled)(struct ixgbe_hw *hw);
 };
 
 struct ixgbe_phy_operations {
@@ -2988,6 +2997,7 @@ struct ixgbe_hw {
 	bool				adapter_stopped;
 	bool				force_full_reset;
 	bool				allow_unsupported_sfp;
+	bool				mng_fw_enabled;
 };
 
 struct ixgbe_info {
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c
index 66c5e94..389324f 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c
@@ -854,6 +854,7 @@ static struct ixgbe_mac_operations mac_ops_X540 = {
 	.enable_rx_buff		= &ixgbe_enable_rx_buff_generic,
 	.get_thermal_sensor_data = NULL,
 	.init_thermal_sensor_thresh = NULL,
+	.mng_fw_enabled		= NULL,
 };
 
 static struct ixgbe_eeprom_operations eeprom_ops_X540 = {
-- 
1.7.11.7

[net-next 10/11] ixgbe: Fix 1G link WoL

[Jeff Kirsher]

From: Don Skidmore <donald.c.skidmore@intel.com>

We reset during the shutdown path which will reset AUTOC register.  This
would change LMS to 10G.  If we were currently linked at 1G we will lose
link, which is a bad thing if we wanted WoL to work.  For the fix I needed
to know if WoL is supported so I created a new bool in the ixgbe_hw struct.
If this is set we will not allow the reset to change the current LMS value
in AUTOC.

Signed-off-by: Don Skidmore <donald.c.skidmore@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c | 5 ++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c  | 4 +++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_type.h  | 1 +
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
index b6289f1..7946da9 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c
@@ -1051,8 +1051,11 @@ mac_reset_top:
 		/* If MNG FW is running on a multi-speed device that
 		 * doesn't autoneg with out driver support we need to
 		 * leave LMS in the state it was before we MAC reset.
+		 * Likewise if we support WoL we don't want change the
+		 * LMS state either.
 		 */
-		if (hw->phy.multispeed_fiber && hw->mng_fw_enabled)
+		if ((hw->phy.multispeed_fiber && hw->mng_fw_enabled) ||
+		    hw->wol_supported)
 			hw->mac.orig_autoc =
 				(hw->mac.orig_autoc & ~IXGBE_AUTOC_LMS_MASK) |
 				curr_lms;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index ed0a6b6..d89d764 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7666,7 +7666,9 @@ skip_sriov:
 	/* WOL not supported for all devices */
 	adapter->wol = 0;
 	hw->eeprom.ops.read(hw, 0x2c, &adapter->eeprom_cap);
-	if (ixgbe_wol_supported(adapter, pdev->device, pdev->subsystem_device))
+	hw->wol_supported = ixgbe_wol_supported(adapter, pdev->device,
+						pdev->subsystem_device);
+	if (hw->wol_supported)
 		adapter->wol = IXGBE_WUFC_MAG;
 
 	device_set_wakeup_enable(&adapter->pdev->dev, adapter->wol);
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
index 200b1a8..155a793 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
@@ -2998,6 +2998,7 @@ struct ixgbe_hw {
 	bool				force_full_reset;
 	bool				allow_unsupported_sfp;
 	bool				mng_fw_enabled;
+	bool				wol_supported;
 };
 
 struct ixgbe_info {
-- 
1.7.11.7

[net-next 11/11] ixgbe: bump version number

[Jeff Kirsher]

From: Don Skidmore <donald.c.skidmore@intel.com>

Bump the version number reflect the corresponding functionality in the
out of tree driver.

Signed-of-by: Don Skidmore <donald.c.skidmore@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>

Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index d89d764..2fd303c 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -63,7 +63,7 @@ char ixgbe_default_device_descr[] =
 static char ixgbe_default_device_descr[] =
 			      "Intel(R) 10 Gigabit Network Connection";
 #endif
-#define DRV_VERSION "3.11.33-k"
+#define DRV_VERSION "3.13.10-k"
 const char ixgbe_driver_version[] = DRV_VERSION;
 static const char ixgbe_copyright[] =
 				"Copyright (c) 1999-2013 Intel Corporation.";
-- 
1.7.11.7

Re: [net-next 01/11] ixgbe: Support using build_skb in the case that jumbo frames are disabled

[Eric Dumazet]

On Fri, 2013-04-12 at 04:24 -0700, Jeff Kirsher wrote:
> From: Alexander Duyck <alexander.h.duyck@intel.com>
> 
> This change makes it so that we can enable the use of build_skb for cases
> where jumbo frames are disabled.  The advantage to this is that we do not have
> to perform a memcpy to populate the header and as a result we see a
> significant performance improvement.
> 

> +	unsigned int size = le16_to_cpu(rx_desc->wb.upper.length);
>  #if (PAGE_SIZE < 8192)
> -	/* if we are only owner of page we can reuse it */
> -	if (unlikely(page_count(page) != 1))
> -		return false;
> +	unsigned int truesize = ixgbe_rx_bufsz(rx_ring);
> +#else
> +	unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +
> +				SKB_DATA_ALIGN(NET_SKB_PAD +
> +					       NET_IP_ALIGN +
> +					       size);
> +#endif
>  
> -	/* flip page offset to other buffer */
> -	rx_buffer->page_offset ^= truesize;
> +	/* If we spanned a buffer we have a huge mess so test for it */
> +	BUG_ON(unlikely(!ixgbe_test_staterr(rx_desc, IXGBE_RXD_STAT_EOP)));
>  
> -	/*
> -	 * since we are the only owner of the page and we need to
> -	 * increment it, just set the value to 2 in order to avoid
> -	 * an unecessary locked operation
> -	 */
> -	atomic_set(&page->_count, 2);
> -#else
> -	/* move offset up to the next cache line */
> -	rx_buffer->page_offset += truesize;
> +	/* Guarantee this function can be used by verifying buffer sizes */
> +	BUILD_BUG_ON(SKB_WITH_OVERHEAD(IXGBE_RXBUFFER_2K) < (NET_SKB_PAD +
> +							     NET_IP_ALIGN +
> +							     VLAN_HLEN +
> +							     ETH_FRAME_LEN +
> +							     ETH_FCS_LEN));
>  
> -	if (rx_buffer->page_offset > last_offset)
> -		return false;
> +	rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean];
> +	page = rx_buffer->page;
> +	prefetchw(page);
>  
> -	/* bump ref count on page before it is given to the stack */
> -	get_page(page);
> +	page_addr = page_address(page) + rx_buffer->page_offset;
> +
> +	/* prefetch first cache line of first page */
> +	prefetch(page_addr + NET_SKB_PAD + NET_IP_ALIGN);
> +#if L1_CACHE_BYTES < 128
> +	prefetch(page_addr + L1_CACHE_BYTES + NET_SKB_PAD + NET_IP_ALIGN);
>  #endif
>  
> -	return true;
> +	/* build an skb to go around the page buffer */
> +	skb = build_skb(page_addr, truesize);


Strange. I feel you overestimate the final skb->truesize

The name 'truesize' is a bit misleading here, as its the size of
skb->head. A better name would have been frag_size, and you should not
include in it the struct skb_shared_info overhead because build_skb()
does it :

skb->truesize will be set to SKB_TRUESIZE(frag_size)

#define SKB_TRUESIZE(X) ((X) +                                          \
                         SKB_DATA_ALIGN(sizeof(struct sk_buff)) +       \
                         SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 04:24 -0700, Jeff Kirsher wrote:
> From: Alexander Duyck <alexander.h.duyck@intel.com>
> 
> We were incorrectly checking the entire frag_off field when we only wanted the
> fragment offset.  As a result we were not pulling in TCP headers when the DNF
> flag was set.
> 
> To correct that we will now check for frag off using the IP_OFFSET mask.
> 
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
> Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
> ---
>  drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> index 0c51a9f..096a221 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> @@ -1347,7 +1347,7 @@ static unsigned int ixgbe_get_headlen(unsigned char *data,
>  			return hdr.network - data;
>  
>  		/* record next protocol if header is present */
> -		if (!hdr.ipv4->frag_off)
> +		if (!(hdr.ipv4->frag_off & htons(IP_OFFSET)))
>  			nexthdr = hdr.ipv4->protocol;
>  	} else if (protocol == __constant_htons(ETH_P_IPV6)) {
>  		if ((hdr.network - data) > (max_len - sizeof(struct ipv6hdr)))

I wonder if you could use core functions instead of all this...

A simple wrapper would be :

static noinline unsigned int ixgbe_get_headlen(unsigned char *data,
						u32 maxlen)
{
    struct skb fake;
    unsigned int res;

    fake->data = data;
    fake->head = data;
    fake->data_len = 0;
    fake->len = maxlen;
    skb_reset_network_header(&fake);
    res = __skb_get_poff(&fake);
    return res ?: maxlen;
}

(completely untested)

Re: [net-next 01/11] ixgbe: Support using build_skb in the case that jumbo frames are disabled

[Eric Dumazet]

On Fri, 2013-04-12 at 06:10 -0700, Eric Dumazet wrote:

> Strange. I feel you overestimate the final skb->truesize
> 
> The name 'truesize' is a bit misleading here, as its the size of
> skb->head. A better name would have been frag_size, and you should not
> include in it the struct skb_shared_info overhead because build_skb()
> does it :
> 

Oh well, build_skb() does :

size -= SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); 

so your patch is fine, only the @truesize name is misleading.

> skb->truesize will be set to SKB_TRUESIZE(frag_size)
> 
> #define SKB_TRUESIZE(X) ((X) +                                          \
>                          SKB_DATA_ALIGN(sizeof(struct sk_buff)) +       \
>                          SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))
> 
> 

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 06:28 -0700, Eric Dumazet wrote:

> I wonder if you could use core functions instead of all this...
> 
> A simple wrapper would be :

Or more something like :

static noinline unsigned int ixgbe_get_headlen(unsigned char *data,
                                                u32 maxlen)
{
    struct skb fake;
    unsigned int res;

    if (maxlen < ETH_HLEN)
	return maxlen;

    fake->data = data + ETH_HLEN;
    fake->head = data;
    fake->data_len = 0;
    fake->len = maxlen - ETH_HLEN;
    skb_reset_network_header(&fake);
    res = __skb_get_poff(&fake);
    return res ? res + ETH_HLEN : maxlen;
}

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Alexander Duyck]

On 04/12/2013 06:45 AM, Eric Dumazet wrote:
> On Fri, 2013-04-12 at 06:28 -0700, Eric Dumazet wrote:
>
>> I wonder if you could use core functions instead of all this...
>>
>> A simple wrapper would be :
> Or more something like :
>
> static noinline unsigned int ixgbe_get_headlen(unsigned char *data,
>                                                 u32 maxlen)
> {
>     struct skb fake;
>     unsigned int res;
>
>     if (maxlen < ETH_HLEN)
> 	return maxlen;
>
>     fake->data = data + ETH_HLEN;
>     fake->head = data;
>     fake->data_len = 0;
>     fake->len = maxlen - ETH_HLEN;
>     skb_reset_network_header(&fake);
>     res = __skb_get_poff(&fake);
>     return res ? res + ETH_HLEN : maxlen;
> }

The problem is this is way more then I need, and I would prefer not to
allocate a 192+ byte structure on the stack in order to just parse a
header that is likely less than 128 bytes.

I could probably do something like create a copy of the
ixgbe_get_headlen function, maybe named something like
etherdev_get_headlen and stored in eth.c that could be used by both igb
and ixgbe.  That way it would be available for anyone else who might
want to do something similar.  If that would work for you I could
probably submit that patch sometime in the next few hours.

Thanks,

Alex

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 09:38 -0700, Alexander Duyck wrote:
> On 04/12/2013 06:45 AM, Eric Dumazet wrote:
> > On Fri, 2013-04-12 at 06:28 -0700, Eric Dumazet wrote:
> >
> >> I wonder if you could use core functions instead of all this...
> >>
> >> A simple wrapper would be :
> > Or more something like :
> >
> > static noinline unsigned int ixgbe_get_headlen(unsigned char *data,
> >                                                 u32 maxlen)
> > {
> >     struct skb fake;
> >     unsigned int res;
> >
> >     if (maxlen < ETH_HLEN)
> > 	return maxlen;
> >
> >     fake->data = data + ETH_HLEN;
> >     fake->head = data;
> >     fake->data_len = 0;
> >     fake->len = maxlen - ETH_HLEN;
> >     skb_reset_network_header(&fake);
> >     res = __skb_get_poff(&fake);
> >     return res ? res + ETH_HLEN : maxlen;
> > }
> 
> The problem is this is way more then I need, and I would prefer not to
> allocate a 192+ byte structure on the stack in order to just parse a
> header that is likely less than 128 bytes.
> 

Thats why I used 'noinline' keyword.

Your code adds significative icache pressure and latencies.


> I could probably do something like create a copy of the
> ixgbe_get_headlen function, maybe named something like
> etherdev_get_headlen and stored in eth.c that could be used by both igb
> and ixgbe.  That way it would be available for anyone else who might
> want to do something similar.  If that would work for you I could
> probably submit that patch sometime in the next few hours.

No please don't do that.

I suggested reusing stuff, not duplicating it.

The main problem is not the cpu cycles spent to parse the header, but
bringing two cache lines for the memcpy() to pull headers.  (TCP uses 66
bytes of headers)

If you use a prefetch(data + 64), chances are good the current generic
code will run before hitting the memory stall

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Alexander Duyck]

On 04/12/2013 09:51 AM, Eric Dumazet wrote:
> On Fri, 2013-04-12 at 09:38 -0700, Alexander Duyck wrote:
>> On 04/12/2013 06:45 AM, Eric Dumazet wrote:
>>> On Fri, 2013-04-12 at 06:28 -0700, Eric Dumazet wrote:
>>>
>>>> I wonder if you could use core functions instead of all this...
>>>>
>>>> A simple wrapper would be :
>>> Or more something like :
>>>
>>> static noinline unsigned int ixgbe_get_headlen(unsigned char *data,
>>>                                                 u32 maxlen)
>>> {
>>>     struct skb fake;
>>>     unsigned int res;
>>>
>>>     if (maxlen < ETH_HLEN)
>>> 	return maxlen;
>>>
>>>     fake->data = data + ETH_HLEN;
>>>     fake->head = data;
>>>     fake->data_len = 0;
>>>     fake->len = maxlen - ETH_HLEN;
>>>     skb_reset_network_header(&fake);
>>>     res = __skb_get_poff(&fake);
>>>     return res ? res + ETH_HLEN : maxlen;
>>> }
>> The problem is this is way more then I need, and I would prefer not to
>> allocate a 192+ byte structure on the stack in order to just parse a
>> header that is likely less than 128 bytes.
>>
> Thats why I used 'noinline' keyword.
>
> Your code adds significative icache pressure and latencies.

The footprint for the code itself is not that large, and the fact is the
behavior is different enough from skb_flow_dissect which is what
__skb_get_poff relies on that I don't think I could get the same
behavior without adding at least one more protocol (FCOE), and probably
some sort of flag because in our case we want the header length
including L4 header for the first frame of a fragmented flow since the
goal is to leave only payload data in the pages.

>> I could probably do something like create a copy of the
>> ixgbe_get_headlen function, maybe named something like
>> etherdev_get_headlen and stored in eth.c that could be used by both igb
>> and ixgbe.  That way it would be available for anyone else who might
>> want to do something similar.  If that would work for you I could
>> probably submit that patch sometime in the next few hours.
> No please don't do that.
>
> I suggested reusing stuff, not duplicating it.
>
> The main problem is not the cpu cycles spent to parse the header, but
> bringing two cache lines for the memcpy() to pull headers.  (TCP uses 66
> bytes of headers)
>
> If you use a prefetch(data + 64), chances are good the current generic
> code will run before hitting the memory stall

The main problem I have with this is the fact that before we are done we
would have had to populate a number of fields within the fake skb before
the parsing could be completed.  This also assumes that at no point in
the future will somebody add anything that requires any other fields to
be set or unset within the skb since all of the values in fake are not
memset to 0 like a standard skb.  It would be a pain to debug this type
of issue.

For example the code snippet you sent likely wouldn't have worked
because it appears to have missed that it would also need to set
skb->protocol before being called.

I appreciate the desire to reuse, and what I meant was that since igb
and ixgbe both use essentially the same function I could move it to one
central location and both of them could use it as well as any other low
level drivers that need to just quickly parse the header out of a linear
block of data.  I just don't feel __skb_get_poff really does what I am
looking for since it assumes it is working with a skb, not just a linear
block of data.  If it could get broken up somehow so that it, or at
least pieces of it could just be used on linear blocks of data then I
might be more interested in reusing it.

Thanks,

Alex

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 11:22 -0700, Alexander Duyck wrote:

> I appreciate the desire to reuse, and what I meant was that since igb
> and ixgbe both use essentially the same function I could move it to one
> central location and both of them could use it as well as any other low
> level drivers that need to just quickly parse the header out of a linear
> block of data.  I just don't feel __skb_get_poff really does what I am
> looking for since it assumes it is working with a skb, not just a linear
> block of data.

Big deal.

Filling skb->data/head/len/data_len/network_offset and you're done in 3
cycles or so.

We are constrained by the ~250 cycles the cpu has for free to fetch 2
cache lines from the received frame.

>   If it could get broken up somehow so that it, or at
> least pieces of it could just be used on linear blocks of data then I
> might be more interested in reusing it.

But this is not going to happen. Our skb code is able to deal with
multiple areas (skb->head and frags). We are not going to duplicate it
for linear skbs 'only'

Thats your call obviously to maintain your own code, but consider you
fix this quite incredible bug only now. (All TCP data segments have the
DF flag set, so you were not pulling tcp headers)

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Alexander Duyck]

On 04/12/2013 11:44 AM, Eric Dumazet wrote:
> On Fri, 2013-04-12 at 11:22 -0700, Alexander Duyck wrote:
>>   If it could get broken up somehow so that it, or at
>> least pieces of it could just be used on linear blocks of data then I
>> might be more interested in reusing it.
> But this is not going to happen. Our skb code is able to deal with
> multiple areas (skb->head and frags). We are not going to duplicate it
> for linear skbs 'only'
>
> Thats your call obviously to maintain your own code, but consider you
> fix this quite incredible bug only now. (All TCP data segments have the
> DF flag set, so you were not pulling tcp headers)

I think part of the trouble is we are debating reworking the wrong
function.  We would probably be better off if we could come up with a
generic way to handle the ixgbe_pull_tail function.  I think the only
complaint I would have in using the __skb_get_poff in there is the fact
that it would be copying the header in multiple chunks.  If it worked
more like GRO where it just used an offset in the first frag instead of
having to copy the headers separately to read them then I would be 100%
on board.

Thanks,

Alex

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 13:12 -0700, Alexander Duyck wrote:

> I think part of the trouble is we are debating reworking the wrong
> function.  We would probably be better off if we could come up with a
> generic way to handle the ixgbe_pull_tail function.  I think the only
> complaint I would have in using the __skb_get_poff in there is the fact
> that it would be copying the header in multiple chunks.  If it worked
> more like GRO where it just used an offset in the first frag instead of
> having to copy the headers separately to read them then I would be 100%
> on board.

__skb_get_poff() does no copy at all if you provide a linear 'skb'

Its really fast, I am really sorry you have wrong idea of what is going
on.

static inline void *skb_header_pointer(const struct sk_buff *skb, int offset,
                                       int len, void *buffer)
{
        int hlen = skb_headlen(skb);

        if (hlen - offset >= len)
                return skb->data + offset;


On the other hand GRO engine is way more expensive, now you mention it, we might
use the flow dissector to speed it.

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Alexander Duyck]

On 04/12/2013 01:29 PM, Eric Dumazet wrote:
> On Fri, 2013-04-12 at 13:12 -0700, Alexander Duyck wrote:
>
>> I think part of the trouble is we are debating reworking the wrong
>> function.  We would probably be better off if we could come up with a
>> generic way to handle the ixgbe_pull_tail function.  I think the only
>> complaint I would have in using the __skb_get_poff in there is the fact
>> that it would be copying the header in multiple chunks.  If it worked
>> more like GRO where it just used an offset in the first frag instead of
>> having to copy the headers separately to read them then I would be 100%
>> on board.
> __skb_get_poff() does no copy at all if you provide a linear 'skb'
>
> Its really fast, I am really sorry you have wrong idea of what is going
> on.
>
> static inline void *skb_header_pointer(const struct sk_buff *skb, int offset,
>                                        int len, void *buffer)
> {
>         int hlen = skb_headlen(skb);
>
>         if (hlen - offset >= len)
>                 return skb->data + offset;
>
>
> On the other hand GRO engine is way more expensive, now you mention it, we might
> use the flow dissector to speed it.

I am sure it is fast with a linear skb.  I understand what is going on
in that function, and perhaps my original complaint was not clear.  The
issue I had before is the fact that we were using a big uninitialized
blob that was the fake skb, and I really didn't want to deal with the
potential of somebody expecting something there was that possibly
uninitialized.  Instead I would prefer to deal with what we have,
instead of possibly introducing new issues by using uninitialized memory.

The problem is I have a non-linear skb, with nothing in the skb->data
region.  All my code does now is parse frag 0 to get the size of the
head, and then memcpy that out to skb->data and then update lengths and
offsets.  The problem right now is if I call __skb_get_poff it will go
through the portion of that path you didn't call out that does
skb_copy_bits.

What I would need in order to give up my current solution is a function
that I can pass a fully formed non-linear skb to that will efficiently
parse the header out of frag 0 and place it in skb->data.  What I do not
want to do is hand a partially initialized skb off to some kernel
function and hope that it doesn't do anything I didn't account for.

Perhaps it is just best for me to wait and see what you do with GRO
since it has many of the same requirements that I essentially do.

Thanks,

Alex

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 14:05 -0700, Alexander Duyck wrote:

>  The problem right now is if I call __skb_get_poff it will go
> through the portion of that path you didn't call out that does
> skb_copy_bits.

Absolutely not : skb->data_len is 0.

How possibly can you say such things ?

OK I give up with this topic.

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Alexander Duyck]

On 04/12/2013 02:12 PM, Eric Dumazet wrote:
> On Fri, 2013-04-12 at 14:05 -0700, Alexander Duyck wrote:
>
>>  The problem right now is if I call __skb_get_poff it will go
>> through the portion of that path you didn't call out that does
>> skb_copy_bits.
> Absolutely not : skb->data_len is 0.
>
> How possibly can you say such things ?
>
> OK I give up with this topic.

Yeah, we should agree to disagree since I think at some point we started
arguing two different things.

As for the patch that started this though we are good right?  No problem
with the patch itself, our discussion was over possible future work correct?

Thanks,

Alex

Re: [net-next 02/11] ixgbe: Mask off check of frag_off as we only want fragment offset

[Eric Dumazet]

On Fri, 2013-04-12 at 14:34 -0700, Alexander Duyck wrote:

> As for the patch that started this though we are good right?  No problem
> with the patch itself, our discussion was over possible future work correct?

No problem.

Re: [net-next 01/11] ixgbe: Support using build_skb in the case that jumbo frames are disabled

[Ben Hutchings]

On Fri, 2013-04-12 at 04:24 -0700, Jeff Kirsher wrote:
> From: Alexander Duyck <alexander.h.duyck@intel.com>
> 
> This change makes it so that we can enable the use of build_skb for cases
> where jumbo frames are disabled.  The advantage to this is that we do not have
> to perform a memcpy to populate the header and as a result we see a
> significant performance improvement.

I thought about doing this in sfc, but:

[...]
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
[...] 
> +	/* build an skb to go around the page buffer */
> +	skb = build_skb(page_addr, truesize);
> +	if (unlikely(!skb)) {
> +		rx_ring->rx_stats.alloc_rx_buff_failed++;
> +		return NULL;
> +	}
> +
> +	/* we are reusing so sync this buffer for CPU use */
> +	dma_sync_single_range_for_cpu(rx_ring->dev,
> +				      rx_buffer->dma,
> +				      rx_buffer->page_offset,
> +				      ixgbe_rx_bufsz(rx_ring),
> +				      DMA_FROM_DEVICE);
> +
> +	/* update pointers within the skb to store the data */
> +	skb_reserve(skb, NET_IP_ALIGN + NET_SKB_PAD);
> +	__skb_put(skb, size);
> +
> +	if (ixgbe_can_reuse_rx_page(rx_ring, rx_buffer, page, truesize)) {
> +		/* hand second half of page back to the ring */
> +		ixgbe_reuse_rx_page(rx_ring, rx_buffer);
[...]

Suppose this branch is taken, and then:
1. skb is forwarded to another device
2. Packet headers are modified and it's put into a queue
3. Second packet is received into the other half of this page
4. Page cannot be reused, so is DMA-unmapped
5. The DMA mapping was non-coherent, so unmap copies or invalidates
cache

The headers added in step 2 get trashed in step 5, don't they?

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [net-next 01/11] ixgbe: Support using build_skb in the case that jumbo frames are disabled

[Alexander Duyck]

On 04/12/2013 03:21 PM, Ben Hutchings wrote:
> On Fri, 2013-04-12 at 04:24 -0700, Jeff Kirsher wrote:
>> From: Alexander Duyck <alexander.h.duyck@intel.com>
>>
>> This change makes it so that we can enable the use of build_skb for cases
>> where jumbo frames are disabled.  The advantage to this is that we do not have
>> to perform a memcpy to populate the header and as a result we see a
>> significant performance improvement.
> I thought about doing this in sfc, but:
>
> [...]
>> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
>> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> [...] 
>> +	/* build an skb to go around the page buffer */
>> +	skb = build_skb(page_addr, truesize);
>> +	if (unlikely(!skb)) {
>> +		rx_ring->rx_stats.alloc_rx_buff_failed++;
>> +		return NULL;
>> +	}
>> +
>> +	/* we are reusing so sync this buffer for CPU use */
>> +	dma_sync_single_range_for_cpu(rx_ring->dev,
>> +				      rx_buffer->dma,
>> +				      rx_buffer->page_offset,
>> +				      ixgbe_rx_bufsz(rx_ring),
>> +				      DMA_FROM_DEVICE);
>> +
>> +	/* update pointers within the skb to store the data */
>> +	skb_reserve(skb, NET_IP_ALIGN + NET_SKB_PAD);
>> +	__skb_put(skb, size);
>> +
>> +	if (ixgbe_can_reuse_rx_page(rx_ring, rx_buffer, page, truesize)) {
>> +		/* hand second half of page back to the ring */
>> +		ixgbe_reuse_rx_page(rx_ring, rx_buffer);
> [...]
>
> Suppose this branch is taken, and then:
> 1. skb is forwarded to another device
> 2. Packet headers are modified and it's put into a queue
> 3. Second packet is received into the other half of this page
> 4. Page cannot be reused, so is DMA-unmapped
> 5. The DMA mapping was non-coherent, so unmap copies or invalidates
> cache
>
> The headers added in step 2 get trashed in step 5, don't they?
>
> Ben.

You're right.  I think they do.  It kind of sucks since this was a
pretty good performance improvement.

This patch should not be applied, and I think I have to submit a patch
to revert a similar patch that has already been applied for igb and is
in the net tree.

Thanks,

Alex

Re: [net-next 08/11] ixgbe: walk pci-e bus to find minimum width

[Or Gerlitz]

On Fri, Apr 12, 2013 at 2:24 PM, Jeff Kirsher
<jeffrey.t.kirsher@intel.com> wrote:
> From: Jacob Keller <jacob.e.keller@intel.com>
>
> This patch adds a function which enables the ixgbe driver to walk up the PCI bus
> for the device and query the PCI config space for the bus width at each
> point. This enables the driver to determine what the minimum PCIe width is for
> the device, and warn the user if it is not enough.

If possible without too much pain, I think we may want to add this
somehow as generic function which would allow other drivers to enjoy
from this logic too, agree?

Or.

>
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
> Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
> ---
>  drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 50 +++++++++++++++++++++++----
>  1 file changed, 44 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> index e02b4e4..8793aae 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> @@ -172,6 +172,49 @@ static int ixgbe_read_pci_cfg_word_parent(struct ixgbe_adapter *adapter,
>         return 0;
>  }
>
> +static void ixgbe_check_minimum_bus_width(struct ixgbe_adapter *adapter)
> +{
> +       int pos = 0;
> +       struct pci_dev *dev = adapter->pdev;
> +       enum ixgbe_bus_width minimum_width = adapter->hw.bus.width;
> +       enum ixgbe_bus_speed minimum_speed = adapter->hw.bus.speed;
> +
> +       do {
> +               u16 link_status;
> +               enum ixgbe_bus_width next_width;
> +               enum ixgbe_bus_speed next_speed;
> +               struct pci_bus *bus;
> +
> +               pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
> +               if (!pos)
> +                       return;
> +
> +               pci_read_config_word(dev, pos + 18, &link_status);
> +               next_width = ixgbe_convert_bus_width(link_status);
> +               next_speed = ixgbe_convert_bus_speed(link_status);
> +
> +               if (next_width < minimum_width)
> +                       minimum_width = next_width;
> +               if (next_speed < minimum_speed)
> +                       minimum_speed = next_speed;
> +
> +               if ((minimum_width < ixgbe_bus_width_pcie_x4) ||
> +                   ((minimum_width == ixgbe_bus_width_pcie_x4) &&
> +                    (minimum_speed <= ixgbe_bus_speed_5000))) {
> +                       e_dev_warn("PCI-Express bandwidth available for this card is not sufficient for optimal performance.\n");
> +                       e_dev_warn("For optimal performance a x8 PCI-Express slot is required.\n");
> +                       return;
> +               }
> +
> +               bus = dev->bus->parent;
> +               if (!bus)
> +                       return;
> +
> +               dev = bus->self;
> +
> +       } while (dev);
> +}
> +
>  static s32 ixgbe_get_parent_bus_info(struct ixgbe_adapter *adapter)
>  {
>         struct ixgbe_hw *hw = &adapter->hw;
> @@ -7656,12 +7699,7 @@ skip_sriov:
>                 e_dev_info("MAC: %d, PHY: %d, PBA No: %s\n",
>                            hw->mac.type, hw->phy.type, part_str);
>
> -       if (hw->bus.width <= ixgbe_bus_width_pcie_x4) {
> -               e_dev_warn("PCI-Express bandwidth available for this card is "
> -                          "not sufficient for optimal performance.\n");
> -               e_dev_warn("For optimal performance a x8 PCI-Express slot "
> -                          "is required.\n");
> -       }
> +       ixgbe_check_minimum_bus_width(adapter);
>
>         /* reset the hardware with the new settings */
>         err = hw->mac.ops.start_hw(hw);
> --
> 1.7.11.7

RE: [net-next 08/11] ixgbe: walk pci-e bus to find minimum width

[Keller, Jacob E]

> -----Original Message-----
> From: Or Gerlitz [mailto:or.gerlitz@gmail.com]
> Sent: Saturday, April 13, 2013 2:28 PM
> To: Kirsher, Jeffrey T
> Cc: davem@davemloft.net; Keller, Jacob E; netdev@vger.kernel.org;
> gospo@redhat.com; sassmann@redhat.com
> Subject: Re: [net-next 08/11] ixgbe: walk pci-e bus to find minimum
> width
> 
> On Fri, Apr 12, 2013 at 2:24 PM, Jeff Kirsher
> <jeffrey.t.kirsher@intel.com> wrote:
> > From: Jacob Keller <jacob.e.keller@intel.com>
> >
> > This patch adds a function which enables the ixgbe driver to walk up
> the PCI bus
> > for the device and query the PCI config space for the bus width at each
> > point. This enables the driver to determine what the minimum PCIe
> width is for
> > the device, and warn the user if it is not enough.
> 
> If possible without too much pain, I think we may want to add this
> somehow as generic function which would allow other drivers to enjoy
> from this logic too, agree?
> 
> Or.

I will look into re-writing this as a generic function.

Thanks!

- Jake