* [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-08-31 5:54 ` Loktionov, Aleksandr
2026-09-03 10:27 ` Simon Horman
2026-08-30 23:21 ` [PATCH iwl-next 2/8] e1000e: dump pages for jumbo Rx buffers Matt Vollrath
` (6 subsequent siblings)
7 siblings, 2 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath
When the hardware is configured to not strip the FCS (CrcStripping=0),
the jumbo Rx path would deliver frames with the FCS still attached.
Follow the standard path's convention by stripping the FCS at EOP and
leaving it out of the byte count.
The jumbo path was omitted when the CrcStripping feature was added.
Since v3.3[1], the jumbo path has only been used where the page size is
over 16K, which practically rules out LOMs, BMC sideband, and the need
to leave FCS on frames. The bug is only reachable only by setting
CrcStripping=0 on a platform with nothing that needs it.
This change prepares for convergence of Rx onto the jumbo path in
following patches. The set of h/w routed through this path will expand
to include LOMs.
[1] Commit 79d4e9087a6e ("e1000e: disable Early Receive DMA on ICH LOMs")
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Fixes: eb7c3adb1ca9 ("e1000e: fix IPMI traffic")
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/netdev.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 844f31ab37ad..599600ad695c 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -1631,13 +1631,23 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
}
}
+ /* strip the Ethernet CRC; it may span fragments */
+ if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
+ !(netdev->features & NETIF_F_RXFCS))
+ pskb_trim(skb, skb->len - 4);
+
/* Receive Checksum Offload */
e1000_rx_checksum(adapter, staterr, skb);
e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
- /* probably a little skewed due to removing CRC */
total_rx_bytes += skb->len;
+ /* If configured to store CRC, keep the FCS bytes out of the
+ * total_rx_bytes counter
+ */
+ if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
+ (netdev->features & NETIF_F_RXFCS))
+ total_rx_bytes -= 4;
total_rx_packets++;
/* eth type trans needs skb->data to point to something */
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* RE: [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping
2026-08-30 23:21 ` [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping Matt Vollrath
@ 2026-08-31 5:54 ` Loktionov, Aleksandr
2026-09-03 10:27 ` Simon Horman
1 sibling, 0 replies; 14+ messages in thread
From: Loktionov, Aleksandr @ 2026-08-31 5:54 UTC (permalink / raw)
To: Matt Vollrath, intel-wired-lan@lists.osuosl.org
Cc: netdev@vger.kernel.org, Nguyen, Anthony L, Kitszel, Przemyslaw,
Lobakin, Aleksander, Andrew Lunn, David S . Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Simon Horman, Jonathan Corbet,
Shuah Khan, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: Matt Vollrath <tactii@gmail.com>
> Sent: Monday, August 31, 2026 1:22 AM
> To: intel-wired-lan@lists.osuosl.org
> Cc: netdev@vger.kernel.org; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>; Kitszel, Przemyslaw
> <przemyslaw.kitszel@intel.com>; Lobakin, Aleksander
> <aleksander.lobakin@intel.com>; Andrew Lunn <andrew+netdev@lunn.ch>;
> David S . Miller <davem@davemloft.net>; Eric Dumazet
> <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo Abeni
> <pabeni@redhat.com>; Simon Horman <horms@kernel.org>; Jonathan Corbet
> <corbet@lwn.net>; Shuah Khan <skhan@linuxfoundation.org>; linux-
> doc@vger.kernel.org; linux-kernel@vger.kernel.org; Matt Vollrath
> <tactii@gmail.com>
> Subject: [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping
>
> When the hardware is configured to not strip the FCS (CrcStripping=0),
> the jumbo Rx path would deliver frames with the FCS still attached.
> Follow the standard path's convention by stripping the FCS at EOP and
> leaving it out of the byte count.
>
> The jumbo path was omitted when the CrcStripping feature was added.
> Since v3.3[1], the jumbo path has only been used where the page size
> is over 16K, which practically rules out LOMs, BMC sideband, and the
> need to leave FCS on frames. The bug is only reachable only by setting
"only" is duplicated
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
> CrcStripping=0 on a platform with nothing that needs it.
>
> This change prepares for convergence of Rx onto the jumbo path in
> following patches. The set of h/w routed through this path will expand
> to include LOMs.
>
> [1] Commit 79d4e9087a6e ("e1000e: disable Early Receive DMA on ICH
> LOMs")
>
> Signed-off-by: Matt Vollrath <tactii@gmail.com>
> Fixes: eb7c3adb1ca9 ("e1000e: fix IPMI traffic")
> Assisted-by: Claude:claude-5-fable
> ---
> drivers/net/ethernet/intel/e1000e/netdev.c | 12 +++++++++++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c
> b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 844f31ab37ad..599600ad695c 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -1631,13 +1631,23 @@ static bool e1000_clean_jumbo_rx_irq(struct
> e1000_ring *rx_ring, int *work_done,
> }
> }
>
> + /* strip the Ethernet CRC; it may span fragments */
> + if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
> + !(netdev->features & NETIF_F_RXFCS))
> + pskb_trim(skb, skb->len - 4);
> +
> /* Receive Checksum Offload */
> e1000_rx_checksum(adapter, staterr, skb);
>
> e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss,
> skb);
>
> - /* probably a little skewed due to removing CRC */
> total_rx_bytes += skb->len;
> + /* If configured to store CRC, keep the FCS bytes out of
> the
> + * total_rx_bytes counter
> + */
> + if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
> + (netdev->features & NETIF_F_RXFCS))
> + total_rx_bytes -= 4;
> total_rx_packets++;
>
> /* eth type trans needs skb->data to point to something
> */
> --
> 2.43.0
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping
2026-08-30 23:21 ` [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping Matt Vollrath
2026-08-31 5:54 ` Loktionov, Aleksandr
@ 2026-09-03 10:27 ` Simon Horman
2026-09-03 15:56 ` Matt Vollrath
1 sibling, 1 reply; 14+ messages in thread
From: Simon Horman @ 2026-09-03 10:27 UTC (permalink / raw)
To: tactii
Cc: Simon Horman, intel-wired-lan, netdev, anthony.l.nguyen,
przemyslaw.kitszel, aleksander.lobakin, andrew+netdev, davem,
edumazet, kuba, pabeni, corbet, skhan, linux-doc, linux-kernel
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev/#/patchset/20260830232146.36948-2-tactii@gmail.com?part=1
---
e1000e: add jumbo Rx CRC stripping
When the hardware is configured to not strip the FCS (CrcStripping=0),
the jumbo Rx path would deliver frames with the FCS still attached.
Follow the standard path's convention by stripping the FCS at EOP and
leaving it out of the byte count.
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 844f31ab37ad4..599600ad695cd 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> @@ -1631,13 +1631,23 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
> }
> }
>
> + /* strip the Ethernet CRC; it may span fragments */
> + if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
> + !(netdev->features & NETIF_F_RXFCS))
> + pskb_trim(skb, skb->len - 4);
> +
> /* Receive Checksum Offload */
> e1000_rx_checksum(adapter, staterr, skb);
>
> e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
>
> - /* probably a little skewed due to removing CRC */
> total_rx_bytes += skb->len;
> + /* If configured to store CRC, keep the FCS bytes out of the
> + * total_rx_bytes counter
> + */
> + if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
> + (netdev->features & NETIF_F_RXFCS))
> + total_rx_bytes -= 4;
[Severity: Medium]
Could these lockless evaluations of adapter->flags2 and netdev->features get
out of sync if userspace changes features via ethtool while a packet is being
processed?
If a concurrent change occurs between the first check and this second check,
it appears it could lead to an inconsistent state where a packet's payload is
incorrectly truncated by 4 bytes, or where the byte statistics are overcounted
or undercounted by 4 bytes.
> total_rx_packets++;
>
> /* eth type trans needs skb->data to point to something */
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping
2026-09-03 10:27 ` Simon Horman
@ 2026-09-03 15:56 ` Matt Vollrath
0 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-09-03 15:56 UTC (permalink / raw)
To: Simon Horman
Cc: intel-wired-lan, netdev, anthony.l.nguyen, przemyslaw.kitszel,
aleksander.lobakin, andrew+netdev, davem, edumazet, kuba, pabeni,
corbet, skhan, linux-doc, linux-kernel
On 9/3/26 06:27, Simon Horman wrote:
> This is an AI-generated review of your patch. The human sending this
> email has considered the AI review valid, or at least plausible.
> Full review at: https://sashiko.dev/#/patchset/20260830232146.36948-2-tactii@gmail.com?part=1
> ---
> e1000e: add jumbo Rx CRC stripping
>
> When the hardware is configured to not strip the FCS (CrcStripping=0),
> the jumbo Rx path would deliver frames with the FCS still attached.
> Follow the standard path's convention by stripping the FCS at EOP and
> leaving it out of the byte count.
>
>> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
>> index 844f31ab37ad4..599600ad695cd 100644
>> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
>> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
>> @@ -1631,13 +1631,23 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
>> }
>> }
>>
>> + /* strip the Ethernet CRC; it may span fragments */
>> + if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
>> + !(netdev->features & NETIF_F_RXFCS))
>> + pskb_trim(skb, skb->len - 4);
>> +
>> /* Receive Checksum Offload */
>> e1000_rx_checksum(adapter, staterr, skb);
>>
>> e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
>>
>> - /* probably a little skewed due to removing CRC */
>> total_rx_bytes += skb->len;
>> + /* If configured to store CRC, keep the FCS bytes out of the
>> + * total_rx_bytes counter
>> + */
>> + if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
>> + (netdev->features & NETIF_F_RXFCS))
>> + total_rx_bytes -= 4;
>
> [Severity: Medium]
> Could these lockless evaluations of adapter->flags2 and netdev->features get
> out of sync if userspace changes features via ethtool while a packet is being
> processed?
Yes, this could be prevented by bringing down the adapter before changing bits
in e1000_set_features().
>
> If a concurrent change occurs between the first check and this second check,
> it appears it could lead to an inconsistent state where a packet's payload is
> incorrectly truncated by 4 bytes, or where the byte statistics are overcounted
> or undercounted by 4 bytes.
>
>> total_rx_packets++;
>>
>> /* eth type trans needs skb->data to point to something */
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH iwl-next 2/8] e1000e: dump pages for jumbo Rx buffers
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 3/8] e1000e: prevent race between PM and reset task Matt Vollrath
` (5 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath, stable
The jumbo Rx path keeps its data in buffer_info->page rather than the
skb, which is only a shell. Previously data beyond the end of the skb
allocation would be dumped. The jumbo path would at best dump useless
garbage. At worst it would dump arbitrary kernel memory. This OOB read
is only reachable when page size is >16K and MTU is >1518.
Dump the page instead when it exists. Skip dumping the shell skb left
behind when a jumbo slot is cleaned.
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Fixes: f0c5dadff3fb ("e1000e: fix panic while dumping packets on Tx hang with IOMMU")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/netdev.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 599600ad695c..47ff3c6ab451 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -465,8 +465,23 @@ static void e1000e_dump(struct e1000_adapter *adapter)
(unsigned long long)buffer_info->dma,
buffer_info->skb, next_desc);
+ /* Jumbo buffers land in the page; a cleaned
+ * jumbo slot keeps only its small shell skb
+ * until it is refilled, so only dump an skb
+ * that can hold a whole buffer.
+ */
if (netif_msg_pktdata(adapter) &&
- buffer_info->skb)
+ buffer_info->page)
+ print_hex_dump(KERN_INFO, "",
+ DUMP_PREFIX_ADDRESS, 16,
+ 1,
+ page_address(buffer_info->page),
+ adapter->rx_buffer_len,
+ true);
+ else if (netif_msg_pktdata(adapter) &&
+ buffer_info->skb &&
+ skb_tailroom(buffer_info->skb) >=
+ adapter->rx_buffer_len)
print_hex_dump(KERN_INFO, "",
DUMP_PREFIX_ADDRESS, 16,
1,
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH iwl-next 3/8] e1000e: prevent race between PM and reset task
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 1/8] e1000e: add jumbo Rx CRC stripping Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 2/8] e1000e: dump pages for jumbo Rx buffers Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 4/8] e1000e: remove packet-split Rx path Matt Vollrath
` (4 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath, stable
e1000e_down could be entered simultaneously by the reset task and the
runtime PM suspend callback. Prevent this by acquiring a runtime PM
reference in the reset task.
The intention is that:
* A reset attempted during runtime suspend should drop silently; the
interface is already going down.
* A runtime suspend attempted during a reset will be retried later.
* All callers of e1000e_down are now serialized.
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Fixes: 23606cf5d119 ("e1000e / PCI / PM: Add basic runtime PM support (rev. 4)")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/netdev.c | 25 ++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 47ff3c6ab451..4c15ca307b08 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -6004,20 +6004,37 @@ static void e1000_tx_timeout(struct net_device *netdev, unsigned int __always_un
static void e1000_reset_task(struct work_struct *work)
{
struct e1000_adapter *adapter;
+ struct device *dev;
+ int rc;
+
adapter = container_of(work, struct e1000_adapter, reset_task);
+ dev = &adapter->pdev->dev;
rtnl_lock();
+
+ /* Runtime suspend downs the device without holding rtnl. Hold a
+ * runtime PM reference so it cannot start underneath the reset, and
+ * skip the reset if the device is already suspending or suspended:
+ * resuming resets the hardware anyway.
+ */
+ rc = pm_runtime_get_if_active(dev);
+ if (!rc)
+ goto out_unlock;
+
/* don't run the task if already down */
- if (test_bit(__E1000_DOWN, &adapter->state)) {
- rtnl_unlock();
- return;
- }
+ if (test_bit(__E1000_DOWN, &adapter->state))
+ goto out_put;
if (!(adapter->flags & FLAG_RESTART_NOW)) {
e1000e_dump(adapter);
e_err("Reset adapter unexpectedly\n");
}
e1000e_reinit_locked(adapter);
+
+out_put:
+ if (rc > 0)
+ pm_runtime_put(dev);
+out_unlock:
rtnl_unlock();
}
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH iwl-next 4/8] e1000e: remove packet-split Rx path
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
` (2 preceding siblings ...)
2026-08-30 23:21 ` [PATCH iwl-next 3/8] e1000e: prevent race between PM and reset task Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 5/8] e1000e: always use jumbo " Matt Vollrath
` (3 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath
This approach to Rx has not stood the test of time. All other Intel
drivers have removed it. Do the same for e1000e.
Use the standard path for any MTU <=1518.
Use the jumbo path for any MTU >1518.
This may not be an ideal breakpoint, but the next step is factoring out
the breakpoint by converging on the jumbo Rx path with page pool.
Changes to e1000e_dump should only be removal of the ps path and
un-indentation of the other path from the switch, except the correct
sized u0 overlay is used instead of u1.
The rx_header_split statistic has been removed.
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/e1000.h | 16 +-
drivers/net/ethernet/intel/e1000e/ethtool.c | 1 -
drivers/net/ethernet/intel/e1000e/netdev.c | 637 +++-----------------
3 files changed, 81 insertions(+), 573 deletions(-)
diff --git a/drivers/net/ethernet/intel/e1000e/e1000.h b/drivers/net/ethernet/intel/e1000e/e1000.h
index 63ebe00376f5..533d1981eb5e 100644
--- a/drivers/net/ethernet/intel/e1000e/e1000.h
+++ b/drivers/net/ethernet/intel/e1000e/e1000.h
@@ -121,11 +121,6 @@ enum e1000_boards {
board_pch_ptp
};
-struct e1000_ps_page {
- struct page *page;
- u64 dma; /* must be u64 - written to hw */
-};
-
/* wrappers around a pointer to a socket buffer,
* so a DMA handle can be stored along with the buffer
*/
@@ -143,11 +138,7 @@ struct e1000_buffer {
u16 mapped_as_page;
};
/* Rx */
- struct {
- /* arrays of page information for packet split */
- struct e1000_ps_page *ps_pages;
- struct page *page;
- };
+ struct page *page;
};
};
@@ -265,15 +256,12 @@ struct e1000_adapter {
/* Rx stats */
u64 hw_csum_err;
u64 hw_csum_good;
- u64 rx_hdr_split;
u32 gorc;
u64 gorc_old;
u32 alloc_rx_buff_failed;
u32 rx_dma_failed;
u32 rx_hwtstamp_cleared;
- unsigned int rx_ps_pages;
- u16 rx_ps_bsize0;
u32 max_frame_size;
u32 min_frame_size;
@@ -464,8 +452,6 @@ s32 e1000e_get_base_timinca(struct e1000_adapter *adapter, u32 *timinca);
#define FLAG2_ENABLE_S0IX_FLOWS BIT(15)
#define FLAG2_DISABLE_K1 BIT(16)
-#define E1000_RX_DESC_PS(R, i) \
- (&(((union e1000_rx_desc_packet_split *)((R).desc))[i]))
#define E1000_RX_DESC_EXT(R, i) \
(&(((union e1000_rx_desc_extended *)((R).desc))[i]))
#define E1000_GET_DESC(R, i, type) (&(((struct type *)((R).desc))[i]))
diff --git a/drivers/net/ethernet/intel/e1000e/ethtool.c b/drivers/net/ethernet/intel/e1000e/ethtool.c
index a8b35ae41141..b209b0c3da5f 100644
--- a/drivers/net/ethernet/intel/e1000e/ethtool.c
+++ b/drivers/net/ethernet/intel/e1000e/ethtool.c
@@ -85,7 +85,6 @@ static const struct e1000_stats e1000_gstrings_stats[] = {
E1000_STAT("tx_flow_control_xoff", stats.xofftxc),
E1000_STAT("rx_csum_offload_good", hw_csum_good),
E1000_STAT("rx_csum_offload_errors", hw_csum_err),
- E1000_STAT("rx_header_split", rx_hdr_split),
E1000_STAT("alloc_rx_buff_failed", alloc_rx_buff_failed),
E1000_STAT("tx_smbus", stats.mgptc),
E1000_STAT("rx_smbus", stats.mgprc),
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 4c15ca307b08..4dc3eeb01329 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -183,24 +183,6 @@ static void e1000_regdump(struct e1000_hw *hw, struct e1000_reg_info *reginfo)
pr_info("%-15s %08x %08x\n", rname, regs[0], regs[1]);
}
-static void e1000e_dump_ps_pages(struct e1000_adapter *adapter,
- struct e1000_buffer *bi)
-{
- int i;
- struct e1000_ps_page *ps_page;
-
- for (i = 0; i < adapter->rx_ps_pages; i++) {
- ps_page = &bi->ps_pages[i];
-
- if (ps_page->page) {
- pr_info("packet dump for ps_page %d:\n", i);
- print_hex_dump(KERN_INFO, "", DUMP_PREFIX_ADDRESS,
- 16, 1, page_address(ps_page->page),
- PAGE_SIZE, true);
- }
- }
-}
-
/**
* e1000e_dump - Print registers, Tx-ring and Rx-ring
* @adapter: board private structure
@@ -218,14 +200,7 @@ static void e1000e_dump(struct e1000_adapter *adapter)
} *u0;
struct e1000_buffer *buffer_info;
struct e1000_ring *rx_ring = adapter->rx_ring;
- union e1000_rx_desc_packet_split *rx_desc_ps;
union e1000_rx_desc_extended *rx_desc;
- struct my_u1 {
- __le64 a;
- __le64 b;
- __le64 c;
- __le64 d;
- } *u1;
u32 staterr;
int i = 0;
@@ -340,155 +315,83 @@ static void e1000e_dump(struct e1000_adapter *adapter)
return;
dev_info(&adapter->pdev->dev, "Rx Ring Dump\n");
- switch (adapter->rx_ps_pages) {
- case 1:
- case 2:
- case 3:
- /* [Extended] Packet Split Receive Descriptor Format
- *
- * +-----------------------------------------------------+
- * 0 | Buffer Address 0 [63:0] |
- * +-----------------------------------------------------+
- * 8 | Buffer Address 1 [63:0] |
- * +-----------------------------------------------------+
- * 16 | Buffer Address 2 [63:0] |
- * +-----------------------------------------------------+
- * 24 | Buffer Address 3 [63:0] |
- * +-----------------------------------------------------+
- */
- pr_info("R [desc] [buffer 0 63:0 ] [buffer 1 63:0 ] [buffer 2 63:0 ] [buffer 3 63:0 ] [bi->dma ] [bi->skb] <-- Ext Pkt Split format\n");
- /* [Extended] Receive Descriptor (Write-Back) Format
- *
- * 63 48 47 32 31 13 12 8 7 4 3 0
- * +------------------------------------------------------+
- * 0 | Packet | IP | Rsvd | MRQ | Rsvd | MRQ RSS |
- * | Checksum | Ident | | Queue | | Type |
- * +------------------------------------------------------+
- * 8 | VLAN Tag | Length | Extended Error | Extended Status |
- * +------------------------------------------------------+
- * 63 48 47 32 31 20 19 0
- */
- pr_info("RWB[desc] [ck ipid mrqhsh] [vl l0 ee es] [ l3 l2 l1 hs] [reserved ] ---------------- [bi->skb] <-- Ext Rx Write-Back format\n");
- for (i = 0; i < rx_ring->count; i++) {
- const char *next_desc;
- buffer_info = &rx_ring->buffer_info[i];
- rx_desc_ps = E1000_RX_DESC_PS(*rx_ring, i);
- u1 = (struct my_u1 *)rx_desc_ps;
- staterr =
- le32_to_cpu(rx_desc_ps->wb.middle.status_error);
-
- if (i == rx_ring->next_to_use)
- next_desc = " NTU";
- else if (i == rx_ring->next_to_clean)
- next_desc = " NTC";
- else
- next_desc = "";
-
- if (staterr & E1000_RXD_STAT_DD) {
- /* Descriptor Done */
- pr_info("%s[0x%03X] %016llX %016llX %016llX %016llX ---------------- %p%s\n",
- "RWB", i,
- (unsigned long long)le64_to_cpu(u1->a),
- (unsigned long long)le64_to_cpu(u1->b),
- (unsigned long long)le64_to_cpu(u1->c),
- (unsigned long long)le64_to_cpu(u1->d),
- buffer_info->skb, next_desc);
- } else {
- pr_info("%s[0x%03X] %016llX %016llX %016llX %016llX %016llX %p%s\n",
- "R ", i,
- (unsigned long long)le64_to_cpu(u1->a),
- (unsigned long long)le64_to_cpu(u1->b),
- (unsigned long long)le64_to_cpu(u1->c),
- (unsigned long long)le64_to_cpu(u1->d),
- (unsigned long long)buffer_info->dma,
- buffer_info->skb, next_desc);
-
- if (netif_msg_pktdata(adapter))
- e1000e_dump_ps_pages(adapter,
- buffer_info);
- }
- }
- break;
- default:
- case 0:
- /* Extended Receive Descriptor (Read) Format
- *
- * +-----------------------------------------------------+
- * 0 | Buffer Address [63:0] |
- * +-----------------------------------------------------+
- * 8 | Reserved |
- * +-----------------------------------------------------+
- */
- pr_info("R [desc] [buf addr 63:0 ] [reserved 63:0 ] [bi->dma ] [bi->skb] <-- Ext (Read) format\n");
- /* Extended Receive Descriptor (Write-Back) Format
- *
- * 63 48 47 32 31 24 23 4 3 0
- * +------------------------------------------------------+
- * | RSS Hash | | | |
- * 0 +-------------------+ Rsvd | Reserved | MRQ RSS |
- * | Packet | IP | | | Type |
- * | Checksum | Ident | | | |
- * +------------------------------------------------------+
- * 8 | VLAN Tag | Length | Extended Error | Extended Status |
- * +------------------------------------------------------+
- * 63 48 47 32 31 20 19 0
- */
- pr_info("RWB[desc] [cs ipid mrq] [vt ln xe xs] [bi->skb] <-- Ext (Write-Back) format\n");
+ /* Extended Receive Descriptor (Read) Format
+ *
+ * +-----------------------------------------------------+
+ * 0 | Buffer Address [63:0] |
+ * +-----------------------------------------------------+
+ * 8 | Reserved |
+ * +-----------------------------------------------------+
+ */
+ pr_info("R [desc] [buf addr 63:0 ] [reserved 63:0 ] [bi->dma ] [bi->skb] <-- Ext (Read) format\n");
+ /* Extended Receive Descriptor (Write-Back) Format
+ *
+ * 63 48 47 32 31 24 23 4 3 0
+ * +------------------------------------------------------+
+ * | RSS Hash | | | |
+ * 0 +-------------------+ Rsvd | Reserved | MRQ RSS |
+ * | Packet | IP | | | Type |
+ * | Checksum | Ident | | | |
+ * +------------------------------------------------------+
+ * 8 | VLAN Tag | Length | Extended Error | Extended Status |
+ * +------------------------------------------------------+
+ * 63 48 47 32 31 20 19 0
+ */
+ pr_info("RWB[desc] [cs ipid mrq] [vt ln xe xs] [bi->skb] <-- Ext (Write-Back) format\n");
- for (i = 0; i < rx_ring->count; i++) {
- const char *next_desc;
+ for (i = 0; i < rx_ring->count; i++) {
+ const char *next_desc;
- buffer_info = &rx_ring->buffer_info[i];
- rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
- u1 = (struct my_u1 *)rx_desc;
- staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
+ buffer_info = &rx_ring->buffer_info[i];
+ rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
+ u0 = (struct my_u0 *)rx_desc;
+ staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
- if (i == rx_ring->next_to_use)
- next_desc = " NTU";
- else if (i == rx_ring->next_to_clean)
- next_desc = " NTC";
- else
- next_desc = "";
-
- if (staterr & E1000_RXD_STAT_DD) {
- /* Descriptor Done */
- pr_info("%s[0x%03X] %016llX %016llX ---------------- %p%s\n",
- "RWB", i,
- (unsigned long long)le64_to_cpu(u1->a),
- (unsigned long long)le64_to_cpu(u1->b),
- buffer_info->skb, next_desc);
- } else {
- pr_info("%s[0x%03X] %016llX %016llX %016llX %p%s\n",
- "R ", i,
- (unsigned long long)le64_to_cpu(u1->a),
- (unsigned long long)le64_to_cpu(u1->b),
- (unsigned long long)buffer_info->dma,
- buffer_info->skb, next_desc);
-
- /* Jumbo buffers land in the page; a cleaned
- * jumbo slot keeps only its small shell skb
- * until it is refilled, so only dump an skb
- * that can hold a whole buffer.
- */
- if (netif_msg_pktdata(adapter) &&
- buffer_info->page)
- print_hex_dump(KERN_INFO, "",
- DUMP_PREFIX_ADDRESS, 16,
- 1,
- page_address(buffer_info->page),
- adapter->rx_buffer_len,
- true);
- else if (netif_msg_pktdata(adapter) &&
- buffer_info->skb &&
- skb_tailroom(buffer_info->skb) >=
- adapter->rx_buffer_len)
- print_hex_dump(KERN_INFO, "",
- DUMP_PREFIX_ADDRESS, 16,
- 1,
- buffer_info->skb->data,
- adapter->rx_buffer_len,
- true);
- }
+ if (i == rx_ring->next_to_use)
+ next_desc = " NTU";
+ else if (i == rx_ring->next_to_clean)
+ next_desc = " NTC";
+ else
+ next_desc = "";
+
+ if (staterr & E1000_RXD_STAT_DD) {
+ /* Descriptor Done */
+ pr_info("%s[0x%03X] %016llX %016llX ---------------- %p%s\n",
+ "RWB", i,
+ (unsigned long long)le64_to_cpu(u0->a),
+ (unsigned long long)le64_to_cpu(u0->b),
+ buffer_info->skb, next_desc);
+ } else {
+ pr_info("%s[0x%03X] %016llX %016llX %016llX %p%s\n",
+ "R ", i,
+ (unsigned long long)le64_to_cpu(u0->a),
+ (unsigned long long)le64_to_cpu(u0->b),
+ (unsigned long long)buffer_info->dma,
+ buffer_info->skb, next_desc);
+
+ /* Jumbo buffers land in the page; a cleaned
+ * jumbo slot keeps only its small shell skb
+ * until it is refilled, so only dump an skb
+ * that can hold a whole buffer.
+ */
+ if (netif_msg_pktdata(adapter) &&
+ buffer_info->page)
+ print_hex_dump(KERN_INFO, "",
+ DUMP_PREFIX_ADDRESS, 16,
+ 1,
+ page_address(buffer_info->page),
+ adapter->rx_buffer_len,
+ true);
+ else if (netif_msg_pktdata(adapter) &&
+ buffer_info->skb &&
+ skb_tailroom(buffer_info->skb) >=
+ adapter->rx_buffer_len)
+ print_hex_dump(KERN_INFO, "",
+ DUMP_PREFIX_ADDRESS, 16,
+ 1,
+ buffer_info->skb->data,
+ adapter->rx_buffer_len,
+ true);
}
}
}
@@ -735,110 +638,6 @@ static void e1000_alloc_rx_buffers(struct e1000_ring *rx_ring,
rx_ring->next_to_use = i;
}
-/**
- * e1000_alloc_rx_buffers_ps - Replace used receive buffers; packet split
- * @rx_ring: Rx descriptor ring
- * @cleaned_count: number to reallocate
- * @gfp: flags for allocation
- **/
-static void e1000_alloc_rx_buffers_ps(struct e1000_ring *rx_ring,
- int cleaned_count, gfp_t gfp)
-{
- struct e1000_adapter *adapter = rx_ring->adapter;
- struct net_device *netdev = adapter->netdev;
- struct pci_dev *pdev = adapter->pdev;
- union e1000_rx_desc_packet_split *rx_desc;
- struct e1000_buffer *buffer_info;
- struct e1000_ps_page *ps_page;
- struct sk_buff *skb;
- unsigned int i, j;
-
- i = rx_ring->next_to_use;
- buffer_info = &rx_ring->buffer_info[i];
-
- while (cleaned_count--) {
- rx_desc = E1000_RX_DESC_PS(*rx_ring, i);
-
- for (j = 0; j < PS_PAGE_BUFFERS; j++) {
- ps_page = &buffer_info->ps_pages[j];
- if (j >= adapter->rx_ps_pages) {
- /* all unused desc entries get hw null ptr */
- rx_desc->read.buffer_addr[j + 1] =
- ~cpu_to_le64(0);
- continue;
- }
- if (!ps_page->page) {
- ps_page->page = alloc_page(gfp);
- if (!ps_page->page) {
- adapter->alloc_rx_buff_failed++;
- goto no_buffers;
- }
- ps_page->dma = dma_map_page(&pdev->dev,
- ps_page->page,
- 0, PAGE_SIZE,
- DMA_FROM_DEVICE);
- if (dma_mapping_error(&pdev->dev,
- ps_page->dma)) {
- dev_err(&adapter->pdev->dev,
- "Rx DMA page map failed\n");
- adapter->rx_dma_failed++;
- goto no_buffers;
- }
- }
- /* Refresh the desc even if buffer_addrs
- * didn't change because each write-back
- * erases this info.
- */
- rx_desc->read.buffer_addr[j + 1] =
- cpu_to_le64(ps_page->dma);
- }
-
- skb = __netdev_alloc_skb_ip_align(netdev, adapter->rx_ps_bsize0,
- gfp);
-
- if (!skb) {
- adapter->alloc_rx_buff_failed++;
- break;
- }
-
- buffer_info->skb = skb;
- buffer_info->dma = dma_map_single(&pdev->dev, skb->data,
- adapter->rx_ps_bsize0,
- DMA_FROM_DEVICE);
- if (dma_mapping_error(&pdev->dev, buffer_info->dma)) {
- dev_err(&pdev->dev, "Rx DMA map failed\n");
- adapter->rx_dma_failed++;
- /* cleanup skb */
- dev_kfree_skb_any(skb);
- buffer_info->skb = NULL;
- break;
- }
-
- rx_desc->read.buffer_addr[0] = cpu_to_le64(buffer_info->dma);
-
- if (unlikely(!(i & (E1000_RX_BUFFER_WRITE - 1)))) {
- /* Force memory writes to complete before letting h/w
- * know there are new descriptors to fetch. (Only
- * applicable for weak-ordered memory model archs,
- * such as IA-64).
- */
- wmb();
- if (adapter->flags2 & FLAG2_PCIM2PCI_ARBITER_WA)
- e1000e_update_rdt_wa(rx_ring, i << 1);
- else
- writel(i << 1, rx_ring->tail);
- }
-
- i++;
- if (i == rx_ring->count)
- i = 0;
- buffer_info = &rx_ring->buffer_info[i];
- }
-
-no_buffers:
- rx_ring->next_to_use = i;
-}
-
/**
* e1000_alloc_jumbo_rx_buffers - Replace used jumbo receive buffers
* @rx_ring: Rx descriptor ring
@@ -1326,193 +1125,6 @@ static bool e1000_clean_tx_irq(struct e1000_ring *tx_ring)
return count < tx_ring->count;
}
-/**
- * e1000_clean_rx_irq_ps - Send received data up the network stack; packet split
- * @rx_ring: Rx descriptor ring
- * @work_done: output parameter for indicating completed work
- * @work_to_do: how many packets we can clean
- *
- * the return value indicates whether actual cleaning was done, there
- * is no guarantee that everything was cleaned
- **/
-static bool e1000_clean_rx_irq_ps(struct e1000_ring *rx_ring, int *work_done,
- int work_to_do)
-{
- struct e1000_adapter *adapter = rx_ring->adapter;
- struct e1000_hw *hw = &adapter->hw;
- union e1000_rx_desc_packet_split *rx_desc, *next_rxd;
- struct net_device *netdev = adapter->netdev;
- struct pci_dev *pdev = adapter->pdev;
- struct e1000_buffer *buffer_info, *next_buffer;
- struct e1000_ps_page *ps_page;
- struct sk_buff *skb;
- unsigned int i, j;
- u32 length, staterr;
- int cleaned_count = 0;
- bool cleaned = false;
- unsigned int total_rx_bytes = 0, total_rx_packets = 0;
-
- i = rx_ring->next_to_clean;
- rx_desc = E1000_RX_DESC_PS(*rx_ring, i);
- staterr = le32_to_cpu(rx_desc->wb.middle.status_error);
- buffer_info = &rx_ring->buffer_info[i];
-
- while (staterr & E1000_RXD_STAT_DD) {
- if (*work_done >= work_to_do)
- break;
- (*work_done)++;
- skb = buffer_info->skb;
- dma_rmb(); /* read descriptor and rx_buffer_info after status DD */
-
- /* in the packet split case this is header only */
- prefetch(skb->data - NET_IP_ALIGN);
-
- i++;
- if (i == rx_ring->count)
- i = 0;
- next_rxd = E1000_RX_DESC_PS(*rx_ring, i);
- prefetch(next_rxd);
-
- next_buffer = &rx_ring->buffer_info[i];
-
- cleaned = true;
- cleaned_count++;
- dma_unmap_single(&pdev->dev, buffer_info->dma,
- adapter->rx_ps_bsize0, DMA_FROM_DEVICE);
- buffer_info->dma = 0;
-
- /* see !EOP comment in other Rx routine */
- if (!(staterr & E1000_RXD_STAT_EOP))
- adapter->flags2 |= FLAG2_IS_DISCARDING;
-
- if (adapter->flags2 & FLAG2_IS_DISCARDING) {
- e_dbg("Packet Split buffers didn't pick up the full packet\n");
- dev_kfree_skb_irq(skb);
- if (staterr & E1000_RXD_STAT_EOP)
- adapter->flags2 &= ~FLAG2_IS_DISCARDING;
- goto next_desc;
- }
-
- if (unlikely((staterr & E1000_RXDEXT_ERR_FRAME_ERR_MASK) &&
- !(netdev->features & NETIF_F_RXALL))) {
- dev_kfree_skb_irq(skb);
- goto next_desc;
- }
-
- length = le16_to_cpu(rx_desc->wb.middle.length0);
-
- if (!length) {
- e_dbg("Last part of the packet spanning multiple descriptors\n");
- dev_kfree_skb_irq(skb);
- goto next_desc;
- }
-
- /* Good Receive */
- skb_put(skb, length);
-
- {
- /* this looks ugly, but it seems compiler issues make
- * it more efficient than reusing j
- */
- int l1 = le16_to_cpu(rx_desc->wb.upper.length[0]);
-
- /* page alloc/put takes too long and effects small
- * packet throughput, so unsplit small packets and
- * save the alloc/put
- */
- if (l1 && (l1 <= copybreak) &&
- ((length + l1) <= adapter->rx_ps_bsize0)) {
- ps_page = &buffer_info->ps_pages[0];
-
- dma_sync_single_for_cpu(&pdev->dev,
- ps_page->dma,
- PAGE_SIZE,
- DMA_FROM_DEVICE);
- memcpy(skb_tail_pointer(skb),
- page_address(ps_page->page), l1);
- dma_sync_single_for_device(&pdev->dev,
- ps_page->dma,
- PAGE_SIZE,
- DMA_FROM_DEVICE);
-
- /* remove the CRC */
- if (!(adapter->flags2 & FLAG2_CRC_STRIPPING)) {
- if (!(netdev->features & NETIF_F_RXFCS))
- l1 -= 4;
- }
-
- skb_put(skb, l1);
- goto copydone;
- } /* if */
- }
-
- for (j = 0; j < PS_PAGE_BUFFERS; j++) {
- length = le16_to_cpu(rx_desc->wb.upper.length[j]);
- if (!length)
- break;
-
- ps_page = &buffer_info->ps_pages[j];
- dma_unmap_page(&pdev->dev, ps_page->dma, PAGE_SIZE,
- DMA_FROM_DEVICE);
- ps_page->dma = 0;
- skb_fill_page_desc(skb, j, ps_page->page, 0, length);
- ps_page->page = NULL;
- skb->len += length;
- skb->data_len += length;
- skb->truesize += PAGE_SIZE;
- }
-
- /* strip the ethernet crc, problem is we're using pages now so
- * this whole operation can get a little cpu intensive
- */
- if (!(adapter->flags2 & FLAG2_CRC_STRIPPING)) {
- if (!(netdev->features & NETIF_F_RXFCS))
- pskb_trim(skb, skb->len - 4);
- }
-
-copydone:
- total_rx_bytes += skb->len;
- total_rx_packets++;
-
- e1000_rx_checksum(adapter, staterr, skb);
-
- e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
-
- if (rx_desc->wb.upper.header_status &
- cpu_to_le16(E1000_RXDPS_HDRSTAT_HDRSP))
- adapter->rx_hdr_split++;
-
- e1000_receive_skb(adapter, netdev, skb, staterr,
- rx_desc->wb.middle.vlan);
-
-next_desc:
- rx_desc->wb.middle.status_error &= cpu_to_le32(~0xFF);
- buffer_info->skb = NULL;
-
- /* return some buffers to hardware, one at a time is too slow */
- if (cleaned_count >= E1000_RX_BUFFER_WRITE) {
- adapter->alloc_rx_buf(rx_ring, cleaned_count,
- GFP_ATOMIC);
- cleaned_count = 0;
- }
-
- /* use prefetched values */
- rx_desc = next_rxd;
- buffer_info = next_buffer;
-
- staterr = le32_to_cpu(rx_desc->wb.middle.status_error);
- }
- rx_ring->next_to_clean = i;
-
- cleaned_count = e1000_desc_unused(rx_ring);
- if (cleaned_count)
- adapter->alloc_rx_buf(rx_ring, cleaned_count, GFP_ATOMIC);
-
- adapter->total_rx_bytes += total_rx_bytes;
- adapter->total_rx_packets += total_rx_packets;
- return cleaned;
-}
-
static void e1000_consume_page(struct e1000_buffer *bi, struct sk_buff *skb,
u16 length)
{
@@ -1710,9 +1322,8 @@ static void e1000_clean_rx_ring(struct e1000_ring *rx_ring)
{
struct e1000_adapter *adapter = rx_ring->adapter;
struct e1000_buffer *buffer_info;
- struct e1000_ps_page *ps_page;
struct pci_dev *pdev = adapter->pdev;
- unsigned int i, j;
+ unsigned int i;
/* Free all the Rx ring sk_buffs */
for (i = 0; i < rx_ring->count; i++) {
@@ -1725,10 +1336,6 @@ static void e1000_clean_rx_ring(struct e1000_ring *rx_ring)
else if (adapter->clean_rx == e1000_clean_jumbo_rx_irq)
dma_unmap_page(&pdev->dev, buffer_info->dma,
PAGE_SIZE, DMA_FROM_DEVICE);
- else if (adapter->clean_rx == e1000_clean_rx_irq_ps)
- dma_unmap_single(&pdev->dev, buffer_info->dma,
- adapter->rx_ps_bsize0,
- DMA_FROM_DEVICE);
buffer_info->dma = 0;
}
@@ -1741,17 +1348,6 @@ static void e1000_clean_rx_ring(struct e1000_ring *rx_ring)
dev_kfree_skb(buffer_info->skb);
buffer_info->skb = NULL;
}
-
- for (j = 0; j < PS_PAGE_BUFFERS; j++) {
- ps_page = &buffer_info->ps_pages[j];
- if (!ps_page->page)
- break;
- dma_unmap_page(&pdev->dev, ps_page->dma, PAGE_SIZE,
- DMA_FROM_DEVICE);
- ps_page->dma = 0;
- put_page(ps_page->page);
- ps_page->page = NULL;
- }
}
/* there also may be some cached data from a chained receive */
@@ -2396,23 +1992,14 @@ int e1000e_setup_tx_resources(struct e1000_ring *tx_ring)
int e1000e_setup_rx_resources(struct e1000_ring *rx_ring)
{
struct e1000_adapter *adapter = rx_ring->adapter;
- struct e1000_buffer *buffer_info;
- int i, size, desc_len, err = -ENOMEM;
+ int size, desc_len, err = -ENOMEM;
size = sizeof(struct e1000_buffer) * rx_ring->count;
rx_ring->buffer_info = vzalloc(size);
if (!rx_ring->buffer_info)
goto err;
- for (i = 0; i < rx_ring->count; i++) {
- buffer_info = &rx_ring->buffer_info[i];
- buffer_info->ps_pages = kzalloc_objs(struct e1000_ps_page,
- PS_PAGE_BUFFERS);
- if (!buffer_info->ps_pages)
- goto err_pages;
- }
-
- desc_len = sizeof(union e1000_rx_desc_packet_split);
+ desc_len = sizeof(union e1000_rx_desc_extended);
/* Round up to nearest 4K */
rx_ring->size = rx_ring->count * desc_len;
@@ -2420,7 +2007,7 @@ int e1000e_setup_rx_resources(struct e1000_ring *rx_ring)
err = e1000_alloc_ring_dma(adapter, rx_ring);
if (err)
- goto err_pages;
+ goto err;
rx_ring->next_to_clean = 0;
rx_ring->next_to_use = 0;
@@ -2428,11 +2015,6 @@ int e1000e_setup_rx_resources(struct e1000_ring *rx_ring)
return 0;
-err_pages:
- for (i = 0; i < rx_ring->count; i++) {
- buffer_info = &rx_ring->buffer_info[i];
- kfree(buffer_info->ps_pages);
- }
err:
vfree(rx_ring->buffer_info);
e_err("Unable to allocate memory for the receive descriptor ring\n");
@@ -2496,13 +2078,9 @@ void e1000e_free_rx_resources(struct e1000_ring *rx_ring)
{
struct e1000_adapter *adapter = rx_ring->adapter;
struct pci_dev *pdev = adapter->pdev;
- int i;
e1000_clean_rx_ring(rx_ring);
- for (i = 0; i < rx_ring->count; i++)
- kfree(rx_ring->buffer_info[i].ps_pages);
-
vfree(rx_ring->buffer_info);
rx_ring->buffer_info = NULL;
@@ -3050,9 +2628,6 @@ static void e1000_configure_tx(struct e1000_adapter *adapter)
}
}
-#define PAGE_USE_COUNT(S) (((S) >> PAGE_SHIFT) + \
- (((S) & (PAGE_SIZE - 1)) ? 1 : 0))
-
/**
* e1000_setup_rctl - configure the receive control registers
* @adapter: Board private structure
@@ -3061,7 +2636,6 @@ static void e1000_setup_rctl(struct e1000_adapter *adapter)
{
struct e1000_hw *hw = &adapter->hw;
u32 rctl, rfctl;
- u32 pages = 0;
/* Workaround Si errata on PCHx - configure jumbo frame flow.
* If jumbo frames not set, program related MAC/PHY registers
@@ -3144,49 +2718,6 @@ static void e1000_setup_rctl(struct e1000_adapter *adapter)
rfctl |= E1000_RFCTL_EXTEN;
ew32(RFCTL, rfctl);
- /* 82571 and greater support packet-split where the protocol
- * header is placed in skb->data and the packet data is
- * placed in pages hanging off of skb_shinfo(skb)->nr_frags.
- * In the case of a non-split, skb->data is linearly filled,
- * followed by the page buffers. Therefore, skb->data is
- * sized to hold the largest protocol header.
- *
- * allocations using alloc_page take too long for regular MTU
- * so only enable packet split for jumbo frames
- *
- * Using pages when the page size is greater than 16k wastes
- * a lot of memory, since we allocate 3 pages at all times
- * per packet.
- */
- pages = PAGE_USE_COUNT(adapter->netdev->mtu);
- if ((pages <= 3) && (PAGE_SIZE <= 16384) && (rctl & E1000_RCTL_LPE))
- adapter->rx_ps_pages = pages;
- else
- adapter->rx_ps_pages = 0;
-
- if (adapter->rx_ps_pages) {
- u32 psrctl = 0;
-
- /* Enable Packet split descriptors */
- rctl |= E1000_RCTL_DTYP_PS;
-
- psrctl |= adapter->rx_ps_bsize0 >> E1000_PSRCTL_BSIZE0_SHIFT;
-
- switch (adapter->rx_ps_pages) {
- case 3:
- psrctl |= PAGE_SIZE << E1000_PSRCTL_BSIZE3_SHIFT;
- fallthrough;
- case 2:
- psrctl |= PAGE_SIZE << E1000_PSRCTL_BSIZE2_SHIFT;
- fallthrough;
- case 1:
- psrctl |= PAGE_SIZE >> E1000_PSRCTL_BSIZE1_SHIFT;
- break;
- }
-
- ew32(PSRCTL, psrctl);
- }
-
/* This is useful for sniffing bad packets. */
if (adapter->netdev->features & NETIF_F_RXALL) {
/* UPE and MPE will be handled by normal PROMISC logic
@@ -3222,18 +2753,11 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
u64 rdba;
u32 rdlen, rctl, rxcsum, ctrl_ext;
- if (adapter->rx_ps_pages) {
- /* this is a 32 byte descriptor */
- rdlen = rx_ring->count *
- sizeof(union e1000_rx_desc_packet_split);
- adapter->clean_rx = e1000_clean_rx_irq_ps;
- adapter->alloc_rx_buf = e1000_alloc_rx_buffers_ps;
- } else if (adapter->netdev->mtu > ETH_FRAME_LEN + ETH_FCS_LEN) {
- rdlen = rx_ring->count * sizeof(union e1000_rx_desc_extended);
+ rdlen = rx_ring->count * sizeof(union e1000_rx_desc_extended);
+ if (adapter->netdev->mtu > ETH_FRAME_LEN + ETH_FCS_LEN) {
adapter->clean_rx = e1000_clean_jumbo_rx_irq;
adapter->alloc_rx_buf = e1000_alloc_jumbo_rx_buffers;
} else {
- rdlen = rx_ring->count * sizeof(union e1000_rx_desc_extended);
adapter->clean_rx = e1000_clean_rx_irq;
adapter->alloc_rx_buf = e1000_alloc_rx_buffers;
}
@@ -4492,7 +4016,6 @@ static int e1000_sw_init(struct e1000_adapter *adapter)
struct net_device *netdev = adapter->netdev;
adapter->rx_buffer_len = VLAN_ETH_FRAME_LEN + ETH_FCS_LEN;
- adapter->rx_ps_bsize0 = 128;
adapter->max_frame_size = netdev->mtu + VLAN_ETH_HLEN + ETH_FCS_LEN;
adapter->min_frame_size = ETH_ZLEN + ETH_FCS_LEN;
adapter->tx_ring_count = E1000_DEFAULT_TXD;
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH iwl-next 5/8] e1000e: always use jumbo Rx path
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
` (3 preceding siblings ...)
2026-08-30 23:21 ` [PATCH iwl-next 4/8] e1000e: remove packet-split Rx path Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 6/8] e1000e: disable NAPI while interface is down Matt Vollrath
` (2 subsequent siblings)
7 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath
Converge on the Rx path which can support any combination of page size
and MTU.
Rename the jumbo path's clean and allocate functions to the standard
path's names. Also, drop the unused return value from the clean
function.
Factor out the function pointers for cleaning and allocation.
Remove the discarding flag because we no longer desire to discard frames
that don't fit in a single descriptor.
Remove rx_buffer_len clamping because buffers are now whole pages.
Additional cost for standard MTU operation will be addressed by the
proceeding page pool conversion.
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/e1000.h | 7 +-
drivers/net/ethernet/intel/e1000e/netdev.c | 311 ++-------------------
2 files changed, 18 insertions(+), 300 deletions(-)
diff --git a/drivers/net/ethernet/intel/e1000e/e1000.h b/drivers/net/ethernet/intel/e1000e/e1000.h
index 533d1981eb5e..e0e4b72c13f3 100644
--- a/drivers/net/ethernet/intel/e1000e/e1000.h
+++ b/drivers/net/ethernet/intel/e1000e/e1000.h
@@ -244,11 +244,7 @@ struct e1000_adapter {
u32 tx_hwtstamp_skipped;
/* Rx */
- bool (*clean_rx)(struct e1000_ring *ring, int *work_done,
- int work_to_do) ____cacheline_aligned_in_smp;
- void (*alloc_rx_buf)(struct e1000_ring *ring, int cleaned_count,
- gfp_t gfp);
- struct e1000_ring *rx_ring;
+ struct e1000_ring *rx_ring ____cacheline_aligned_in_smp;
u32 rx_int_delay;
u32 rx_abs_int_delay;
@@ -436,7 +432,6 @@ s32 e1000e_get_base_timinca(struct e1000_adapter *adapter, u32 *timinca);
#define FLAG2_CRC_STRIPPING BIT(0)
#define FLAG2_HAS_PHY_WAKEUP BIT(1)
-#define FLAG2_IS_DISCARDING BIT(2)
#define FLAG2_DISABLE_ASPM_L1 BIT(3)
#define FLAG2_HAS_PHY_STATS BIT(4)
#define FLAG2_HAS_EEE BIT(5)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 4dc3eeb01329..194166531bc8 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -369,29 +369,11 @@ static void e1000e_dump(struct e1000_adapter *adapter)
(unsigned long long)buffer_info->dma,
buffer_info->skb, next_desc);
- /* Jumbo buffers land in the page; a cleaned
- * jumbo slot keeps only its small shell skb
- * until it is refilled, so only dump an skb
- * that can hold a whole buffer.
- */
- if (netif_msg_pktdata(adapter) &&
- buffer_info->page)
+ if (netif_msg_pktdata(adapter) && buffer_info->page)
print_hex_dump(KERN_INFO, "",
- DUMP_PREFIX_ADDRESS, 16,
- 1,
+ DUMP_PREFIX_ADDRESS, 16, 1,
page_address(buffer_info->page),
- adapter->rx_buffer_len,
- true);
- else if (netif_msg_pktdata(adapter) &&
- buffer_info->skb &&
- skb_tailroom(buffer_info->skb) >=
- adapter->rx_buffer_len)
- print_hex_dump(KERN_INFO, "",
- DUMP_PREFIX_ADDRESS, 16,
- 1,
- buffer_info->skb->data,
- adapter->rx_buffer_len,
- true);
+ adapter->rx_buffer_len, true);
}
}
}
@@ -571,82 +553,11 @@ static void e1000e_update_tdt_wa(struct e1000_ring *tx_ring, unsigned int i)
/**
* e1000_alloc_rx_buffers - Replace used receive buffers
* @rx_ring: Rx descriptor ring
- * @cleaned_count: number to reallocate
+ * @cleaned_count: number of buffers to allocate this pass
* @gfp: flags for allocation
**/
static void e1000_alloc_rx_buffers(struct e1000_ring *rx_ring,
int cleaned_count, gfp_t gfp)
-{
- struct e1000_adapter *adapter = rx_ring->adapter;
- struct net_device *netdev = adapter->netdev;
- struct pci_dev *pdev = adapter->pdev;
- union e1000_rx_desc_extended *rx_desc;
- struct e1000_buffer *buffer_info;
- struct sk_buff *skb;
- unsigned int i;
- unsigned int bufsz = adapter->rx_buffer_len;
-
- i = rx_ring->next_to_use;
- buffer_info = &rx_ring->buffer_info[i];
-
- while (cleaned_count--) {
- skb = buffer_info->skb;
- if (skb) {
- skb_trim(skb, 0);
- goto map_skb;
- }
-
- skb = __netdev_alloc_skb_ip_align(netdev, bufsz, gfp);
- if (!skb) {
- /* Better luck next round */
- adapter->alloc_rx_buff_failed++;
- break;
- }
-
- buffer_info->skb = skb;
-map_skb:
- buffer_info->dma = dma_map_single(&pdev->dev, skb->data,
- adapter->rx_buffer_len,
- DMA_FROM_DEVICE);
- if (dma_mapping_error(&pdev->dev, buffer_info->dma)) {
- dev_err(&pdev->dev, "Rx DMA map failed\n");
- adapter->rx_dma_failed++;
- break;
- }
-
- rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
- rx_desc->read.buffer_addr = cpu_to_le64(buffer_info->dma);
-
- if (unlikely(!(i & (E1000_RX_BUFFER_WRITE - 1)))) {
- /* Force memory writes to complete before letting h/w
- * know there are new descriptors to fetch. (Only
- * applicable for weak-ordered memory model archs,
- * such as IA-64).
- */
- wmb();
- if (adapter->flags2 & FLAG2_PCIM2PCI_ARBITER_WA)
- e1000e_update_rdt_wa(rx_ring, i);
- else
- writel(i, rx_ring->tail);
- }
- i++;
- if (i == rx_ring->count)
- i = 0;
- buffer_info = &rx_ring->buffer_info[i];
- }
-
- rx_ring->next_to_use = i;
-}
-
-/**
- * e1000_alloc_jumbo_rx_buffers - Replace used jumbo receive buffers
- * @rx_ring: Rx descriptor ring
- * @cleaned_count: number of buffers to allocate this pass
- * @gfp: flags for allocation
- **/
-
-static void e1000_alloc_jumbo_rx_buffers(struct e1000_ring *rx_ring,
- int cleaned_count, gfp_t gfp)
{
struct e1000_adapter *adapter = rx_ring->adapter;
struct net_device *netdev = adapter->netdev;
@@ -729,163 +640,6 @@ static inline void e1000_rx_hash(struct net_device *netdev, __le32 rss,
skb_set_hash(skb, le32_to_cpu(rss), PKT_HASH_TYPE_L3);
}
-/**
- * e1000_clean_rx_irq - Send received data up the network stack
- * @rx_ring: Rx descriptor ring
- * @work_done: output parameter for indicating completed work
- * @work_to_do: how many packets we can clean
- *
- * the return value indicates whether actual cleaning was done, there
- * is no guarantee that everything was cleaned
- **/
-static bool e1000_clean_rx_irq(struct e1000_ring *rx_ring, int *work_done,
- int work_to_do)
-{
- struct e1000_adapter *adapter = rx_ring->adapter;
- struct net_device *netdev = adapter->netdev;
- struct pci_dev *pdev = adapter->pdev;
- struct e1000_hw *hw = &adapter->hw;
- union e1000_rx_desc_extended *rx_desc, *next_rxd;
- struct e1000_buffer *buffer_info, *next_buffer;
- u32 length, staterr;
- unsigned int i;
- int cleaned_count = 0;
- bool cleaned = false;
- unsigned int total_rx_bytes = 0, total_rx_packets = 0;
-
- i = rx_ring->next_to_clean;
- rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
- staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
- buffer_info = &rx_ring->buffer_info[i];
-
- while (staterr & E1000_RXD_STAT_DD) {
- struct sk_buff *skb;
-
- if (*work_done >= work_to_do)
- break;
- (*work_done)++;
- dma_rmb(); /* read descriptor and rx_buffer_info after status DD */
-
- skb = buffer_info->skb;
- buffer_info->skb = NULL;
-
- prefetch(skb->data - NET_IP_ALIGN);
-
- i++;
- if (i == rx_ring->count)
- i = 0;
- next_rxd = E1000_RX_DESC_EXT(*rx_ring, i);
- prefetch(next_rxd);
-
- next_buffer = &rx_ring->buffer_info[i];
-
- cleaned = true;
- cleaned_count++;
- dma_unmap_single(&pdev->dev, buffer_info->dma,
- adapter->rx_buffer_len, DMA_FROM_DEVICE);
- buffer_info->dma = 0;
-
- length = le16_to_cpu(rx_desc->wb.upper.length);
-
- /* !EOP means multiple descriptors were used to store a single
- * packet, if that's the case we need to toss it. In fact, we
- * need to toss every packet with the EOP bit clear and the
- * next frame that _does_ have the EOP bit set, as it is by
- * definition only a frame fragment
- */
- if (unlikely(!(staterr & E1000_RXD_STAT_EOP)))
- adapter->flags2 |= FLAG2_IS_DISCARDING;
-
- if (adapter->flags2 & FLAG2_IS_DISCARDING) {
- /* All receives must fit into a single buffer */
- e_dbg("Receive packet consumed multiple buffers\n");
- /* recycle */
- buffer_info->skb = skb;
- if (staterr & E1000_RXD_STAT_EOP)
- adapter->flags2 &= ~FLAG2_IS_DISCARDING;
- goto next_desc;
- }
-
- if (unlikely((staterr & E1000_RXDEXT_ERR_FRAME_ERR_MASK) &&
- !(netdev->features & NETIF_F_RXALL))) {
- /* recycle */
- buffer_info->skb = skb;
- goto next_desc;
- }
-
- /* adjust length to remove Ethernet CRC */
- if (!(adapter->flags2 & FLAG2_CRC_STRIPPING)) {
- /* If configured to store CRC, don't subtract FCS,
- * but keep the FCS bytes out of the total_rx_bytes
- * counter
- */
- if (netdev->features & NETIF_F_RXFCS)
- total_rx_bytes -= 4;
- else
- length -= 4;
- }
-
- total_rx_bytes += length;
- total_rx_packets++;
-
- /* code added for copybreak, this should improve
- * performance for small packets with large amounts
- * of reassembly being done in the stack
- */
- if (length < copybreak) {
- struct sk_buff *new_skb =
- napi_alloc_skb(&adapter->napi, length);
- if (new_skb) {
- skb_copy_to_linear_data_offset(new_skb,
- -NET_IP_ALIGN,
- (skb->data -
- NET_IP_ALIGN),
- (length +
- NET_IP_ALIGN));
- /* save the skb in buffer_info as good */
- buffer_info->skb = skb;
- skb = new_skb;
- }
- /* else just continue with the old one */
- }
- /* end copybreak code */
- skb_put(skb, length);
-
- /* Receive Checksum Offload */
- e1000_rx_checksum(adapter, staterr, skb);
-
- e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
-
- e1000_receive_skb(adapter, netdev, skb, staterr,
- rx_desc->wb.upper.vlan);
-
-next_desc:
- rx_desc->wb.upper.status_error &= cpu_to_le32(~0xFF);
-
- /* return some buffers to hardware, one at a time is too slow */
- if (cleaned_count >= E1000_RX_BUFFER_WRITE) {
- adapter->alloc_rx_buf(rx_ring, cleaned_count,
- GFP_ATOMIC);
- cleaned_count = 0;
- }
-
- /* use prefetched values */
- rx_desc = next_rxd;
- buffer_info = next_buffer;
-
- staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
- }
- rx_ring->next_to_clean = i;
-
- cleaned_count = e1000_desc_unused(rx_ring);
- if (cleaned_count)
- adapter->alloc_rx_buf(rx_ring, cleaned_count, GFP_ATOMIC);
-
- adapter->total_rx_bytes += total_rx_bytes;
- adapter->total_rx_packets += total_rx_packets;
- return cleaned;
-}
-
static void e1000_put_txbuf(struct e1000_ring *tx_ring,
struct e1000_buffer *buffer_info,
bool drop)
@@ -1135,16 +889,13 @@ static void e1000_consume_page(struct e1000_buffer *bi, struct sk_buff *skb,
}
/**
- * e1000_clean_jumbo_rx_irq - Send received data up the network stack; legacy
+ * e1000_clean_rx_irq - Send received data up the network stack
* @rx_ring: Rx descriptor ring
* @work_done: output parameter for indicating completed work
* @work_to_do: how many packets we can clean
- *
- * the return value indicates whether actual cleaning was done, there
- * is no guarantee that everything was cleaned
**/
-static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
- int work_to_do)
+static void e1000_clean_rx_irq(struct e1000_ring *rx_ring, int *work_done,
+ int work_to_do)
{
struct e1000_adapter *adapter = rx_ring->adapter;
struct net_device *netdev = adapter->netdev;
@@ -1154,7 +905,6 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
u32 length, staterr;
unsigned int i;
int cleaned_count = 0;
- bool cleaned = false;
unsigned int total_rx_bytes = 0, total_rx_packets = 0;
struct skb_shared_info *shinfo;
@@ -1182,7 +932,6 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
next_buffer = &rx_ring->buffer_info[i];
- cleaned = true;
cleaned_count++;
dma_unmap_page(&pdev->dev, buffer_info->dma, PAGE_SIZE,
DMA_FROM_DEVICE);
@@ -1292,8 +1041,8 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
/* return some buffers to hardware, one at a time is too slow */
if (unlikely(cleaned_count >= E1000_RX_BUFFER_WRITE)) {
- adapter->alloc_rx_buf(rx_ring, cleaned_count,
- GFP_ATOMIC);
+ e1000_alloc_rx_buffers(rx_ring, cleaned_count,
+ GFP_ATOMIC);
cleaned_count = 0;
}
@@ -1307,11 +1056,10 @@ static bool e1000_clean_jumbo_rx_irq(struct e1000_ring *rx_ring, int *work_done,
cleaned_count = e1000_desc_unused(rx_ring);
if (cleaned_count)
- adapter->alloc_rx_buf(rx_ring, cleaned_count, GFP_ATOMIC);
+ e1000_alloc_rx_buffers(rx_ring, cleaned_count, GFP_ATOMIC);
adapter->total_rx_bytes += total_rx_bytes;
adapter->total_rx_packets += total_rx_packets;
- return cleaned;
}
/**
@@ -1329,13 +1077,8 @@ static void e1000_clean_rx_ring(struct e1000_ring *rx_ring)
for (i = 0; i < rx_ring->count; i++) {
buffer_info = &rx_ring->buffer_info[i];
if (buffer_info->dma) {
- if (adapter->clean_rx == e1000_clean_rx_irq)
- dma_unmap_single(&pdev->dev, buffer_info->dma,
- adapter->rx_buffer_len,
- DMA_FROM_DEVICE);
- else if (adapter->clean_rx == e1000_clean_jumbo_rx_irq)
- dma_unmap_page(&pdev->dev, buffer_info->dma,
- PAGE_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_page(&pdev->dev, buffer_info->dma,
+ PAGE_SIZE, DMA_FROM_DEVICE);
buffer_info->dma = 0;
}
@@ -1361,7 +1104,6 @@ static void e1000_clean_rx_ring(struct e1000_ring *rx_ring)
rx_ring->next_to_clean = 0;
rx_ring->next_to_use = 0;
- adapter->flags2 &= ~FLAG2_IS_DISCARDING;
}
static void e1000e_downshift_workaround(struct work_struct *work)
@@ -2282,7 +2024,7 @@ static int e1000e_poll(struct napi_struct *napi, int budget)
(adapter->rx_ring->ims_val & adapter->tx_ring->ims_val))
tx_cleaned = e1000_clean_tx_irq(adapter->tx_ring);
- adapter->clean_rx(adapter->rx_ring, &work_done, budget);
+ e1000_clean_rx_irq(adapter->rx_ring, &work_done, budget);
if (!tx_cleaned || work_done == budget)
return budget;
@@ -2754,13 +2496,6 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
u32 rdlen, rctl, rxcsum, ctrl_ext;
rdlen = rx_ring->count * sizeof(union e1000_rx_desc_extended);
- if (adapter->netdev->mtu > ETH_FRAME_LEN + ETH_FCS_LEN) {
- adapter->clean_rx = e1000_clean_jumbo_rx_irq;
- adapter->alloc_rx_buf = e1000_alloc_jumbo_rx_buffers;
- } else {
- adapter->clean_rx = e1000_clean_rx_irq;
- adapter->alloc_rx_buf = e1000_alloc_rx_buffers;
- }
/* disable receives while setting up the descriptors */
rctl = er32(RCTL);
@@ -3327,7 +3062,7 @@ static void e1000_configure(struct e1000_adapter *adapter)
e1000e_setup_rss_hash(adapter);
e1000_setup_rctl(adapter);
e1000_configure_rx(adapter);
- adapter->alloc_rx_buf(rx_ring, e1000_desc_unused(rx_ring), GFP_KERNEL);
+ e1000_alloc_rx_buffers(rx_ring, e1000_desc_unused(rx_ring), GFP_KERNEL);
}
/**
@@ -4015,7 +3750,7 @@ static int e1000_sw_init(struct e1000_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
- adapter->rx_buffer_len = VLAN_ETH_FRAME_LEN + ETH_FCS_LEN;
+ adapter->rx_buffer_len = 2048;
adapter->max_frame_size = netdev->mtu + VLAN_ETH_HLEN + ETH_FCS_LEN;
adapter->min_frame_size = ETH_ZLEN + ETH_FCS_LEN;
adapter->tx_ring_count = E1000_DEFAULT_TXD;
@@ -4227,8 +3962,8 @@ int e1000e_open(struct net_device *netdev)
/* before we allocate an interrupt, we must be ready to handle it.
* Setting DEBUG_SHIRQ in the kernel makes it fire an interrupt
- * as soon as we call pci_request_irq, so we have to setup our
- * clean_rx handler before we do so.
+ * as soon as we call pci_request_irq, so we have to configure the
+ * Rx ring before we do so.
*/
e1000_configure(adapter);
@@ -5647,23 +5382,11 @@ static int e1000_change_mtu(struct net_device *netdev, int new_mtu)
if (netif_running(netdev))
e1000e_down(adapter, true);
- /* NOTE: netdev_alloc_skb reserves 16 bytes, and typically NET_IP_ALIGN
- * means we reserve 2 more, this pushes us to allocate from the next
- * larger slab size.
- * i.e. RXBUFFER_2048 --> size-4096 slab
- * However with the new *_jumbo_rx* routines, jumbo receives will use
- * fragmented skbs
- */
-
if (max_frame <= 2048)
adapter->rx_buffer_len = 2048;
else
adapter->rx_buffer_len = 4096;
- /* adjust allocation if LPE protects us, and we aren't using SBP */
- if (max_frame <= (VLAN_ETH_FRAME_LEN + ETH_FCS_LEN))
- adapter->rx_buffer_len = VLAN_ETH_FRAME_LEN + ETH_FCS_LEN;
-
if (netif_running(netdev))
e1000e_up(adapter);
else
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH iwl-next 6/8] e1000e: disable NAPI while interface is down
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
` (4 preceding siblings ...)
2026-08-30 23:21 ` [PATCH iwl-next 5/8] e1000e: always use jumbo " Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-09-03 10:27 ` Simon Horman
2026-08-30 23:21 ` [PATCH iwl-next 7/8] e1000e: use libeth page_pool for Rx Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 8/8] e1000e: return skbs to NAPI cache Matt Vollrath
7 siblings, 1 reply; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath
The upcoming page pool conversion requires that NAPI is disabled during
reconfiguration windows. Otherwise pool destruction would fail the
assertion that NAPI will not race, causing a benign but loud warning.
Move napi_disable into e1000e_down and add a napi_enable to e1000e_up.
This follows convention of all other Intel drivers.
Because napi_disable would hang if called twice, all callers of
e1000e_down must now be serialized. As of this patch, I'm convinced this
is true.
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/netdev.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 194166531bc8..6d846cacb026 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3527,6 +3527,8 @@ void e1000e_up(struct e1000_adapter *adapter)
/* hardware has been reset, we need to reload some things */
e1000_configure(adapter);
+ napi_enable(&adapter->napi);
+
clear_bit(__E1000_DOWN, &adapter->state);
if (adapter->msix_entries)
@@ -3601,7 +3603,7 @@ void e1000e_down(struct e1000_adapter *adapter, bool reset)
e1000_irq_disable(adapter);
- napi_synchronize(&adapter->napi);
+ napi_disable(&adapter->napi);
timer_delete_sync(&adapter->watchdog_timer);
timer_delete_sync(&adapter->phy_info_timer);
@@ -4055,7 +4057,6 @@ int e1000e_close(struct net_device *netdev)
netif_queue_set_napi(netdev, 0, NETDEV_QUEUE_TYPE_RX, NULL);
netif_queue_set_napi(netdev, 0, NETDEV_QUEUE_TYPE_TX, NULL);
- napi_disable(&adapter->napi);
e1000e_free_tx_resources(adapter->tx_ring);
e1000e_free_rx_resources(adapter->rx_ring);
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH iwl-next 6/8] e1000e: disable NAPI while interface is down
2026-08-30 23:21 ` [PATCH iwl-next 6/8] e1000e: disable NAPI while interface is down Matt Vollrath
@ 2026-09-03 10:27 ` Simon Horman
2026-09-03 15:43 ` Matt Vollrath
0 siblings, 1 reply; 14+ messages in thread
From: Simon Horman @ 2026-09-03 10:27 UTC (permalink / raw)
To: tactii
Cc: Simon Horman, intel-wired-lan, netdev, anthony.l.nguyen,
przemyslaw.kitszel, aleksander.lobakin, andrew+netdev, davem,
edumazet, kuba, pabeni, corbet, skhan, linux-doc, linux-kernel
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev/#/patchset/20260830232146.36948-2-tactii@gmail.com?part=6
---
e1000e: disable NAPI while interface is down
This commit moves napi_disable into e1000e_down and adds napi_enable to
e1000e_up to ensure NAPI is disabled during reconfiguration windows.
> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
> index 194166531bc84..6d846cacb026e 100644
> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
[ ... ]
> @@ -3601,7 +3603,7 @@ void e1000e_down(struct e1000_adapter *adapter, bool reset)
>
> e1000_irq_disable(adapter);
>
> - napi_synchronize(&adapter->napi);
> + napi_disable(&adapter->napi);
[Severity: High]
Does this change introduce an infinite hang in napi_disable() due to
concurrent or duplicate e1000e_down() calls?
A runtime PM suspend callback (e1000e_pm_runtime_suspend) can execute
concurrently with an ethtool operation like e1000_set_ringparam. The ethtool
operation sets the __E1000_RESETTING bit and holds it while allocating
resources.
The PM suspend callback waits on this bit but uses a bounded timeout (300ms).
If the ethtool operation takes longer than 300ms, the PM callback times out,
prints a WARN_ON, and unconditionally calls e1000e_down():
e1000e_pm_runtime_suspend() {
while (test_bit(__E1000_RESETTING, &adapter->state) && count--)
usleep_range(10000, 11000);
WARN_ON(test_bit(__E1000_RESETTING, &adapter->state));
/* Down the device without resetting the hardware */
e1000e_down(adapter, false);
}
Because the ethtool operation also called (or is calling) e1000e_down(),
napi_disable() is executed either concurrently or sequentially on an
already-disabled NAPI struct, leading to an infinite hang waiting for
NAPI_STATE_SCHED.
[Severity: High]
Could this also lead to a sequential double napi_disable() hang due to state
mismatches where e1000e_down() is called but netif_running() remains true?
The e1000e_pm_thaw() function attempts to bring the device back up after a
freeze/suspend. If e1000_request_irq() fails, it exits via the err_irq label
without calling e1000e_up(), leaving NAPI disabled:
e1000e_pm_thaw() {
rc = e1000_request_irq(adapter);
if (rc)
goto err_irq;
e1000e_up(adapter);
}
However, the device's IFF_UP flag is untouched, so netif_running() remains
true. Subsequently, if an administrator changes the MTU via
e1000_change_mtu(), the driver checks netif_running() and blindly calls
e1000e_down():
e1000_change_mtu() {
if (netif_running(netdev))
e1000e_down(adapter, true);
}
This invokes napi_disable() a second time sequentially, which hangs
indefinitely because the NAPI instance was never re-enabled.
>
> timer_delete_sync(&adapter->watchdog_timer);
> timer_delete_sync(&adapter->phy_info_timer);
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [PATCH iwl-next 6/8] e1000e: disable NAPI while interface is down
2026-09-03 10:27 ` Simon Horman
@ 2026-09-03 15:43 ` Matt Vollrath
0 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-09-03 15:43 UTC (permalink / raw)
To: Simon Horman
Cc: intel-wired-lan, netdev, anthony.l.nguyen, przemyslaw.kitszel,
aleksander.lobakin, andrew+netdev, davem, edumazet, kuba, pabeni,
corbet, skhan, linux-doc, linux-kernel
On 9/3/26 06:27, Simon Horman wrote:
> This is an AI-generated review of your patch. The human sending this
> email has considered the AI review valid, or at least plausible.
> Full review at: https://sashiko.dev/#/patchset/20260830232146.36948-2-tactii@gmail.com?part=6
> ---
> e1000e: disable NAPI while interface is down
>
> This commit moves napi_disable into e1000e_down and adds napi_enable to
> e1000e_up to ensure NAPI is disabled during reconfiguration windows.
>
>> diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
>> index 194166531bc84..6d846cacb026e 100644
>> --- a/drivers/net/ethernet/intel/e1000e/netdev.c
>> +++ b/drivers/net/ethernet/intel/e1000e/netdev.c
> [ ... ]
>> @@ -3601,7 +3603,7 @@ void e1000e_down(struct e1000_adapter *adapter, bool reset)
>>
>> e1000_irq_disable(adapter);
>>
>> - napi_synchronize(&adapter->napi);
>> + napi_disable(&adapter->napi);
>
> [Severity: High]
> Does this change introduce an infinite hang in napi_disable() due to
> concurrent or duplicate e1000e_down() calls?
>
> A runtime PM suspend callback (e1000e_pm_runtime_suspend) can execute
> concurrently with an ethtool operation like e1000_set_ringparam. The ethtool
> operation sets the __E1000_RESETTING bit and holds it while allocating
> resources.
No, ethtool and runtime PM suspend are synchronized:
* ethtool core calls pm_runtime_get_sync() before any operation. This
increments dev->power.usage_count and waits for any ongoing suspend to
finish.
* PM core rpm_suspend exits without retry (rpm_check_suspend_allowed()).
>
> The PM suspend callback waits on this bit but uses a bounded timeout (300ms).
> If the ethtool operation takes longer than 300ms, the PM callback times out,
> prints a WARN_ON, and unconditionally calls e1000e_down():
>
> e1000e_pm_runtime_suspend() {
> while (test_bit(__E1000_RESETTING, &adapter->state) && count--)
> usleep_range(10000, 11000);
>
> WARN_ON(test_bit(__E1000_RESETTING, &adapter->state));
>
> /* Down the device without resetting the hardware */
> e1000e_down(adapter, false);
> }
>
> Because the ethtool operation also called (or is calling) e1000e_down(),
> napi_disable() is executed either concurrently or sequentially on an
> already-disabled NAPI struct, leading to an infinite hang waiting for
> NAPI_STATE_SCHED.
>
> [Severity: High]
> Could this also lead to a sequential double napi_disable() hang due to state
> mismatches where e1000e_down() is called but netif_running() remains true?
>
> The e1000e_pm_thaw() function attempts to bring the device back up after a
> freeze/suspend. If e1000_request_irq() fails, it exits via the err_irq label
> without calling e1000e_up(), leaving NAPI disabled:
>
> e1000e_pm_thaw() {
> rc = e1000_request_irq(adapter);
> if (rc)
> goto err_irq;
>
> e1000e_up(adapter);
> }
>
> However, the device's IFF_UP flag is untouched, so netif_running() remains
> true. Subsequently, if an administrator changes the MTU via
> e1000_change_mtu(), the driver checks netif_running() and blindly calls
> e1000e_down():
>
> e1000_change_mtu() {
> if (netif_running(netdev))
> e1000e_down(adapter, true);
> }
Core does not call ndo_change_mtu on detached devices, it checks
netif_device_present() in the typical path. Bonding and team paths will
close the device before changing MTU.
Same netif_device_present() check in ethtool core and e1000e_pm_freeze().
However, the PM runtime suspend and resume ops only check IFF_UP and not
netif_device_present(). PM core does not check this or gate it on a known
thaw failure (by design). That is a real pre-existing bug made more
consequential by this change.
>
> This invokes napi_disable() a second time sequentially, which hangs
> indefinitely because the NAPI instance was never re-enabled.
>
>>
>> timer_delete_sync(&adapter->watchdog_timer);
>> timer_delete_sync(&adapter->phy_info_timer);
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH iwl-next 7/8] e1000e: use libeth page_pool for Rx
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
` (5 preceding siblings ...)
2026-08-30 23:21 ` [PATCH iwl-next 6/8] e1000e: disable NAPI while interface is down Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
2026-08-30 23:21 ` [PATCH iwl-next 8/8] e1000e: return skbs to NAPI cache Matt Vollrath
7 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath
Convert this driver's Rx to use a page pool managed by libeth. This
change potentially eliminates most DMA map/unmap operations from the Rx
hot path, which was most of the overhead with IOMMU enabled.
The previous standard path's behavior of publishing descriptors back to
the hardware every 16 iterations is retained because it was the safest
choice.
libeth is pulled in as a dependency because it is a convenient wrapper
around pool creation, destruction, and geometry.
The copybreak feature has been removed because it no longer consistently
avoids the cost of DMA remapping (which was broken in this driver).
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Assisted-by: Claude:claude-5-fable
---
.../device_drivers/ethernet/intel/e1000e.rst | 15 -
drivers/net/ethernet/intel/Kconfig | 1 +
drivers/net/ethernet/intel/e1000e/e1000.h | 32 +-
drivers/net/ethernet/intel/e1000e/netdev.c | 567 ++++++++++--------
drivers/net/ethernet/intel/e1000e/param.c | 6 -
5 files changed, 332 insertions(+), 289 deletions(-)
diff --git a/Documentation/networking/device_drivers/ethernet/intel/e1000e.rst b/Documentation/networking/device_drivers/ethernet/intel/e1000e.rst
index d8f810afdd49..fab6ebc0cf95 100644
--- a/Documentation/networking/device_drivers/ethernet/intel/e1000e.rst
+++ b/Documentation/networking/device_drivers/ethernet/intel/e1000e.rst
@@ -163,21 +163,6 @@ It ensures that an interrupt is generated after the initial Packet is sent on
the wire within the set amount of time. Proper tuning, along with TxIntDelay,
may improve traffic throughput in specific network conditions.
-copybreak
----------
-:Valid Range: 0-xxxxxxx (0=off)
-:Default Value: 256
-
-The driver copies all packets below or equaling this size to a fresh receive
-buffer before handing it up the stack.
-This parameter differs from other parameters because it is a single (not 1,1,1
-etc.) parameter applied to all driver instances and it is also available
-during runtime at /sys/module/e1000e/parameters/copybreak.
-
-To use copybreak, type::
-
- modprobe e1000e.ko copybreak=128
-
SmartPowerDownEnable
--------------------
:Valid Range: 0,1
diff --git a/drivers/net/ethernet/intel/Kconfig b/drivers/net/ethernet/intel/Kconfig
index 780f113986ea..86fddcdabda3 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -63,6 +63,7 @@ config E1000E
depends on PCI && (!SPARC32 || BROKEN)
depends on PTP_1588_CLOCK_OPTIONAL
select CRC32
+ select LIBETH
help
This driver supports the PCI-Express Intel(R) PRO/1000 gigabit
ethernet family of adapters. For PCI or PCI-X e1000 adapters,
diff --git a/drivers/net/ethernet/intel/e1000e/e1000.h b/drivers/net/ethernet/intel/e1000e/e1000.h
index e0e4b72c13f3..f8e9f3fecf3e 100644
--- a/drivers/net/ethernet/intel/e1000e/e1000.h
+++ b/drivers/net/ethernet/intel/e1000e/e1000.h
@@ -23,6 +23,7 @@
#include <linux/mdio.h>
#include <linux/mutex.h>
#include <linux/pm_qos.h>
+#include <net/libeth/rx.h>
#include "hw.h"
struct e1000_info;
@@ -121,25 +122,18 @@ enum e1000_boards {
board_pch_ptp
};
-/* wrappers around a pointer to a socket buffer,
+/* wrapper around a pointer to a Tx socket buffer,
* so a DMA handle can be stored along with the buffer
*/
struct e1000_buffer {
dma_addr_t dma;
struct sk_buff *skb;
- union {
- /* Tx */
- struct {
- unsigned long time_stamp;
- u16 length;
- u16 next_to_watch;
- unsigned int segs;
- unsigned int bytecount;
- u16 mapped_as_page;
- };
- /* Rx */
- struct page *page;
- };
+ unsigned long time_stamp;
+ u16 length;
+ u16 next_to_watch;
+ unsigned int segs;
+ unsigned int bytecount;
+ u16 mapped_as_page;
};
struct e1000_ring {
@@ -158,6 +152,13 @@ struct e1000_ring {
/* array of buffer information structs */
struct e1000_buffer *buffer_info;
+ /* libeth fill queue backing the Rx path */
+ struct page_pool *pp;
+ struct libeth_fqe *rx_fqes;
+ u32 rx_truesize;
+ u32 rx_buf_len;
+ u32 rx_fq_mtu; /* MTU the fill queue was sized for */
+
char name[IFNAMSIZ + 5];
u32 ims_val;
u32 itr_val;
@@ -192,7 +193,6 @@ struct e1000_adapter {
unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
u32 bd_number;
- u32 rx_buffer_len;
u16 mng_vlan_id;
u16 link_speed;
u16 link_duplex;
@@ -491,8 +491,6 @@ void e1000e_get_hw_control(struct e1000_adapter *adapter);
void e1000e_release_hw_control(struct e1000_adapter *adapter);
void e1000e_write_itr(struct e1000_adapter *adapter, u32 itr);
-extern unsigned int copybreak;
-
extern const struct e1000_info e1000_82571_info;
extern const struct e1000_info e1000_82572_info;
extern const struct e1000_info e1000_82573_info;
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 6d846cacb026..c906175ad38d 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -201,7 +201,7 @@ static void e1000e_dump(struct e1000_adapter *adapter)
struct e1000_buffer *buffer_info;
struct e1000_ring *rx_ring = adapter->rx_ring;
union e1000_rx_desc_extended *rx_desc;
- u32 staterr;
+ u32 staterr, hr;
int i = 0;
if (!netif_msg_hw(adapter))
@@ -311,8 +311,10 @@ static void e1000e_dump(struct e1000_adapter *adapter)
0, rx_ring->next_to_use, rx_ring->next_to_clean);
/* Print Rx Ring */
- if (!netif_msg_rx_status(adapter))
+ if (!netif_msg_rx_status(adapter) || !rx_ring->pp)
return;
+ /* frames land past the pool's headroom, as the cleaner reads them */
+ hr = rx_ring->pp->p.offset;
dev_info(&adapter->pdev->dev, "Rx Ring Dump\n");
/* Extended Receive Descriptor (Read) Format
@@ -323,7 +325,7 @@ static void e1000e_dump(struct e1000_adapter *adapter)
* 8 | Reserved |
* +-----------------------------------------------------+
*/
- pr_info("R [desc] [buf addr 63:0 ] [reserved 63:0 ] [bi->dma ] [bi->skb] <-- Ext (Read) format\n");
+ pr_info("R [desc] [buf addr 63:0 ] [reserved 63:0 ] [fqe page ] offs <-- Ext (Read) format\n");
/* Extended Receive Descriptor (Write-Back) Format
*
* 63 48 47 32 31 24 23 4 3 0
@@ -337,12 +339,23 @@ static void e1000e_dump(struct e1000_adapter *adapter)
* +------------------------------------------------------+
* 63 48 47 32 31 20 19 0
*/
- pr_info("RWB[desc] [cs ipid mrq] [vt ln xe xs] [bi->skb] <-- Ext (Write-Back) format\n");
+ pr_info("RWB[desc] [cs ipid mrq] [vt ln xe xs] [fqe page ] offs <-- Ext (Write-Back) format\n");
for (i = 0; i < rx_ring->count; i++) {
+ const struct libeth_fqe *fqe = &rx_ring->rx_fqes[i];
const char *next_desc;
+ struct page *page;
+ bool posted;
+
+ /* fill queue entries outside the posted window are stale */
+ if (rx_ring->next_to_use >= rx_ring->next_to_clean)
+ posted = i >= rx_ring->next_to_clean &&
+ i < rx_ring->next_to_use;
+ else
+ posted = i >= rx_ring->next_to_clean ||
+ i < rx_ring->next_to_use;
+ page = posted ? __netmem_to_page(fqe->netmem) : NULL;
- buffer_info = &rx_ring->buffer_info[i];
rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
u0 = (struct my_u0 *)rx_desc;
staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
@@ -354,27 +367,17 @@ static void e1000e_dump(struct e1000_adapter *adapter)
else
next_desc = "";
- if (staterr & E1000_RXD_STAT_DD) {
- /* Descriptor Done */
- pr_info("%s[0x%03X] %016llX %016llX ---------------- %p%s\n",
- "RWB", i,
- (unsigned long long)le64_to_cpu(u0->a),
- (unsigned long long)le64_to_cpu(u0->b),
- buffer_info->skb, next_desc);
- } else {
- pr_info("%s[0x%03X] %016llX %016llX %016llX %p%s\n",
- "R ", i,
- (unsigned long long)le64_to_cpu(u0->a),
- (unsigned long long)le64_to_cpu(u0->b),
- (unsigned long long)buffer_info->dma,
- buffer_info->skb, next_desc);
-
- if (netif_msg_pktdata(adapter) && buffer_info->page)
- print_hex_dump(KERN_INFO, "",
- DUMP_PREFIX_ADDRESS, 16, 1,
- page_address(buffer_info->page),
- adapter->rx_buffer_len, true);
- }
+ pr_info("%s[0x%03X] %016llX %016llX %p %04X%s\n",
+ (staterr & E1000_RXD_STAT_DD) ? "RWB" : "R ", i,
+ (unsigned long long)le64_to_cpu(u0->a),
+ (unsigned long long)le64_to_cpu(u0->b),
+ page, page ? fqe->offset : 0, next_desc);
+
+ if (netif_msg_pktdata(adapter) && page)
+ print_hex_dump(KERN_INFO, "", DUMP_PREFIX_ADDRESS,
+ 16, 1,
+ page_address(page) + fqe->offset + hr,
+ rx_ring->rx_buf_len, true);
}
}
@@ -550,87 +553,124 @@ static void e1000e_update_tdt_wa(struct e1000_ring *tx_ring, unsigned int i)
}
}
+/**
+ * e1000_setup_rx_fq - create the libeth fill queue for the Rx path
+ * @rx_ring: Rx descriptor ring
+ *
+ * Returns 0 on success, negative on failure
+ **/
+static int e1000_setup_rx_fq(struct e1000_ring *rx_ring)
+{
+ struct e1000_adapter *adapter = rx_ring->adapter;
+ struct libeth_fq fq = {
+ .count = rx_ring->count,
+ .type = LIBETH_FQE_SHORT,
+ .buf_len = 2048, /* per-descriptor HW capacity (RCTL) */
+ .nid = NUMA_NO_NODE,
+ };
+ int err;
+
+ /* At MTU <= 1500 we are guaranteed that all frames fit in a 2 KB
+ * buffer because h/w discards frames longer than 1522 bytes when
+ * LPE is off.
+ * At higher MTU, LPE is enabled and we need to reserve the entire
+ * page to fit a 2 KB chunk from h/w plus overhead.
+ */
+ if (adapter->netdev->mtu > ETH_DATA_LEN)
+ fq.truesize = 4096;
+ else
+ fq.truesize = 2048;
+
+ err = libeth_rx_fq_create(&fq, &adapter->napi);
+ if (err)
+ return err;
+
+ rx_ring->pp = fq.pp;
+ rx_ring->rx_fqes = fq.fqes;
+ rx_ring->rx_truesize = fq.truesize;
+ rx_ring->rx_buf_len = fq.buf_len;
+ rx_ring->rx_fq_mtu = adapter->netdev->mtu;
+
+ return 0;
+}
+
+/**
+ * e1000_free_rx_fq - destroy the libeth fill queue, if any
+ * @rx_ring: Rx descriptor ring
+ *
+ * The ring must be cleaned first: all fill queue buffers recycled.
+ **/
+static void e1000_free_rx_fq(struct e1000_ring *rx_ring)
+{
+ struct libeth_fq fq = {
+ .fqes = rx_ring->rx_fqes,
+ .pp = rx_ring->pp,
+ };
+
+ if (!rx_ring->pp)
+ return;
+
+ libeth_rx_fq_destroy(&fq);
+ rx_ring->rx_fqes = NULL;
+ rx_ring->pp = NULL;
+}
+
/**
* e1000_alloc_rx_buffers - Replace used receive buffers
* @rx_ring: Rx descriptor ring
- * @cleaned_count: number of buffers to allocate this pass
- * @gfp: flags for allocation
+ * @cleaned_count: number to reallocate
**/
static void e1000_alloc_rx_buffers(struct e1000_ring *rx_ring,
- int cleaned_count, gfp_t gfp)
-{
+ int cleaned_count)
+{
+ const struct libeth_fq_fp fq = {
+ .pp = rx_ring->pp,
+ .fqes = rx_ring->rx_fqes,
+ .truesize = rx_ring->rx_truesize,
+ .count = rx_ring->count,
+ };
struct e1000_adapter *adapter = rx_ring->adapter;
- struct net_device *netdev = adapter->netdev;
- struct pci_dev *pdev = adapter->pdev;
union e1000_rx_desc_extended *rx_desc;
- struct e1000_buffer *buffer_info;
- struct sk_buff *skb;
unsigned int i;
- unsigned int bufsz = 256 - 16; /* for skb_reserve */
+
+ if (unlikely(!fq.pp)) {
+ adapter->alloc_rx_buff_failed += cleaned_count;
+ return;
+ }
i = rx_ring->next_to_use;
- buffer_info = &rx_ring->buffer_info[i];
while (cleaned_count--) {
- skb = buffer_info->skb;
- if (skb) {
- skb_trim(skb, 0);
- goto check_page;
- }
+ dma_addr_t addr;
- skb = __netdev_alloc_skb_ip_align(netdev, bufsz, gfp);
- if (unlikely(!skb)) {
+ addr = libeth_rx_alloc(&fq, i);
+ if (unlikely(addr == DMA_MAPPING_ERROR)) {
/* Better luck next round */
adapter->alloc_rx_buff_failed++;
break;
}
- buffer_info->skb = skb;
-check_page:
- /* allocate a new page if necessary */
- if (!buffer_info->page) {
- buffer_info->page = alloc_page(gfp);
- if (unlikely(!buffer_info->page)) {
- adapter->alloc_rx_buff_failed++;
- break;
- }
- }
-
- if (!buffer_info->dma) {
- buffer_info->dma = dma_map_page(&pdev->dev,
- buffer_info->page, 0,
- PAGE_SIZE,
- DMA_FROM_DEVICE);
- if (dma_mapping_error(&pdev->dev, buffer_info->dma)) {
- adapter->alloc_rx_buff_failed++;
- break;
- }
- }
-
rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
- rx_desc->read.buffer_addr = cpu_to_le64(buffer_info->dma);
+ rx_desc->read.buffer_addr = cpu_to_le64(addr);
- if (unlikely(++i == rx_ring->count))
+ if (unlikely(!(i & (E1000_RX_BUFFER_WRITE - 1)))) {
+ /* Force memory writes to complete before letting h/w
+ * know there are new descriptors to fetch. (Only
+ * applicable for weak-ordered memory model archs,
+ * such as IA-64).
+ */
+ wmb();
+ if (adapter->flags2 & FLAG2_PCIM2PCI_ARBITER_WA)
+ e1000e_update_rdt_wa(rx_ring, i);
+ else
+ writel(i, rx_ring->tail);
+ }
+ i++;
+ if (i == rx_ring->count)
i = 0;
- buffer_info = &rx_ring->buffer_info[i];
}
- if (likely(rx_ring->next_to_use != i)) {
- rx_ring->next_to_use = i;
- if (unlikely(i-- == 0))
- i = (rx_ring->count - 1);
-
- /* Force memory writes to complete before letting h/w
- * know there are new descriptors to fetch. (Only
- * applicable for weak-ordered memory model archs,
- * such as IA-64).
- */
- wmb();
- if (adapter->flags2 & FLAG2_PCIM2PCI_ARBITER_WA)
- e1000e_update_rdt_wa(rx_ring, i);
- else
- writel(i, rx_ring->tail);
- }
+ rx_ring->next_to_use = i;
}
static inline void e1000_rx_hash(struct net_device *netdev, __le32 rss,
@@ -640,6 +680,37 @@ static inline void e1000_rx_hash(struct net_device *netdev, __le32 rss,
skb_set_hash(skb, le32_to_cpu(rss), PKT_HASH_TYPE_L3);
}
+/**
+ * e1000_build_rx_skb - build an skb around a fill queue buffer
+ * @fqe: fill queue buffer holding the received frame
+ * @size: frame length
+ * @hr: buffer headroom, loop-invariant in the caller
+ *
+ * Returns the skb, or NULL on allocation failure. The buffer escapes to
+ * the stack and returns to the page pool when the skb is freed.
+ **/
+static struct sk_buff *e1000_build_rx_skb(const struct libeth_fqe *fqe,
+ u32 size, u32 hr)
+{
+ struct page *page = __netmem_to_page(fqe->netmem);
+ struct sk_buff *skb;
+ void *va;
+
+ /* the caller prefetched the headers at the top of its loop */
+ va = page_address(page) + fqe->offset;
+
+ skb = napi_build_skb(va, fqe->truesize);
+ if (unlikely(!skb))
+ return NULL;
+
+ skb_mark_for_recycle(skb);
+
+ skb_reserve(skb, hr);
+ __skb_put(skb, size);
+
+ return skb;
+}
+
static void e1000_put_txbuf(struct e1000_ring *tx_ring,
struct e1000_buffer *buffer_info,
bool drop)
@@ -879,144 +950,129 @@ static bool e1000_clean_tx_irq(struct e1000_ring *tx_ring)
return count < tx_ring->count;
}
-static void e1000_consume_page(struct e1000_buffer *bi, struct sk_buff *skb,
- u16 length)
-{
- bi->page = NULL;
- skb->len += length;
- skb->data_len += length;
- skb->truesize += PAGE_SIZE;
-}
-
/**
* e1000_clean_rx_irq - Send received data up the network stack
* @rx_ring: Rx descriptor ring
* @work_done: output parameter for indicating completed work
* @work_to_do: how many packets we can clean
+ *
+ * On an skb allocation failure the descriptor is left in place and the
+ * full budget is claimed, so the frame is retried on the next poll
+ * instead of dropped.
**/
static void e1000_clean_rx_irq(struct e1000_ring *rx_ring, int *work_done,
int work_to_do)
{
struct e1000_adapter *adapter = rx_ring->adapter;
struct net_device *netdev = adapter->netdev;
- struct pci_dev *pdev = adapter->pdev;
+ struct page_pool *pp = rx_ring->pp;
union e1000_rx_desc_extended *rx_desc, *next_rxd;
- struct e1000_buffer *buffer_info, *next_buffer;
- u32 length, staterr;
+ struct sk_buff *skb = rx_ring->rx_skb_top;
+ u32 hr, length, staterr;
unsigned int i;
int cleaned_count = 0;
unsigned int total_rx_bytes = 0, total_rx_packets = 0;
- struct skb_shared_info *shinfo;
+
+ /* The fill queue can be missing after failing to recreate it in
+ * e1000_configure_rx(). We may still end up here, because any
+ * MSI or legacy interrupt will schedule a poll.
+ */
+ if (unlikely(!pp))
+ return;
+ hr = pp->p.offset;
i = rx_ring->next_to_clean;
rx_desc = E1000_RX_DESC_EXT(*rx_ring, i);
staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
- buffer_info = &rx_ring->buffer_info[i];
while (staterr & E1000_RXD_STAT_DD) {
- struct sk_buff *skb;
+ const struct libeth_fqe *fqe;
+ struct page *page;
+ unsigned int next_i;
if (*work_done >= work_to_do)
break;
(*work_done)++;
- dma_rmb(); /* read descriptor and rx_buffer_info after status DD */
+ dma_rmb(); /* read descriptor after status DD */
- skb = buffer_info->skb;
- buffer_info->skb = NULL;
+ fqe = &rx_ring->rx_fqes[i];
+ page = __netmem_to_page(fqe->netmem);
- ++i;
- if (i == rx_ring->count)
- i = 0;
- next_rxd = E1000_RX_DESC_EXT(*rx_ring, i);
- prefetch(next_rxd);
+ /* Every outcome of this iteration touches the buffer's struct
+ * page.
+ */
+ prefetch(page);
- next_buffer = &rx_ring->buffer_info[i];
+ /* If this is the first chunk of a frame, pull in the headers
+ * too.
+ */
+ if (!skb)
+ net_prefetch(page_address(page) + fqe->offset + hr);
- cleaned_count++;
- dma_unmap_page(&pdev->dev, buffer_info->dma, PAGE_SIZE,
- DMA_FROM_DEVICE);
- buffer_info->dma = 0;
+ next_i = i + 1;
+ if (next_i == rx_ring->count)
+ next_i = 0;
+ next_rxd = E1000_RX_DESC_EXT(*rx_ring, next_i);
+ prefetch(next_rxd);
length = le16_to_cpu(rx_desc->wb.upper.length);
- /* errors is only valid for DD + EOP descriptors */
- if (unlikely((staterr & E1000_RXD_STAT_EOP) &&
- ((staterr & E1000_RXDEXT_ERR_FRAME_ERR_MASK) &&
- !(netdev->features & NETIF_F_RXALL)))) {
- /* recycle both page and skb */
- buffer_info->skb = skb;
+ /* Errors are only valid for DD + EOP descriptors. Test the
+ * rarely-set error mask first.
+ */
+ if (unlikely((staterr & E1000_RXDEXT_ERR_FRAME_ERR_MASK) &&
+ (staterr & E1000_RXD_STAT_EOP) &&
+ !(netdev->features & NETIF_F_RXALL))) {
/* an error means any chain goes out the window too */
- if (rx_ring->rx_skb_top)
- dev_kfree_skb_irq(rx_ring->rx_skb_top);
- rx_ring->rx_skb_top = NULL;
- goto next_desc;
- }
-#define rxtop (rx_ring->rx_skb_top)
- if (!(staterr & E1000_RXD_STAT_EOP)) {
- /* this descriptor is only the beginning (or middle) */
- if (!rxtop) {
- /* this is the beginning of a chain */
- rxtop = skb;
- skb_fill_page_desc(rxtop, 0, buffer_info->page,
- 0, length);
- } else {
- /* this is the middle of a chain */
- shinfo = skb_shinfo(rxtop);
- skb_fill_page_desc(rxtop, shinfo->nr_frags,
- buffer_info->page, 0,
- length);
- /* re-use the skb, only consumed the page */
- buffer_info->skb = skb;
+ if (skb) {
+ dev_kfree_skb_any(skb);
+ skb = NULL;
}
- e1000_consume_page(buffer_info, rxtop, length);
+ /* the cleaner only runs in the pool's NAPI context, so
+ * the buffer can go straight back to the pool's cache
+ */
+ page_pool_put_full_netmem(pp, fqe->netmem, true);
goto next_desc;
+ }
+
+ /* A zero-length fragment only returns to the pool; it can
+ * still carry EOP when a frame ends on a buffer boundary.
+ */
+ if (!libeth_rx_sync_for_cpu(fqe, length))
+ goto no_data;
+
+ if (skb) {
+ /* the frame continues from the previous descriptor */
+ skb_add_rx_frag_netmem(skb, skb_shinfo(skb)->nr_frags,
+ fqe->netmem, fqe->offset + hr,
+ length, fqe->truesize);
} else {
- if (rxtop) {
- /* end of the chain */
- shinfo = skb_shinfo(rxtop);
- skb_fill_page_desc(rxtop, shinfo->nr_frags,
- buffer_info->page, 0,
- length);
- /* re-use the current skb, we only consumed the
- * page
+ skb = e1000_build_rx_skb(fqe, length, hr);
+ if (unlikely(!skb)) {
+ /* leave the descriptor in place to retry the
+ * frame on the next poll, and claim the full
+ * budget to keep NAPI polling
*/
- buffer_info->skb = skb;
- skb = rxtop;
- rxtop = NULL;
- e1000_consume_page(buffer_info, skb, length);
- } else {
- /* no chain, got EOP, this buf is the packet
- * copybreak to save the put_page/alloc_page
- */
- if (length <= copybreak &&
- skb_tailroom(skb) >= length) {
- memcpy(skb_tail_pointer(skb),
- page_address(buffer_info->page),
- length);
- /* re-use the page, so don't erase
- * buffer_info->page
- */
- skb_put(skb, length);
- } else {
- skb_fill_page_desc(skb, 0,
- buffer_info->page, 0,
- length);
- e1000_consume_page(buffer_info, skb,
- length);
- }
+ adapter->alloc_rx_buff_failed++;
+ *work_done = work_to_do;
+ break;
}
}
+no_data:
+ /* non-EOP: hold the partial frame for the next descriptor */
+ if (!(staterr & E1000_RXD_STAT_EOP))
+ goto next_desc;
+
+ /* a zero-length frame with nothing accumulated */
+ if (unlikely(!skb))
+ goto next_desc;
+
/* strip the Ethernet CRC; it may span fragments */
if (!(adapter->flags2 & FLAG2_CRC_STRIPPING) &&
!(netdev->features & NETIF_F_RXFCS))
pskb_trim(skb, skb->len - 4);
- /* Receive Checksum Offload */
- e1000_rx_checksum(adapter, staterr, skb);
-
- e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
-
total_rx_bytes += skb->len;
/* If configured to store CRC, keep the FCS bytes out of the
* total_rx_bytes counter
@@ -1026,37 +1082,38 @@ static void e1000_clean_rx_irq(struct e1000_ring *rx_ring, int *work_done,
total_rx_bytes -= 4;
total_rx_packets++;
- /* eth type trans needs skb->data to point to something */
- if (!pskb_may_pull(skb, ETH_HLEN)) {
- e_err("pskb_may_pull failed.\n");
- dev_kfree_skb_irq(skb);
- goto next_desc;
- }
+ /* Receive Checksum Offload */
+ e1000_rx_checksum(adapter, staterr, skb);
+
+ e1000_rx_hash(netdev, rx_desc->wb.lower.hi_dword.rss, skb);
e1000_receive_skb(adapter, netdev, skb, staterr,
rx_desc->wb.upper.vlan);
+ skb = NULL;
next_desc:
rx_desc->wb.upper.status_error &= cpu_to_le32(~0xFF);
+ cleaned_count++;
/* return some buffers to hardware, one at a time is too slow */
- if (unlikely(cleaned_count >= E1000_RX_BUFFER_WRITE)) {
- e1000_alloc_rx_buffers(rx_ring, cleaned_count,
- GFP_ATOMIC);
+ if (cleaned_count >= E1000_RX_BUFFER_WRITE) {
+ e1000_alloc_rx_buffers(rx_ring, cleaned_count);
cleaned_count = 0;
}
/* use prefetched values */
+ i = next_i;
rx_desc = next_rxd;
- buffer_info = next_buffer;
staterr = le32_to_cpu(rx_desc->wb.upper.status_error);
}
rx_ring->next_to_clean = i;
+ /* an incomplete frame is finished on a later poll */
+ rx_ring->rx_skb_top = skb;
cleaned_count = e1000_desc_unused(rx_ring);
if (cleaned_count)
- e1000_alloc_rx_buffers(rx_ring, cleaned_count, GFP_ATOMIC);
+ e1000_alloc_rx_buffers(rx_ring, cleaned_count);
adapter->total_rx_bytes += total_rx_bytes;
adapter->total_rx_packets += total_rx_packets;
@@ -1068,28 +1125,15 @@ static void e1000_clean_rx_irq(struct e1000_ring *rx_ring, int *work_done,
**/
static void e1000_clean_rx_ring(struct e1000_ring *rx_ring)
{
- struct e1000_adapter *adapter = rx_ring->adapter;
- struct e1000_buffer *buffer_info;
- struct pci_dev *pdev = adapter->pdev;
unsigned int i;
- /* Free all the Rx ring sk_buffs */
- for (i = 0; i < rx_ring->count; i++) {
- buffer_info = &rx_ring->buffer_info[i];
- if (buffer_info->dma) {
- dma_unmap_page(&pdev->dev, buffer_info->dma,
- PAGE_SIZE, DMA_FROM_DEVICE);
- buffer_info->dma = 0;
- }
+ /* Return fill queue buffers owned by hardware to the page pool */
+ if (rx_ring->pp) {
+ for (i = rx_ring->next_to_clean; i != rx_ring->next_to_use;) {
+ libeth_rx_recycle_slow(rx_ring->rx_fqes[i].netmem);
- if (buffer_info->page) {
- put_page(buffer_info->page);
- buffer_info->page = NULL;
- }
-
- if (buffer_info->skb) {
- dev_kfree_skb(buffer_info->skb);
- buffer_info->skb = NULL;
+ if (unlikely(++i == rx_ring->count))
+ i = 0;
}
}
@@ -1734,33 +1778,29 @@ int e1000e_setup_tx_resources(struct e1000_ring *tx_ring)
int e1000e_setup_rx_resources(struct e1000_ring *rx_ring)
{
struct e1000_adapter *adapter = rx_ring->adapter;
- int size, desc_len, err = -ENOMEM;
-
- size = sizeof(struct e1000_buffer) * rx_ring->count;
- rx_ring->buffer_info = vzalloc(size);
- if (!rx_ring->buffer_info)
- goto err;
-
- desc_len = sizeof(union e1000_rx_desc_extended);
+ int err;
/* Round up to nearest 4K */
- rx_ring->size = rx_ring->count * desc_len;
+ rx_ring->size = rx_ring->count * sizeof(union e1000_rx_desc_extended);
rx_ring->size = ALIGN(rx_ring->size, 4096);
err = e1000_alloc_ring_dma(adapter, rx_ring);
- if (err)
- goto err;
+ if (err) {
+ e_err("Unable to allocate memory for the receive descriptor ring\n");
+ return err;
+ }
rx_ring->next_to_clean = 0;
rx_ring->next_to_use = 0;
rx_ring->rx_skb_top = NULL;
- return 0;
+ /* the fill queue belongs to the old ring resources until freed;
+ * e1000_configure_rx() creates one for this ring when needed
+ */
+ rx_ring->pp = NULL;
+ rx_ring->rx_fqes = NULL;
-err:
- vfree(rx_ring->buffer_info);
- e_err("Unable to allocate memory for the receive descriptor ring\n");
- return err;
+ return 0;
}
/**
@@ -1822,9 +1862,7 @@ void e1000e_free_rx_resources(struct e1000_ring *rx_ring)
struct pci_dev *pdev = adapter->pdev;
e1000_clean_rx_ring(rx_ring);
-
- vfree(rx_ring->buffer_info);
- rx_ring->buffer_info = NULL;
+ e1000_free_rx_fq(rx_ring);
dma_free_coherent(&pdev->dev, rx_ring->size, rx_ring->desc,
rx_ring->dma);
@@ -2435,25 +2473,13 @@ static void e1000_setup_rctl(struct e1000_adapter *adapter)
e1e_wphy(hw, 22, phy_data);
}
- /* Setup buffer sizes */
- rctl &= ~E1000_RCTL_SZ_4096;
- rctl |= E1000_RCTL_BSEX;
- switch (adapter->rx_buffer_len) {
- case 2048:
- default:
- rctl |= E1000_RCTL_SZ_2048;
- rctl &= ~E1000_RCTL_BSEX;
- break;
- case 4096:
- rctl |= E1000_RCTL_SZ_4096;
- break;
- case 8192:
- rctl |= E1000_RCTL_SZ_8192;
- break;
- case 16384:
- rctl |= E1000_RCTL_SZ_16384;
- break;
- }
+ /* Default to maximum-size 2048-byte chunks (E1000_RCTL_SZ_256 is the
+ * BSIZE field mask); e1000_configure_rx() lowers the chunk size if
+ * the fill queue buffers are smaller. Frames longer than one chunk
+ * are chained across descriptors.
+ */
+ rctl &= ~(E1000_RCTL_BSEX | E1000_RCTL_SZ_256);
+ rctl |= E1000_RCTL_SZ_2048;
/* Enable Extended Status in all Receive Descriptors */
rfctl = er32(RFCTL);
@@ -2497,8 +2523,42 @@ static void e1000_configure_rx(struct e1000_adapter *adapter)
rdlen = rx_ring->count * sizeof(union e1000_rx_desc_extended);
+ /* The fill queue geometry depends on the MTU. e1000e_open() creates
+ * the fill queue and can fail cleanly; here a creation failure only
+ * logs, and the guards in the allocator and the cleaner keep an
+ * fq-less ring safe: no buffers are ever posted, so the hardware
+ * drops frames in silicon until a reconfigure retries.
+ */
+ if (rx_ring->pp && rx_ring->rx_fq_mtu != adapter->netdev->mtu)
+ e1000_free_rx_fq(rx_ring);
+ if (!rx_ring->pp && e1000_setup_rx_fq(rx_ring))
+ e_err("Failed to create Rx fill queue\n");
+
/* disable receives while setting up the descriptors */
rctl = er32(RCTL);
+
+ /* Pair the per-descriptor chunk size with the fill queue buffers: a
+ * chunk must never overrun one buffer. Without LPE the hardware
+ * caps frames at 1522 bytes, so any buffer at least that large
+ * takes every frame in a single maximum-size chunk.
+ */
+ if (rx_ring->pp) {
+ u32 bsize = E1000_RCTL_SZ_2048;
+
+ if (rx_ring->rx_buf_len < 2048 &&
+ ((rctl & E1000_RCTL_LPE) ||
+ rx_ring->rx_buf_len < VLAN_ETH_FRAME_LEN + ETH_FCS_LEN)) {
+ if (rx_ring->rx_buf_len >= 1024)
+ bsize = E1000_RCTL_SZ_1024;
+ else if (rx_ring->rx_buf_len >= 512)
+ bsize = E1000_RCTL_SZ_512;
+ else
+ bsize = E1000_RCTL_SZ_256;
+ }
+
+ rctl &= ~(E1000_RCTL_BSEX | E1000_RCTL_SZ_256);
+ rctl |= bsize;
+ }
if (!(adapter->flags2 & FLAG2_NO_DISABLE_RX))
ew32(RCTL, rctl & ~E1000_RCTL_EN);
e1e_flush();
@@ -3062,7 +3122,7 @@ static void e1000_configure(struct e1000_adapter *adapter)
e1000e_setup_rss_hash(adapter);
e1000_setup_rctl(adapter);
e1000_configure_rx(adapter);
- e1000_alloc_rx_buffers(rx_ring, e1000_desc_unused(rx_ring), GFP_KERNEL);
+ e1000_alloc_rx_buffers(rx_ring, e1000_desc_unused(rx_ring));
}
/**
@@ -3752,7 +3812,6 @@ static int e1000_sw_init(struct e1000_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
- adapter->rx_buffer_len = 2048;
adapter->max_frame_size = netdev->mtu + VLAN_ETH_HLEN + ETH_FCS_LEN;
adapter->min_frame_size = ETH_ZLEN + ETH_FCS_LEN;
adapter->tx_ring_count = E1000_DEFAULT_TXD;
@@ -3945,6 +4004,11 @@ int e1000e_open(struct net_device *netdev)
if (err)
goto err_setup_rx;
+ /* create the Rx fill queue backing the receive descriptors */
+ err = e1000_setup_rx_fq(adapter->rx_ring);
+ if (err)
+ goto err_setup_fq;
+
/* If AMT is enabled, let the firmware know that the network
* interface is now open and reset the part to a known state.
*/
@@ -4013,6 +4077,7 @@ int e1000e_open(struct net_device *netdev)
cpu_latency_qos_remove_request(&adapter->pm_qos_req);
e1000e_release_hw_control(adapter);
e1000_power_down_phy(adapter);
+err_setup_fq:
e1000e_free_rx_resources(adapter->rx_ring);
err_setup_rx:
e1000e_free_tx_resources(adapter->tx_ring);
@@ -5383,10 +5448,9 @@ static int e1000_change_mtu(struct net_device *netdev, int new_mtu)
if (netif_running(netdev))
e1000e_down(adapter, true);
- if (max_frame <= 2048)
- adapter->rx_buffer_len = 2048;
- else
- adapter->rx_buffer_len = 4096;
+ /* the Rx fill queue geometry and the RCTL chunk size are derived
+ * from the new MTU when the interface comes back up
+ */
if (netif_running(netdev))
e1000e_up(adapter);
@@ -7520,5 +7584,6 @@ module_exit(e1000_exit_module);
MODULE_DESCRIPTION("Intel(R) PRO/1000 Network Driver");
MODULE_LICENSE("GPL v2");
+MODULE_IMPORT_NS("LIBETH");
/* netdev.c */
diff --git a/drivers/net/ethernet/intel/e1000e/param.c b/drivers/net/ethernet/intel/e1000e/param.c
index 3132d8f2f207..bf8a006686d5 100644
--- a/drivers/net/ethernet/intel/e1000e/param.c
+++ b/drivers/net/ethernet/intel/e1000e/param.c
@@ -16,12 +16,6 @@
#define OPTION_DISABLED 0
#define OPTION_ENABLED 1
-#define COPYBREAK_DEFAULT 256
-unsigned int copybreak = COPYBREAK_DEFAULT;
-module_param(copybreak, uint, 0644);
-MODULE_PARM_DESC(copybreak,
- "Maximum size of packet that is copied to a new buffer on receive");
-
/* All parameters are treated the same, as an integer array of values.
* This macro just reduces the need to repeat the same declaration code
* over and over (plus this helps to avoid typo bugs).
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH iwl-next 8/8] e1000e: return skbs to NAPI cache
2026-08-30 23:21 [PATCH iwl-next 0/8] e1000e: use page pool Matt Vollrath
` (6 preceding siblings ...)
2026-08-30 23:21 ` [PATCH iwl-next 7/8] e1000e: use libeth page_pool for Rx Matt Vollrath
@ 2026-08-30 23:21 ` Matt Vollrath
7 siblings, 0 replies; 14+ messages in thread
From: Matt Vollrath @ 2026-08-30 23:21 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, Tony Nguyen, Przemek Kitszel, Alexander Lobakin,
Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kernel, Matt Vollrath
Use napi_consume_skb to hand skbs back to the NAPI cache. This closes
the loop with changes to the Rx path.
Signed-off-by: Matt Vollrath <tactii@gmail.com>
Assisted-by: Claude:claude-5-fable
---
drivers/net/ethernet/intel/e1000e/netdev.c | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index c906175ad38d..a28ee0750f4b 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -713,7 +713,7 @@ static struct sk_buff *e1000_build_rx_skb(const struct libeth_fqe *fqe,
static void e1000_put_txbuf(struct e1000_ring *tx_ring,
struct e1000_buffer *buffer_info,
- bool drop)
+ bool drop, int budget)
{
struct e1000_adapter *adapter = tx_ring->adapter;
@@ -730,7 +730,7 @@ static void e1000_put_txbuf(struct e1000_ring *tx_ring,
if (drop)
dev_kfree_skb_any(buffer_info->skb);
else
- dev_consume_skb_any(buffer_info->skb);
+ napi_consume_skb(buffer_info->skb, budget);
buffer_info->skb = NULL;
}
buffer_info->time_stamp = 0;
@@ -860,11 +860,12 @@ static void e1000e_tx_hwtstamp_work(struct work_struct *work)
/**
* e1000_clean_tx_irq - Reclaim resources after transmit completes
* @tx_ring: Tx descriptor ring
+ * @napi_budget: NAPI polling budget, or 0 when called outside NAPI context
*
* the return value indicates whether actual cleaning was done, there
* is no guarantee that everything was cleaned
**/
-static bool e1000_clean_tx_irq(struct e1000_ring *tx_ring)
+static bool e1000_clean_tx_irq(struct e1000_ring *tx_ring, int napi_budget)
{
struct e1000_adapter *adapter = tx_ring->adapter;
struct net_device *netdev = adapter->netdev;
@@ -899,7 +900,8 @@ static bool e1000_clean_tx_irq(struct e1000_ring *tx_ring)
}
}
- e1000_put_txbuf(tx_ring, buffer_info, false);
+ e1000_put_txbuf(tx_ring, buffer_info, false,
+ napi_budget);
tx_desc->upper.data = 0;
i++;
@@ -1340,7 +1342,7 @@ static irqreturn_t e1000_intr_msix_tx(int __always_unused irq, void *data)
adapter->total_tx_bytes = 0;
adapter->total_tx_packets = 0;
- if (!e1000_clean_tx_irq(tx_ring))
+ if (!e1000_clean_tx_irq(tx_ring, 0))
/* Ring was not completely cleaned, so fire another interrupt */
ew32(ICS, tx_ring->ims_val);
@@ -1816,7 +1818,7 @@ static void e1000_clean_tx_ring(struct e1000_ring *tx_ring)
for (i = 0; i < tx_ring->count; i++) {
buffer_info = &tx_ring->buffer_info[i];
- e1000_put_txbuf(tx_ring, buffer_info, false);
+ e1000_put_txbuf(tx_ring, buffer_info, false, 0);
}
netdev_reset_queue(adapter->netdev);
@@ -2060,7 +2062,7 @@ static int e1000e_poll(struct napi_struct *napi, int budget)
if (!adapter->msix_entries ||
(adapter->rx_ring->ims_val & adapter->tx_ring->ims_val))
- tx_cleaned = e1000_clean_tx_irq(adapter->tx_ring);
+ tx_cleaned = e1000_clean_tx_irq(adapter->tx_ring, budget);
e1000_clean_rx_irq(adapter->rx_ring, &work_done, budget);
@@ -5019,7 +5021,7 @@ static int e1000_tx_map(struct e1000_ring *tx_ring, struct sk_buff *skb,
i += tx_ring->count;
i--;
buffer_info = &tx_ring->buffer_info[i];
- e1000_put_txbuf(tx_ring, buffer_info, true);
+ e1000_put_txbuf(tx_ring, buffer_info, true, 0);
}
return 0;
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread