[PATCH net v3] igb: cope with large MAX_SKB_FRAGS.

[PATCH net v3] igb: cope with large MAX_SKB_FRAGS.

[Corinna Vinschen]

From: Paolo Abeni <pabeni@redhat.com>

Sabrina reports that the igb driver does not cope well with large
MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
corruption on TX.

An easy reproducer is to run ssh to connect to the machine.  With
MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.

The root cause of the issue is that the driver does not take into
account properly the (possibly large) shared info size when selecting
the ring layout, and will try to fit two packets inside the same 4K
page even when the 1st fraglist will trump over the 2nd head.

Address the issue forcing the driver to fit a single packet per page,
leaving there enough room to store the (currently) largest possible
skb_shared_info.

Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAGS")
Reported-by: Jan Tluka <jtluka@redhat.com>
Reported-by: Jirka Hladky <jhladky@redhat.com>
Reported-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Corinna Vinschen <vinschen@redhat.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
v2: fix subject, add a simple reproducer
v3: fix Fixes, tested with all MTUs from 1200 to 1280 per Eric's suggestion

 drivers/net/ethernet/intel/igb/igb_main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 11be39f435f3..232d6cb836a9 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -4808,6 +4808,7 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
 
 #if (PAGE_SIZE < 8192)
 	if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB ||
+	    SKB_HEAD_ALIGN(adapter->max_frame_size) > (PAGE_SIZE / 2) ||
 	    rd32(E1000_RCTL) & E1000_RCTL_SBP)
 		set_ring_uses_large_buffer(rx_ring);
 #endif
-- 
2.45.2

Re: [PATCH net v3] igb: cope with large MAX_SKB_FRAGS.

[Paolo Abeni]

On 7/18/24 10:56, Corinna Vinschen wrote:
> From: Paolo Abeni <pabeni@redhat.com>
> 
> Sabrina reports that the igb driver does not cope well with large
> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> corruption on TX.
> 
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
> 
> The root cause of the issue is that the driver does not take into
> account properly the (possibly large) shared info size when selecting
> the ring layout, and will try to fit two packets inside the same 4K
> page even when the 1st fraglist will trump over the 2nd head.
> 
> Address the issue forcing the driver to fit a single packet per page,
> leaving there enough room to store the (currently) largest possible
> skb_shared_info.
> 
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAGS")
> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>

@Tony: would you like to take this one in your tree first, or we can 
merge it directly?

Thanks!

Paolo

Re: [PATCH net v3] igb: cope with large MAX_SKB_FRAGS.

[Tony Nguyen]

On 7/23/2024 1:27 AM, Paolo Abeni wrote:
> On 7/18/24 10:56, Corinna Vinschen wrote:
>> From: Paolo Abeni <pabeni@redhat.com>
>>
>> Sabrina reports that the igb driver does not cope well with large
>> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
>> corruption on TX.
>>
>> An easy reproducer is to run ssh to connect to the machine.  With
>> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
>>
>> The root cause of the issue is that the driver does not take into
>> account properly the (possibly large) shared info size when selecting
>> the ring layout, and will try to fit two packets inside the same 4K
>> page even when the 1st fraglist will trump over the 2nd head.
>>
>> Address the issue forcing the driver to fit a single packet per page,
>> leaving there enough room to store the (currently) largest possible
>> skb_shared_info.
>>
>> Fixes: 3948b05950fd ("net: introduce a config option to tweak 
>> MAX_SKB_FRAGS")
>> Reported-by: Jan Tluka <jtluka@redhat.com>
>> Reported-by: Jirka Hladky <jhladky@redhat.com>
>> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
>> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
>> Tested-by: Corinna Vinschen <vinschen@redhat.com>
>> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> 
> @Tony: would you like to take this one in your tree first, or we can 
> merge it directly?

Hi Paolo,

I can take it through IWL unless you need to get it in sooner, in which 
case, feel free to take it directly. If so...

Reviewed-by: Tony Nguyen <anthony.l.nguyen@intel.com>

Thanks,
Tony

Re: [PATCH net v3] igb: cope with large MAX_SKB_FRAGS.

[Eric Dumazet]

On Thu, Jul 18, 2024 at 10:56 AM Corinna Vinschen <vinschen@redhat.com> wrote:
>
> From: Paolo Abeni <pabeni@redhat.com>
>
> Sabrina reports that the igb driver does not cope well with large
> MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload
> corruption on TX.
>
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
>
> The root cause of the issue is that the driver does not take into
> account properly the (possibly large) shared info size when selecting
> the ring layout, and will try to fit two packets inside the same 4K
> page even when the 1st fraglist will trump over the 2nd head.
>
> Address the issue forcing the driver to fit a single packet per page,
> leaving there enough room to store the (currently) largest possible
> skb_shared_info.
>
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAGS")
> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>

Reviewed-by: Eric Dumazet <edumazet@google.com>

RE: [Intel-wired-lan] [PATCH net v3] igb: cope with large MAX_SKB_FRAGS.

[Pucha, HimasekharX Reddy]

> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Corinna Vinschen
> Sent: Thursday, July 18, 2024 2:27 PM
> To: netdev@vger.kernel.org; intel-wired-lan@lists.osuosl.org; Eric Dumazet <edumazet@google.com>
> Cc: Jason Xing <kerneljasonxing@gmail.com>; Nikolay Aleksandrov <razor@blackwall.org>; linux-kernel@vger.kernel.org; Nguyen, Anthony L <anthony.l.nguyen@intel.com>; Jakub Kicinski <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>; David S . Miller <davem@davemloft.net>
> Subject: [Intel-wired-lan] [PATCH net v3] igb: cope with large MAX_SKB_FRAGS.
>
> From: Paolo Abeni <pabeni@redhat.com>
>
> Sabrina reports that the igb driver does not cope well with large MAX_SKB_FRAG values: setting MAX_SKB_FRAG to 45 causes payload corruption on TX.
>
> An easy reproducer is to run ssh to connect to the machine.  With
> MAX_SKB_FRAGS=17 it works, with MAX_SKB_FRAGS=45 it fails.
>
> The root cause of the issue is that the driver does not take into account properly the (possibly large) shared info size when selecting the ring layout, and will try to fit two packets inside the same 4K page even when the 1st fraglist will trump over the 2nd head.
>
> Address the issue forcing the driver to fit a single packet per page, leaving there enough room to store the (currently) largest possible skb_shared_info.
>
> Fixes: 3948b05950fd ("net: introduce a config option to tweak MAX_SKB_FRAGS")
> Reported-by: Jan Tluka <jtluka@redhat.com>
> Reported-by: Jirka Hladky <jhladky@redhat.com>
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Sabrina Dubroca <sd@queasysnail.net>
> Tested-by: Corinna Vinschen <vinschen@redhat.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> v2: fix subject, add a simple reproducer
> v3: fix Fixes, tested with all MTUs from 1200 to 1280 per Eric's suggestion
>
>  drivers/net/ethernet/intel/igb/igb_main.c | 1 +
>  1 file changed, 1 insertion(+)
>

Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)