* Re: [Intel-wired-lan] Intel-wired-lan Digest, Vol 405, Issue 6
[not found] <mailman.5156.1673329730.179342.intel-wired-lan@osuosl.org>
@ 2023-01-24 14:46 ` Zulkifli, Muhammad Husaini
2023-01-25 7:38 ` Neftin, Sasha
0 siblings, 1 reply; 2+ messages in thread
From: Zulkifli, Muhammad Husaini @ 2023-01-24 14:46 UTC (permalink / raw)
To: intel-wired-lan@osuosl.org
Hi Sasha,
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> intel-wired-lan-request@osuosl.org
> Sent: Tuesday, 10 January, 2023 1:49 PM
> To: intel-wired-lan@osuosl.org
> Subject: Intel-wired-lan Digest, Vol 405, Issue 6
>
> Send Intel-wired-lan mailing list submissions to
> intel-wired-lan@osuosl.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
> or, via email, send a message with subject or body 'help' to
> intel-wired-lan-request@osuosl.org
>
> You can reach the person managing the list at
> intel-wired-lan-owner@osuosl.org
>
> When replying, please edit your Subject line so it is more specific than "Re:
> Contents of Intel-wired-lan digest..."
>
>
> Today's Topics:
>
> 1. [PATCH 1/1] ice: WiP support for BIG TCP packets
> (Pawel Chmielewski)
> 2. Re: [PATCH 1/1] ice: WiP support for BIG TCP packets
> (Alexander H Duyck)
> 3. [tnguy-net-queue:dev-queue] BUILD SUCCESS
> 4cb425d20a6ddbf9fd40989c31f5c6f8f304dc35 (kernel test robot)
> 4. [PATCH net v1 1/1] igc: Add ndo_tx_timeout support (Sasha Neftin)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 9 Jan 2023 17:18:33 +0100
> From: Pawel Chmielewski <pawel.chmielewski@intel.com>
> To: netdev@vger.kernel.org
> Cc: intel-wired-lan@osuosl.org, Pawel Chmielewski
> <pawel.chmielewski@intel.com>
> Subject: [Intel-wired-lan] [PATCH 1/1] ice: WiP support for BIG TCP
> packets
> Message-ID: <20230109161833.223510-1-pawel.chmielewski@intel.com>
>
> This patch is a proof of concept for testing BIG TCP feature in ice driver.
> Please see letter below.
>
> Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
> ---
> Hi All
> I'm writing on the list, as you may be able to provide me some feedback.
> I want to enable BIG TCP feature in intel ice drive, but I think I'm missing
> something.
> In the code itself, I've set 128k as a maximum tso size for the netif, and
> added stripping the HBH option from the header.
> For testing purposes, gso_max_size & gro_max_size were set to 128k and
> mtu to 9000.
> I've assumed that the ice tso offload will do the rest of the job.
> However- while running netperf TCP_RR and TCP_STREAM tests, I saw that
> only up to ~20% of the transmitted test packets have the specified size.
> Other packets to be transmitted, appear from the stack as splitted.
>
> I've been running the following testcases:
> netperf -t TCP_RR -H 2001:db8:0:f101::1 -- -r80000,80000 -O
> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
> netperf -l-1 -t TCP_STREAM -H 2001:db8:0:f101::1 -- -m 128K -O
> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
> I suspected a shrinking tcp window size, but sniffing with tcpdump showed
> rather big scaling factor (usually 128x).
> Apart from using netperf, I also tried a simple IPv6 user space application
> (with SO_SNDBUF option set to 192k and TCP_WINDOW_CLAMP to 96k) -
> similar results.
>
> I'd be very grateful for any feedback/suggestions
>
> Pawel
> ---
> drivers/net/ethernet/intel/ice/ice_main.c | 4 ++++
> drivers/net/ethernet/intel/ice/ice_txrx.c | 9 +++++++++
> 2 files changed, 13 insertions(+)
>
> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c
> b/drivers/net/ethernet/intel/ice/ice_main.c
> index 2b23b4714a26..4e657820e55d 100644
> --- a/drivers/net/ethernet/intel/ice/ice_main.c
> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> @@ -48,6 +48,8 @@ static DEFINE_IDA(ice_aux_ida);
> DEFINE_STATIC_KEY_FALSE(ice_xdp_locking_key);
> EXPORT_SYMBOL(ice_xdp_locking_key);
>
> +#define ICE_MAX_TSO_SIZE 131072
> +
> /**
> * ice_hw_to_dev - Get device pointer from the hardware structure
> * @hw: pointer to the device HW structure @@ -3422,6 +3424,8 @@ static
> void ice_set_netdev_features(struct net_device *netdev)
> * be changed at runtime
> */
> netdev->hw_features |= NETIF_F_RXFCS;
> +
> + netif_set_tso_max_size(netdev, ICE_MAX_TSO_SIZE);
> }
>
> /**
> diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c
> b/drivers/net/ethernet/intel/ice/ice_txrx.c
> index 086f0b3ab68d..7e0ac483cad9 100644
> --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
> +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
> @@ -23,6 +23,9 @@
> #define FDIR_DESC_RXDID 0x40
> #define ICE_FDIR_CLEAN_DELAY 10
>
> +#define HBH_HDR_SIZE sizeof(struct hop_jumbo_hdr) #define
> HBH_OFFSET
> +ETH_HLEN + sizeof(struct ipv6hdr)
> +
> /**
> * ice_prgm_fdir_fltr - Program a Flow Director filter
> * @vsi: VSI to send dummy packet
> @@ -2300,6 +2303,12 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct
> ice_tx_ring *tx_ring)
>
> ice_trace(xmit_frame_ring, tx_ring, skb);
>
> + if (ipv6_has_hopopt_jumbo(skb)) {
> + memmove(skb->data + HBH_HDR_SIZE, skb->data,
> HBH_OFFSET);
> + __skb_pull(skb, HBH_HDR_SIZE);
> + skb_reset_mac_header(skb);
> + }
> +
> count = ice_xmit_desc_count(skb);
> if (ice_chk_linearize(skb, count)) {
> if (__skb_linearize(skb))
> --
> 2.37.3
>
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 09 Jan 2023 10:22:22 -0800
> From: Alexander H Duyck <alexander.duyck@gmail.com>
> To: Pawel Chmielewski <pawel.chmielewski@intel.com>,
> netdev@vger.kernel.org
> Cc: intel-wired-lan@osuosl.org
> Subject: Re: [Intel-wired-lan] [PATCH 1/1] ice: WiP support for BIG
> TCP packets
> Message-ID:
> <9b68abc5e8613e02207e9c0c3619b1b07bc5bb8c.camel@gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> On Mon, 2023-01-09 at 17:18 +0100, Pawel Chmielewski wrote:
> > This patch is a proof of concept for testing BIG TCP feature in ice driver.
> > Please see letter below.
> >
> > Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
> > ---
> > Hi All
> > I'm writing on the list, as you may be able to provide me some feedback.
> > I want to enable BIG TCP feature in intel ice drive, but I think I'm
> > missing something.
> > In the code itself, I've set 128k as a maximum tso size for the netif,
> > and added stripping the HBH option from the header.
> > For testing purposes, gso_max_size & gro_max_size were set to 128k and
> > mtu to 9000.
> > I've assumed that the ice tso offload will do the rest of the job.
> > However- while running netperf TCP_RR and TCP_STREAM tests,
> > I saw that only up to ~20% of the transmitted test packets have
> > the specified size.
> > Other packets to be transmitted, appear from the stack as splitted.
> >
> > I've been running the following testcases:
> > netperf -t TCP_RR -H 2001:db8:0:f101::1 -- -r80000,80000 -O
> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
> > netperf -l-1 -t TCP_STREAM -H 2001:db8:0:f101::1 -- -m 128K -O
> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
> > I suspected a shrinking tcp window size, but sniffing with tcpdump showed
> rather big scaling factor (usually 128x).
> > Apart from using netperf, I also tried a simple IPv6 user space application
> > (with SO_SNDBUF option set to 192k and TCP_WINDOW_CLAMP to 96k) -
> similar results.
> >
> > I'd be very grateful for any feedback/suggestions
> >
> > Pawel
> > ---
> > drivers/net/ethernet/intel/ice/ice_main.c | 4 ++++
> > drivers/net/ethernet/intel/ice/ice_txrx.c | 9 +++++++++
> > 2 files changed, 13 insertions(+)
> >
> > diff --git a/drivers/net/ethernet/intel/ice/ice_main.c
> b/drivers/net/ethernet/intel/ice/ice_main.c
> > index 2b23b4714a26..4e657820e55d 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_main.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> > @@ -48,6 +48,8 @@ static DEFINE_IDA(ice_aux_ida);
> > DEFINE_STATIC_KEY_FALSE(ice_xdp_locking_key);
> > EXPORT_SYMBOL(ice_xdp_locking_key);
> >
> > +#define ICE_MAX_TSO_SIZE 131072
> > +
> > /**
> > * ice_hw_to_dev - Get device pointer from the hardware structure
> > * @hw: pointer to the device HW structure
> > @@ -3422,6 +3424,8 @@ static void ice_set_netdev_features(struct
> net_device *netdev)
> > * be changed at runtime
> > */
> > netdev->hw_features |= NETIF_F_RXFCS;
> > +
> > + netif_set_tso_max_size(netdev, ICE_MAX_TSO_SIZE);
> > }
> >
> > /**
> > diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c
> b/drivers/net/ethernet/intel/ice/ice_txrx.c
> > index 086f0b3ab68d..7e0ac483cad9 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
> > @@ -23,6 +23,9 @@
> > #define FDIR_DESC_RXDID 0x40
> > #define ICE_FDIR_CLEAN_DELAY 10
> >
> > +#define HBH_HDR_SIZE sizeof(struct hop_jumbo_hdr)
> > +#define HBH_OFFSET ETH_HLEN + sizeof(struct ipv6hdr)
> > +
> > /**
> > * ice_prgm_fdir_fltr - Program a Flow Director filter
> > * @vsi: VSI to send dummy packet
> > @@ -2300,6 +2303,12 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct
> ice_tx_ring *tx_ring)
> >
> > ice_trace(xmit_frame_ring, tx_ring, skb);
> >
> > + if (ipv6_has_hopopt_jumbo(skb)) {
> > + memmove(skb->data + HBH_HDR_SIZE, skb->data,
> HBH_OFFSET);
> > + __skb_pull(skb, HBH_HDR_SIZE);
> > + skb_reset_mac_header(skb);
> > + }
> > +
> > count = ice_xmit_desc_count(skb);
> > if (ice_chk_linearize(skb, count)) {
> > if (__skb_linearize(skb))
>
> Your removal code here is forgetting to handle the network header. As a
> result your frames will be pointer mangled in terms of header location.
>
> You might be better off using ipv6_hopopt_jumbo_remove() rather than
> just coding your own bit to remove it.
>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 10 Jan 2023 13:10:09 +0800
> From: kernel test robot <lkp@intel.com>
> To: Intel Wired LAN <intel-wired-lan@lists.osuosl.org>
> Subject: [Intel-wired-lan] [tnguy-net-queue:dev-queue] BUILD SUCCESS
> 4cb425d20a6ddbf9fd40989c31f5c6f8f304dc35
> Message-ID: <63bcf331.kDz0I6QwE35Zcf1P%lkp@intel.com>
> Content-Type: text/plain; charset=us-ascii
>
> tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/tnguy/net-
> queue.git dev-queue
> branch HEAD: 4cb425d20a6ddbf9fd40989c31f5c6f8f304dc35 ice: fix out-of-
> bounds KASAN warning in virtchnl
>
> elapsed time: 723m
>
> configs tested: 84
> configs skipped: 2
>
> The following configs have been built successfully.
> More configs may be tested in the coming days.
>
> gcc tested configs:
> x86_64 allnoconfig
> arc defconfig
> alpha defconfig
> powerpc allnoconfig
> um i386_defconfig
> um x86_64_defconfig
> alpha allyesconfig
> s390 allmodconfig
> m68k allmodconfig
> arc allyesconfig
> s390 defconfig
> m68k allyesconfig
> x86_64 defconfig
> s390 allyesconfig
> i386 defconfig
> arm defconfig
> i386 allyesconfig
> x86_64 randconfig-a011-20230109
> x86_64 randconfig-a013-20230109
> i386 randconfig-a011-20230109
> x86_64 randconfig-a012-20230109
> sh allmodconfig
> i386 randconfig-a013-20230109
> x86_64 rhel-8.3-bpf
> x86_64 rhel-8.3
> x86_64 randconfig-a014-20230109
> x86_64 randconfig-a016-20230109
> x86_64 randconfig-a015-20230109
> ia64 allmodconfig
> x86_64 allyesconfig
> i386 randconfig-a012-20230109
> x86_64 rhel-8.3-kselftests
> x86_64 rhel-8.3-syz
> x86_64 rhel-8.3-kunit
> x86_64 rhel-8.3-func
> i386 randconfig-a014-20230109
> x86_64 rhel-8.3-kvm
> i386 randconfig-a016-20230109
> i386 randconfig-a015-20230109
> mips allyesconfig
> powerpc allmodconfig
> arm64 allyesconfig
> arm allyesconfig
> riscv randconfig-r042-20230109
> s390 randconfig-r044-20230109
> arm randconfig-r046-20230108
> arc randconfig-r043-20230108
> arc randconfig-r043-20230109
> i386 randconfig-c001
> arm trizeps4_defconfig
> arm s3c6400_defconfig
> sh alldefconfig
> ia64 defconfig
> riscv nommu_virt_defconfig
> riscv rv32_defconfig
> riscv nommu_k210_defconfig
> riscv allnoconfig
> i386 debian-10.3-kselftests
> i386 debian-10.3
> m68k sun3_defconfig
> parisc generic-32bit_defconfig
> sh sh7710voipgw_defconfig
> mips gpr_defconfig
>
> clang tested configs:
> i386 randconfig-a004-20230109
> x86_64 randconfig-a003-20230109
> x86_64 randconfig-a002-20230109
> x86_64 rhel-8.3-rust
> x86_64 randconfig-a004-20230109
> hexagon randconfig-r045-20230109
> i386 randconfig-a002-20230109
> x86_64 randconfig-a005-20230109
> i386 randconfig-a003-20230109
> x86_64 randconfig-a001-20230109
> arm randconfig-r046-20230109
> riscv randconfig-r042-20230108
> x86_64 randconfig-a006-20230109
> hexagon randconfig-r041-20230108
> i386 randconfig-a006-20230109
> i386 randconfig-a001-20230109
> hexagon randconfig-r041-20230109
> i386 randconfig-a005-20230109
> hexagon randconfig-r045-20230108
> s390 randconfig-r044-20230108
> x86_64 randconfig-k001
>
> --
> 0-DAY CI Kernel Test Service
> https://01.org/lkp
>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 10 Jan 2023 07:48:41 +0200
> From: Sasha Neftin <sasha.neftin@intel.com>
> To: intel-wired-lan@lists.osuosl.org
> Subject: [Intel-wired-lan] [PATCH net v1 1/1] igc: Add ndo_tx_timeout
> support
> Message-ID: <20230110054841.1857688-1-sasha.neftin@intel.com>
>
> On some platforms, 100/1000/2500 speeds seem to have sometimes
> problems
> reporting false positive tx unit hang during stressful UDP traffic. Likely
> other Intel drivers introduce responses to a tx hang. Update the 'tx hang'
> comparator with the comparison of the head and tail of ring pointers and
> restore the tx_timeout_factor to the previous value (one).
>
> This can be test by using netperf or iperf3 applications.
> Example:
> iperf3 -s -p 5001
> iperf3 -c 192.168.0.2 --udp -p 5001 --time 600 -b 0
>
> netserver -p 16604
> netperf -H 192.168.0.2 -l 600 -p 16604 -t UDP_STREAM -- -m 64000
>
> Fixes: b27b8dc77b5e ("igc: Increase timeout value for Speed 100/1000/2500")
> Signed-off-by: Sasha Neftin <sasha.neftin@intel.com>
> ---
> drivers/net/ethernet/intel/igc/igc_main.c | 25 +++++++++++++++++++++--
> 1 file changed, 23 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c
> b/drivers/net/ethernet/intel/igc/igc_main.c
> index 13088b5bcf5b..b1031d5b32bc 100644
> --- a/drivers/net/ethernet/intel/igc/igc_main.c
> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
> @@ -2957,7 +2957,9 @@ static bool igc_clean_tx_irq(struct igc_q_vector
> *q_vector, int napi_budget)
> if (tx_buffer->next_to_watch &&
> time_after(jiffies, tx_buffer->time_stamp +
> (adapter->tx_timeout_factor * HZ)) &&
> - !(rd32(IGC_STATUS) & IGC_STATUS_TXOFF)) {
> + !(rd32(IGC_STATUS) & IGC_STATUS_TXOFF) &&
> + (rd32(IGC_TDH(tx_ring->reg_idx)) !=
> + readl(tx_ring->tail))) {
> /* detected Tx unit hang */
> netdev_err(tx_ring->netdev,
> "Detected Tx Unit Hang\n"
> @@ -5083,6 +5085,24 @@ static int igc_change_mtu(struct net_device
> *netdev, int new_mtu)
> return 0;
> }
>
> +/**
> + * igc_tx_timeout - Respond to a Tx Hang
> + * @netdev: network interface device structure
> + * @txqueue: queue number that timed out
> + **/
> +static void igc_tx_timeout(struct net_device *netdev,
> + unsigned int __always_unused txqueue)
> +{
> + struct igc_adapter *adapter = netdev_priv(netdev);
> + struct igc_hw *hw = &adapter->hw;
> +
> + /* Do the reset outside of interrupt context */
> + adapter->tx_timeout_count++;
> + schedule_work(&adapter->reset_task);
> + wr32(IGC_EICS,
> + (adapter->eims_enable_mask & ~adapter->eims_other));
> +}
> +
> /**
> * igc_get_stats64 - Get System Network Statistics
> * @netdev: network interface device structure
> @@ -5510,7 +5530,7 @@ static void igc_watchdog_task(struct work_struct
> *work)
> case SPEED_100:
> case SPEED_1000:
> case SPEED_2500:
> - adapter->tx_timeout_factor = 7;
> + adapter->tx_timeout_factor = 1;
I was running more test with UDP packets with changing of Qbv Gates for TSN Mode.
The tx timeout factor appears to need to stay at 7 in order to avoid the hanging issue
for some extremely high traffic situation gate switching. Given that it had no impact
on the first head and tail pointer problem as per what you resolved in igc_clean_tx_irq(),
I advise keeping this number.
> break;
> }
>
> @@ -6352,6 +6372,7 @@ static const struct net_device_ops igc_netdev_ops
> = {
> .ndo_set_rx_mode = igc_set_rx_mode,
> .ndo_set_mac_address = igc_set_mac,
> .ndo_change_mtu = igc_change_mtu,
> + .ndo_tx_timeout = igc_tx_timeout,
> .ndo_get_stats64 = igc_get_stats64,
> .ndo_fix_features = igc_fix_features,
> .ndo_set_features = igc_set_features,
> --
> 2.25.1
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan@osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
>
>
> ------------------------------
>
> End of Intel-wired-lan Digest, Vol 405, Issue 6
> ***********************************************
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [Intel-wired-lan] Intel-wired-lan Digest, Vol 405, Issue 6
2023-01-24 14:46 ` [Intel-wired-lan] Intel-wired-lan Digest, Vol 405, Issue 6 Zulkifli, Muhammad Husaini
@ 2023-01-25 7:38 ` Neftin, Sasha
0 siblings, 0 replies; 2+ messages in thread
From: Neftin, Sasha @ 2023-01-25 7:38 UTC (permalink / raw)
To: Zulkifli, Muhammad Husaini, intel-wired-lan@osuosl.org,
Ruinskiy, Dima, Avargil, Raanan, naamax.meir, Meir, NaamaX,
Efrati, Nir, Lifshits, Vitaly
On 1/24/2023 16:46, Zulkifli, Muhammad Husaini wrote:
> Hi Sasha,
>
>> -----Original Message-----
>> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
>> intel-wired-lan-request@osuosl.org
>> Sent: Tuesday, 10 January, 2023 1:49 PM
>> To: intel-wired-lan@osuosl.org
>> Subject: Intel-wired-lan Digest, Vol 405, Issue 6
>>
>> Send Intel-wired-lan mailing list submissions to
>> intel-wired-lan@osuosl.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
>> or, via email, send a message with subject or body 'help' to
>> intel-wired-lan-request@osuosl.org
>>
>> You can reach the person managing the list at
>> intel-wired-lan-owner@osuosl.org
>>
>> When replying, please edit your Subject line so it is more specific than "Re:
>> Contents of Intel-wired-lan digest..."
>>
>>
>> Today's Topics:
>>
>> 1. [PATCH 1/1] ice: WiP support for BIG TCP packets
>> (Pawel Chmielewski)
>> 2. Re: [PATCH 1/1] ice: WiP support for BIG TCP packets
>> (Alexander H Duyck)
>> 3. [tnguy-net-queue:dev-queue] BUILD SUCCESS
>> 4cb425d20a6ddbf9fd40989c31f5c6f8f304dc35 (kernel test robot)
>> 4. [PATCH net v1 1/1] igc: Add ndo_tx_timeout support (Sasha Neftin)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Mon, 9 Jan 2023 17:18:33 +0100
>> From: Pawel Chmielewski <pawel.chmielewski@intel.com>
>> To: netdev@vger.kernel.org
>> Cc: intel-wired-lan@osuosl.org, Pawel Chmielewski
>> <pawel.chmielewski@intel.com>
>> Subject: [Intel-wired-lan] [PATCH 1/1] ice: WiP support for BIG TCP
>> packets
>> Message-ID: <20230109161833.223510-1-pawel.chmielewski@intel.com>
>>
>> This patch is a proof of concept for testing BIG TCP feature in ice driver.
>> Please see letter below.
>>
>> Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
>> ---
>> Hi All
>> I'm writing on the list, as you may be able to provide me some feedback.
>> I want to enable BIG TCP feature in intel ice drive, but I think I'm missing
>> something.
>> In the code itself, I've set 128k as a maximum tso size for the netif, and
>> added stripping the HBH option from the header.
>> For testing purposes, gso_max_size & gro_max_size were set to 128k and
>> mtu to 9000.
>> I've assumed that the ice tso offload will do the rest of the job.
>> However- while running netperf TCP_RR and TCP_STREAM tests, I saw that
>> only up to ~20% of the transmitted test packets have the specified size.
>> Other packets to be transmitted, appear from the stack as splitted.
>>
>> I've been running the following testcases:
>> netperf -t TCP_RR -H 2001:db8:0:f101::1 -- -r80000,80000 -O
>> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
>> netperf -l-1 -t TCP_STREAM -H 2001:db8:0:f101::1 -- -m 128K -O
>> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
>> I suspected a shrinking tcp window size, but sniffing with tcpdump showed
>> rather big scaling factor (usually 128x).
>> Apart from using netperf, I also tried a simple IPv6 user space application
>> (with SO_SNDBUF option set to 192k and TCP_WINDOW_CLAMP to 96k) -
>> similar results.
>>
>> I'd be very grateful for any feedback/suggestions
>>
>> Pawel
>> ---
>> drivers/net/ethernet/intel/ice/ice_main.c | 4 ++++
>> drivers/net/ethernet/intel/ice/ice_txrx.c | 9 +++++++++
>> 2 files changed, 13 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c
>> b/drivers/net/ethernet/intel/ice/ice_main.c
>> index 2b23b4714a26..4e657820e55d 100644
>> --- a/drivers/net/ethernet/intel/ice/ice_main.c
>> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
>> @@ -48,6 +48,8 @@ static DEFINE_IDA(ice_aux_ida);
>> DEFINE_STATIC_KEY_FALSE(ice_xdp_locking_key);
>> EXPORT_SYMBOL(ice_xdp_locking_key);
>>
>> +#define ICE_MAX_TSO_SIZE 131072
>> +
>> /**
>> * ice_hw_to_dev - Get device pointer from the hardware structure
>> * @hw: pointer to the device HW structure @@ -3422,6 +3424,8 @@ static
>> void ice_set_netdev_features(struct net_device *netdev)
>> * be changed at runtime
>> */
>> netdev->hw_features |= NETIF_F_RXFCS;
>> +
>> + netif_set_tso_max_size(netdev, ICE_MAX_TSO_SIZE);
>> }
>>
>> /**
>> diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c
>> b/drivers/net/ethernet/intel/ice/ice_txrx.c
>> index 086f0b3ab68d..7e0ac483cad9 100644
>> --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
>> +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
>> @@ -23,6 +23,9 @@
>> #define FDIR_DESC_RXDID 0x40
>> #define ICE_FDIR_CLEAN_DELAY 10
>>
>> +#define HBH_HDR_SIZE sizeof(struct hop_jumbo_hdr) #define
>> HBH_OFFSET
>> +ETH_HLEN + sizeof(struct ipv6hdr)
>> +
>> /**
>> * ice_prgm_fdir_fltr - Program a Flow Director filter
>> * @vsi: VSI to send dummy packet
>> @@ -2300,6 +2303,12 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct
>> ice_tx_ring *tx_ring)
>>
>> ice_trace(xmit_frame_ring, tx_ring, skb);
>>
>> + if (ipv6_has_hopopt_jumbo(skb)) {
>> + memmove(skb->data + HBH_HDR_SIZE, skb->data,
>> HBH_OFFSET);
>> + __skb_pull(skb, HBH_HDR_SIZE);
>> + skb_reset_mac_header(skb);
>> + }
>> +
>> count = ice_xmit_desc_count(skb);
>> if (ice_chk_linearize(skb, count)) {
>> if (__skb_linearize(skb))
>> --
>> 2.37.3
>>
>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Mon, 09 Jan 2023 10:22:22 -0800
>> From: Alexander H Duyck <alexander.duyck@gmail.com>
>> To: Pawel Chmielewski <pawel.chmielewski@intel.com>,
>> netdev@vger.kernel.org
>> Cc: intel-wired-lan@osuosl.org
>> Subject: Re: [Intel-wired-lan] [PATCH 1/1] ice: WiP support for BIG
>> TCP packets
>> Message-ID:
>> <9b68abc5e8613e02207e9c0c3619b1b07bc5bb8c.camel@gmail.com>
>> Content-Type: text/plain; charset="UTF-8"
>>
>> On Mon, 2023-01-09 at 17:18 +0100, Pawel Chmielewski wrote:
>>> This patch is a proof of concept for testing BIG TCP feature in ice driver.
>>> Please see letter below.
>>>
>>> Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
>>> ---
>>> Hi All
>>> I'm writing on the list, as you may be able to provide me some feedback.
>>> I want to enable BIG TCP feature in intel ice drive, but I think I'm
>>> missing something.
>>> In the code itself, I've set 128k as a maximum tso size for the netif,
>>> and added stripping the HBH option from the header.
>>> For testing purposes, gso_max_size & gro_max_size were set to 128k and
>>> mtu to 9000.
>>> I've assumed that the ice tso offload will do the rest of the job.
>>> However- while running netperf TCP_RR and TCP_STREAM tests,
>>> I saw that only up to ~20% of the transmitted test packets have
>>> the specified size.
>>> Other packets to be transmitted, appear from the stack as splitted.
>>>
>>> I've been running the following testcases:
>>> netperf -t TCP_RR -H 2001:db8:0:f101::1 -- -r80000,80000 -O
>> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
>>> netperf -l-1 -t TCP_STREAM -H 2001:db8:0:f101::1 -- -m 128K -O
>> MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT
>>> I suspected a shrinking tcp window size, but sniffing with tcpdump showed
>> rather big scaling factor (usually 128x).
>>> Apart from using netperf, I also tried a simple IPv6 user space application
>>> (with SO_SNDBUF option set to 192k and TCP_WINDOW_CLAMP to 96k) -
>> similar results.
>>>
>>> I'd be very grateful for any feedback/suggestions
>>>
>>> Pawel
>>> ---
>>> drivers/net/ethernet/intel/ice/ice_main.c | 4 ++++
>>> drivers/net/ethernet/intel/ice/ice_txrx.c | 9 +++++++++
>>> 2 files changed, 13 insertions(+)
>>>
>>> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c
>> b/drivers/net/ethernet/intel/ice/ice_main.c
>>> index 2b23b4714a26..4e657820e55d 100644
>>> --- a/drivers/net/ethernet/intel/ice/ice_main.c
>>> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
>>> @@ -48,6 +48,8 @@ static DEFINE_IDA(ice_aux_ida);
>>> DEFINE_STATIC_KEY_FALSE(ice_xdp_locking_key);
>>> EXPORT_SYMBOL(ice_xdp_locking_key);
>>>
>>> +#define ICE_MAX_TSO_SIZE 131072
>>> +
>>> /**
>>> * ice_hw_to_dev - Get device pointer from the hardware structure
>>> * @hw: pointer to the device HW structure
>>> @@ -3422,6 +3424,8 @@ static void ice_set_netdev_features(struct
>> net_device *netdev)
>>> * be changed at runtime
>>> */
>>> netdev->hw_features |= NETIF_F_RXFCS;
>>> +
>>> + netif_set_tso_max_size(netdev, ICE_MAX_TSO_SIZE);
>>> }
>>>
>>> /**
>>> diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c
>> b/drivers/net/ethernet/intel/ice/ice_txrx.c
>>> index 086f0b3ab68d..7e0ac483cad9 100644
>>> --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
>>> +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
>>> @@ -23,6 +23,9 @@
>>> #define FDIR_DESC_RXDID 0x40
>>> #define ICE_FDIR_CLEAN_DELAY 10
>>>
>>> +#define HBH_HDR_SIZE sizeof(struct hop_jumbo_hdr)
>>> +#define HBH_OFFSET ETH_HLEN + sizeof(struct ipv6hdr)
>>> +
>>> /**
>>> * ice_prgm_fdir_fltr - Program a Flow Director filter
>>> * @vsi: VSI to send dummy packet
>>> @@ -2300,6 +2303,12 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct
>> ice_tx_ring *tx_ring)
>>>
>>> ice_trace(xmit_frame_ring, tx_ring, skb);
>>>
>>> + if (ipv6_has_hopopt_jumbo(skb)) {
>>> + memmove(skb->data + HBH_HDR_SIZE, skb->data,
>> HBH_OFFSET);
>>> + __skb_pull(skb, HBH_HDR_SIZE);
>>> + skb_reset_mac_header(skb);
>>> + }
>>> +
>>> count = ice_xmit_desc_count(skb);
>>> if (ice_chk_linearize(skb, count)) {
>>> if (__skb_linearize(skb))
>>
>> Your removal code here is forgetting to handle the network header. As a
>> result your frames will be pointer mangled in terms of header location.
>>
>> You might be better off using ipv6_hopopt_jumbo_remove() rather than
>> just coding your own bit to remove it.
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Tue, 10 Jan 2023 13:10:09 +0800
>> From: kernel test robot <lkp@intel.com>
>> To: Intel Wired LAN <intel-wired-lan@lists.osuosl.org>
>> Subject: [Intel-wired-lan] [tnguy-net-queue:dev-queue] BUILD SUCCESS
>> 4cb425d20a6ddbf9fd40989c31f5c6f8f304dc35
>> Message-ID: <63bcf331.kDz0I6QwE35Zcf1P%lkp@intel.com>
>> Content-Type: text/plain; charset=us-ascii
>>
>> tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/tnguy/net-
>> queue.git dev-queue
>> branch HEAD: 4cb425d20a6ddbf9fd40989c31f5c6f8f304dc35 ice: fix out-of-
>> bounds KASAN warning in virtchnl
>>
>> elapsed time: 723m
>>
>> configs tested: 84
>> configs skipped: 2
>>
>> The following configs have been built successfully.
>> More configs may be tested in the coming days.
>>
>> gcc tested configs:
>> x86_64 allnoconfig
>> arc defconfig
>> alpha defconfig
>> powerpc allnoconfig
>> um i386_defconfig
>> um x86_64_defconfig
>> alpha allyesconfig
>> s390 allmodconfig
>> m68k allmodconfig
>> arc allyesconfig
>> s390 defconfig
>> m68k allyesconfig
>> x86_64 defconfig
>> s390 allyesconfig
>> i386 defconfig
>> arm defconfig
>> i386 allyesconfig
>> x86_64 randconfig-a011-20230109
>> x86_64 randconfig-a013-20230109
>> i386 randconfig-a011-20230109
>> x86_64 randconfig-a012-20230109
>> sh allmodconfig
>> i386 randconfig-a013-20230109
>> x86_64 rhel-8.3-bpf
>> x86_64 rhel-8.3
>> x86_64 randconfig-a014-20230109
>> x86_64 randconfig-a016-20230109
>> x86_64 randconfig-a015-20230109
>> ia64 allmodconfig
>> x86_64 allyesconfig
>> i386 randconfig-a012-20230109
>> x86_64 rhel-8.3-kselftests
>> x86_64 rhel-8.3-syz
>> x86_64 rhel-8.3-kunit
>> x86_64 rhel-8.3-func
>> i386 randconfig-a014-20230109
>> x86_64 rhel-8.3-kvm
>> i386 randconfig-a016-20230109
>> i386 randconfig-a015-20230109
>> mips allyesconfig
>> powerpc allmodconfig
>> arm64 allyesconfig
>> arm allyesconfig
>> riscv randconfig-r042-20230109
>> s390 randconfig-r044-20230109
>> arm randconfig-r046-20230108
>> arc randconfig-r043-20230108
>> arc randconfig-r043-20230109
>> i386 randconfig-c001
>> arm trizeps4_defconfig
>> arm s3c6400_defconfig
>> sh alldefconfig
>> ia64 defconfig
>> riscv nommu_virt_defconfig
>> riscv rv32_defconfig
>> riscv nommu_k210_defconfig
>> riscv allnoconfig
>> i386 debian-10.3-kselftests
>> i386 debian-10.3
>> m68k sun3_defconfig
>> parisc generic-32bit_defconfig
>> sh sh7710voipgw_defconfig
>> mips gpr_defconfig
>>
>> clang tested configs:
>> i386 randconfig-a004-20230109
>> x86_64 randconfig-a003-20230109
>> x86_64 randconfig-a002-20230109
>> x86_64 rhel-8.3-rust
>> x86_64 randconfig-a004-20230109
>> hexagon randconfig-r045-20230109
>> i386 randconfig-a002-20230109
>> x86_64 randconfig-a005-20230109
>> i386 randconfig-a003-20230109
>> x86_64 randconfig-a001-20230109
>> arm randconfig-r046-20230109
>> riscv randconfig-r042-20230108
>> x86_64 randconfig-a006-20230109
>> hexagon randconfig-r041-20230108
>> i386 randconfig-a006-20230109
>> i386 randconfig-a001-20230109
>> hexagon randconfig-r041-20230109
>> i386 randconfig-a005-20230109
>> hexagon randconfig-r045-20230108
>> s390 randconfig-r044-20230108
>> x86_64 randconfig-k001
>>
>> --
>> 0-DAY CI Kernel Test Service
>> https://01.org/lkp
>>
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Tue, 10 Jan 2023 07:48:41 +0200
>> From: Sasha Neftin <sasha.neftin@intel.com>
>> To: intel-wired-lan@lists.osuosl.org
>> Subject: [Intel-wired-lan] [PATCH net v1 1/1] igc: Add ndo_tx_timeout
>> support
>> Message-ID: <20230110054841.1857688-1-sasha.neftin@intel.com>
>>
>> On some platforms, 100/1000/2500 speeds seem to have sometimes
>> problems
>> reporting false positive tx unit hang during stressful UDP traffic. Likely
>> other Intel drivers introduce responses to a tx hang. Update the 'tx hang'
>> comparator with the comparison of the head and tail of ring pointers and
>> restore the tx_timeout_factor to the previous value (one).
>>
>> This can be test by using netperf or iperf3 applications.
>> Example:
>> iperf3 -s -p 5001
>> iperf3 -c 192.168.0.2 --udp -p 5001 --time 600 -b 0
>>
>> netserver -p 16604
>> netperf -H 192.168.0.2 -l 600 -p 16604 -t UDP_STREAM -- -m 64000
>>
>> Fixes: b27b8dc77b5e ("igc: Increase timeout value for Speed 100/1000/2500")
>> Signed-off-by: Sasha Neftin <sasha.neftin@intel.com>
>> ---
>> drivers/net/ethernet/intel/igc/igc_main.c | 25 +++++++++++++++++++++--
>> 1 file changed, 23 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c
>> b/drivers/net/ethernet/intel/igc/igc_main.c
>> index 13088b5bcf5b..b1031d5b32bc 100644
>> --- a/drivers/net/ethernet/intel/igc/igc_main.c
>> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
>> @@ -2957,7 +2957,9 @@ static bool igc_clean_tx_irq(struct igc_q_vector
>> *q_vector, int napi_budget)
>> if (tx_buffer->next_to_watch &&
>> time_after(jiffies, tx_buffer->time_stamp +
>> (adapter->tx_timeout_factor * HZ)) &&
>> - !(rd32(IGC_STATUS) & IGC_STATUS_TXOFF)) {
>> + !(rd32(IGC_STATUS) & IGC_STATUS_TXOFF) &&
>> + (rd32(IGC_TDH(tx_ring->reg_idx)) !=
>> + readl(tx_ring->tail))) {
>> /* detected Tx unit hang */
>> netdev_err(tx_ring->netdev,
>> "Detected Tx Unit Hang\n"
>> @@ -5083,6 +5085,24 @@ static int igc_change_mtu(struct net_device
>> *netdev, int new_mtu)
>> return 0;
>> }
>>
>> +/**
>> + * igc_tx_timeout - Respond to a Tx Hang
>> + * @netdev: network interface device structure
>> + * @txqueue: queue number that timed out
>> + **/
>> +static void igc_tx_timeout(struct net_device *netdev,
>> + unsigned int __always_unused txqueue)
>> +{
>> + struct igc_adapter *adapter = netdev_priv(netdev);
>> + struct igc_hw *hw = &adapter->hw;
>> +
>> + /* Do the reset outside of interrupt context */
>> + adapter->tx_timeout_count++;
>> + schedule_work(&adapter->reset_task);
>> + wr32(IGC_EICS,
>> + (adapter->eims_enable_mask & ~adapter->eims_other));
>> +}
>> +
>> /**
>> * igc_get_stats64 - Get System Network Statistics
>> * @netdev: network interface device structure
>> @@ -5510,7 +5530,7 @@ static void igc_watchdog_task(struct work_struct
>> *work)
>> case SPEED_100:
>> case SPEED_1000:
>> case SPEED_2500:
>> - adapter->tx_timeout_factor = 7;
>> + adapter->tx_timeout_factor = 1;
>
> I was running more test with UDP packets with changing of Qbv Gates for TSN Mode.
> The tx timeout factor appears to need to stay at 7 in order to avoid the hanging issue
> for some extremely high traffic situation gate switching. Given that it had no impact
> on the first head and tail pointer problem as per what you resolved in igc_clean_tx_irq(),
> I advise keeping this number.
Hello Muhammad,
I have a concern. Increasing the tx_timeout_factor will delay the
'.ndo_tx_timeout' report to the 'netdev' and lead to a stuck controller.
I saw it on production boards (few CRB board with iperf3 tests).Let's
leave it 1 for now. I would suggest understanding your TSN corner cases
and directly discussing the solution with Dima and me(Intel).
Sasha
>
>> break;
>> }
>>
>> @@ -6352,6 +6372,7 @@ static const struct net_device_ops igc_netdev_ops
>> = {
>> .ndo_set_rx_mode = igc_set_rx_mode,
>> .ndo_set_mac_address = igc_set_mac,
>> .ndo_change_mtu = igc_change_mtu,
>> + .ndo_tx_timeout = igc_tx_timeout,
>> .ndo_get_stats64 = igc_get_stats64,
>> .ndo_fix_features = igc_fix_features,
>> .ndo_set_features = igc_set_features,
>> --
>> 2.25.1
>>
>>
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> Intel-wired-lan mailing list
>> Intel-wired-lan@osuosl.org
>> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
>>
>>
>> ------------------------------
>>
>> End of Intel-wired-lan Digest, Vol 405, Issue 6
>> ***********************************************
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan@osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2023-01-25 7:39 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <mailman.5156.1673329730.179342.intel-wired-lan@osuosl.org>
2023-01-24 14:46 ` [Intel-wired-lan] Intel-wired-lan Digest, Vol 405, Issue 6 Zulkifli, Muhammad Husaini
2023-01-25 7:38 ` Neftin, Sasha
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox