From: Leon Romanovsky <leon@kernel.org>
To: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Jakub Kicinski <kuba@kernel.org>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
netdev@vger.kernel.org, Paolo Abeni <pabeni@redhat.com>,
Raed Salem <raeds@nvidia.com>, Saeed Mahameed <saeedm@nvidia.com>,
Bharat Bhushan <bbhushan2@marvell.com>
Subject: Re: [PATCH RFC xfrm-next v3 0/8] Extend XFRM core to allow full offload configuration
Date: Mon, 26 Sep 2022 09:55:45 +0300 [thread overview]
Message-ID: <YzFM8RF0suHc4cKI@unreal> (raw)
In-Reply-To: <20220925094039.GV2602992@gauss3.secunet.de>
On Sun, Sep 25, 2022 at 11:40:39AM +0200, Steffen Klassert wrote:
> On Wed, Sep 21, 2022 at 08:37:06PM +0300, Leon Romanovsky wrote:
> > On Wed, Sep 21, 2022 at 07:59:27AM -0700, Jakub Kicinski wrote:
> > > On Thu, 8 Sep 2022 12:56:16 +0300 Leon Romanovsky wrote:
> > > > I have TX traces too and can add if RX are not sufficient.
> > >
> > > The perf trace is good, but for those of us not intimately familiar
> > > with xfrm, could you provide some analysis here?
> >
> > The perf trace presented is for RX path of IPsec crypto offload mode. In that
> > mode, decrypted packet enters the netdev stack to perform various XFRM specific
> > checks.
>
> Can you provide the perf traces and analysis for the TX side too? That
> would be interesting in particular, because the policy and state lookups
> there happen still in software.
Single core TX (crypto mode) from the same run:
Please notice that it is not really bottleneck, probably RX caused to the situation
where TX was not executed enough. It is also lighter path than RX.
# Children Self Samples Command Shared Object Symbol
# ........ ........ ............ ............... .................. ..............................................
#
86.58% 0.00% 0 swapper [kernel.vmlinux] [k] secondary_startup_64_no_verify
|
---secondary_startup_64_no_verify
start_secondary
cpu_startup_entry
do_idle
|
--86.37%--cpu_idle_poll
|
--24.53%--asm_common_interrupt
|
--24.48%--common_interrupt
|
|--23.47%--irq_exit_rcu
| |
| --23.23%--do_softirq_own_stack
| |
| --23.17%--asm_call_irq_on_stack
| __do_softirq
| |
| |--22.23%--net_rx_action
| | |
| | |--20.17%--gro_cell_poll
| | | |
| | | --20.02%--napi_complete_done
| | | |
| | | --19.98%--gro_normal_list.part.154
| | | |
| | | --19.96%--netif_receive_skb_list_internal
| | | |
| | | --19.89%--__netif_receive_skb_list_core
| | | |
| | | --19.77%--ip_list_rcv
| | | |
| | | --19.67%--ip_sublist_rcv
| | | |
| | | --19.56%--ip_sublist_rcv_finish
| | | |
| | | --19.54%--ip_local_deliver
| | | |
| | | --19.49%--ip_local_deliver_finish
| | | |
| | | --19.47%--ip_protocol_deliver_rcu
| | | |
| | | --19.43%--tcp_v4_rcv
| | | |
| | | --18.87%--tcp_v4_do_rcv
| | | |
| | | --18.83%--tcp_rcv_established
| | | |
| | | |--16.41%--__tcp_push_pending_frames
| | | | |
| | | | --16.38%--tcp_write_xmit
| | | | |
| | | | |--6.35%--tcp_event_new_data_sent
| | | | | |
| | | | | --6.22%--sk_reset_timer
| | | | | |
| | | | | --6.21%--mod_timer
| | | | | |
| | | | | --6.10%--get_nohz_timer_target
| | | | | |
| | | | | --1.87%--cpumask_next_and
| | | | | |
| | | | | --1.07%--_find_next_bit.constprop.1
| | | | |
| | | | |--5.50%--tcp_schedule_loss_probe
| | | | | |
| | | | | --5.49%--sk_reset_timer
| | | | | mod_timer
| | | | | |
| | | | | --5.43%--get_nohz_timer_target
| | | | | |
| | | | | --1.37%--cpumask_next_and
| | | | | |
| | | | | --0.71%--_find_next_bit.constprop.1
| | | | |
| | | | --4.31%--__tcp_transmit_skb
| | | | |
| | | | --3.87%--__ip_queue_xmit
| | | | |
| | | | --3.54%--xfrm4_output
| | | | |
| | | | --3.26%--xfrm_output_resume
| | | | |
| | | | --2.88%--ip_output
| | | | |
| | | | --2.78%--ip_finish_output2
| | | | |
| | | | --2.73%--__dev_queue_xmit
| | | | |
| | | | --2.49%--sch_direct_xmit
| | | | |
| | | | |--1.50%--validate_xmit_skb_list
| | | | | | |
| | | | | --1.32%--validate_xmit_skb
| | | | | | |
| | | | | --1.06%--__skb_gso_segment
| | | | | | |
| | | | | --1.04%--skb_mac_gso_segment
| | | | | |
| | | | --1.02%--inet_gso_segment
| | | | | |
| | | | --0.93%--esp4_gso_segment
| | | | | |
| | | | --0.86%--tcp_gso_segment
| | | | |
| | | --0.78%--skb_segment
| | | | |
| | | | --0.77%--dev_hard_start_xmit
| | | | | |
| | | | --0.75%--mlx5e_xmit
| | | |
| | | --1.87%--tcp_ack
| | | |
| | | --1.66%--tcp_clean_rtx_queue
| | | |
| | | --1.35%--__kfree_skb
| | | |
| | | --1.21%--skb_release_data
| | |
| | --1.92%--mlx5e_napi_poll
| | |
| | --1.38%--mlx5e_poll_rx_cq
| | |
| | --1.33%--mlx5e_handle_rx_cqe
| | |
| | --0.53%--napi_gro_receive
| | |
| | --0.52%--dev_gro_receive
| |
| --0.77%--tasklet_action_common.isra.17
|
--0.80%--asm_call_irq_on_stack
|
--0.78%--handle_edge_irq
|
--0.74%--handle_irq_event
|
--0.71%--handle_irq_event_percpu
|
--0.64%--__handle_irq_event_percpu
|
--0.60%--mlx5_irq_int_handler
|
--0.58%--atomic_notifier_call_chain
|
--0.57%--mlx5_eq_comp_int
next prev parent reply other threads:[~2022-09-26 6:55 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-09-04 13:15 [PATCH RFC xfrm-next v3 0/8] Extend XFRM core to allow full offload configuration Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 1/8] xfrm: add new full offload flag Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 2/8] xfrm: allow state full offload mode Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 3/8] xfrm: add an interface to offload policy Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 4/8] xfrm: add TX datapath support for IPsec full offload mode Leon Romanovsky 2022-09-25 9:16 ` Steffen Klassert 2022-09-26 6:06 ` Leon Romanovsky 2022-09-27 5:04 ` Steffen Klassert 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 5/8] xfrm: add RX datapath protection " Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 6/8] xfrm: enforce separation between priorities of HW/SW policies Leon Romanovsky 2022-09-25 9:34 ` Steffen Klassert 2022-09-26 6:38 ` Leon Romanovsky 2022-09-27 5:48 ` Steffen Klassert 2022-09-27 10:21 ` Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 7/8] xfrm: add support to HW update soft and hard limits Leon Romanovsky 2022-09-25 9:20 ` Steffen Klassert 2022-09-26 6:07 ` Leon Romanovsky 2022-09-27 5:49 ` Steffen Klassert 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 8/8] xfrm: document IPsec full offload mode Leon Romanovsky 2022-09-04 13:19 ` [PATCH RFC xfrm-next v3 0/8] Extend XFRM core to allow full offload configuration Leon Romanovsky 2022-09-08 9:56 ` Leon Romanovsky 2022-09-21 14:59 ` Jakub Kicinski 2022-09-21 17:37 ` Leon Romanovsky 2022-09-25 9:40 ` Steffen Klassert 2022-09-26 6:55 ` Leon Romanovsky [this message] 2022-09-27 5:59 ` Steffen Klassert 2022-09-27 10:02 ` Leon Romanovsky 2022-09-19 9:31 ` Leon Romanovsky 2022-09-22 7:17 ` Leon Romanovsky 2022-09-22 7:35 ` Steffen Klassert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YzFM8RF0suHc4cKI@unreal \
--to=leon@kernel.org \
--cc=bbhushan2@marvell.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=herbert@gondor.apana.org.au \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=raeds@nvidia.com \
--cc=saeedm@nvidia.com \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.