From: Leon Romanovsky <leon@kernel.org>
To: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Jakub Kicinski <kuba@kernel.org>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
netdev@vger.kernel.org, Paolo Abeni <pabeni@redhat.com>,
Raed Salem <raeds@nvidia.com>, Saeed Mahameed <saeedm@nvidia.com>,
Bharat Bhushan <bbhushan2@marvell.com>
Subject: Re: [PATCH RFC xfrm-next v3 0/8] Extend XFRM core to allow full offload configuration
Date: Mon, 26 Sep 2022 09:55:45 +0300 [thread overview]
Message-ID: <YzFM8RF0suHc4cKI@unreal> (raw)
In-Reply-To: <20220925094039.GV2602992@gauss3.secunet.de>
On Sun, Sep 25, 2022 at 11:40:39AM +0200, Steffen Klassert wrote:
> On Wed, Sep 21, 2022 at 08:37:06PM +0300, Leon Romanovsky wrote:
> > On Wed, Sep 21, 2022 at 07:59:27AM -0700, Jakub Kicinski wrote:
> > > On Thu, 8 Sep 2022 12:56:16 +0300 Leon Romanovsky wrote:
> > > > I have TX traces too and can add if RX are not sufficient.
> > >
> > > The perf trace is good, but for those of us not intimately familiar
> > > with xfrm, could you provide some analysis here?
> >
> > The perf trace presented is for RX path of IPsec crypto offload mode. In that
> > mode, decrypted packet enters the netdev stack to perform various XFRM specific
> > checks.
>
> Can you provide the perf traces and analysis for the TX side too? That
> would be interesting in particular, because the policy and state lookups
> there happen still in software.
Single core TX (crypto mode) from the same run:
Please notice that it is not really bottleneck, probably RX caused to the situation
where TX was not executed enough. It is also lighter path than RX.
# Children Self Samples Command Shared Object Symbol
# ........ ........ ............ ............... .................. ..............................................
#
86.58% 0.00% 0 swapper [kernel.vmlinux] [k] secondary_startup_64_no_verify
|
---secondary_startup_64_no_verify
start_secondary
cpu_startup_entry
do_idle
|
--86.37%--cpu_idle_poll
|
--24.53%--asm_common_interrupt
|
--24.48%--common_interrupt
|
|--23.47%--irq_exit_rcu
| |
| --23.23%--do_softirq_own_stack
| |
| --23.17%--asm_call_irq_on_stack
| __do_softirq
| |
| |--22.23%--net_rx_action
| | |
| | |--20.17%--gro_cell_poll
| | | |
| | | --20.02%--napi_complete_done
| | | |
| | | --19.98%--gro_normal_list.part.154
| | | |
| | | --19.96%--netif_receive_skb_list_internal
| | | |
| | | --19.89%--__netif_receive_skb_list_core
| | | |
| | | --19.77%--ip_list_rcv
| | | |
| | | --19.67%--ip_sublist_rcv
| | | |
| | | --19.56%--ip_sublist_rcv_finish
| | | |
| | | --19.54%--ip_local_deliver
| | | |
| | | --19.49%--ip_local_deliver_finish
| | | |
| | | --19.47%--ip_protocol_deliver_rcu
| | | |
| | | --19.43%--tcp_v4_rcv
| | | |
| | | --18.87%--tcp_v4_do_rcv
| | | |
| | | --18.83%--tcp_rcv_established
| | | |
| | | |--16.41%--__tcp_push_pending_frames
| | | | |
| | | | --16.38%--tcp_write_xmit
| | | | |
| | | | |--6.35%--tcp_event_new_data_sent
| | | | | |
| | | | | --6.22%--sk_reset_timer
| | | | | |
| | | | | --6.21%--mod_timer
| | | | | |
| | | | | --6.10%--get_nohz_timer_target
| | | | | |
| | | | | --1.87%--cpumask_next_and
| | | | | |
| | | | | --1.07%--_find_next_bit.constprop.1
| | | | |
| | | | |--5.50%--tcp_schedule_loss_probe
| | | | | |
| | | | | --5.49%--sk_reset_timer
| | | | | mod_timer
| | | | | |
| | | | | --5.43%--get_nohz_timer_target
| | | | | |
| | | | | --1.37%--cpumask_next_and
| | | | | |
| | | | | --0.71%--_find_next_bit.constprop.1
| | | | |
| | | | --4.31%--__tcp_transmit_skb
| | | | |
| | | | --3.87%--__ip_queue_xmit
| | | | |
| | | | --3.54%--xfrm4_output
| | | | |
| | | | --3.26%--xfrm_output_resume
| | | | |
| | | | --2.88%--ip_output
| | | | |
| | | | --2.78%--ip_finish_output2
| | | | |
| | | | --2.73%--__dev_queue_xmit
| | | | |
| | | | --2.49%--sch_direct_xmit
| | | | |
| | | | |--1.50%--validate_xmit_skb_list
| | | | | | |
| | | | | --1.32%--validate_xmit_skb
| | | | | | |
| | | | | --1.06%--__skb_gso_segment
| | | | | | |
| | | | | --1.04%--skb_mac_gso_segment
| | | | | |
| | | | --1.02%--inet_gso_segment
| | | | | |
| | | | --0.93%--esp4_gso_segment
| | | | | |
| | | | --0.86%--tcp_gso_segment
| | | | |
| | | --0.78%--skb_segment
| | | | |
| | | | --0.77%--dev_hard_start_xmit
| | | | | |
| | | | --0.75%--mlx5e_xmit
| | | |
| | | --1.87%--tcp_ack
| | | |
| | | --1.66%--tcp_clean_rtx_queue
| | | |
| | | --1.35%--__kfree_skb
| | | |
| | | --1.21%--skb_release_data
| | |
| | --1.92%--mlx5e_napi_poll
| | |
| | --1.38%--mlx5e_poll_rx_cq
| | |
| | --1.33%--mlx5e_handle_rx_cqe
| | |
| | --0.53%--napi_gro_receive
| | |
| | --0.52%--dev_gro_receive
| |
| --0.77%--tasklet_action_common.isra.17
|
--0.80%--asm_call_irq_on_stack
|
--0.78%--handle_edge_irq
|
--0.74%--handle_irq_event
|
--0.71%--handle_irq_event_percpu
|
--0.64%--__handle_irq_event_percpu
|
--0.60%--mlx5_irq_int_handler
|
--0.58%--atomic_notifier_call_chain
|
--0.57%--mlx5_eq_comp_int
next prev parent reply other threads:[~2022-09-26 6:55 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-09-04 13:15 [PATCH RFC xfrm-next v3 0/8] Extend XFRM core to allow full offload configuration Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 1/8] xfrm: add new full offload flag Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 2/8] xfrm: allow state full offload mode Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 3/8] xfrm: add an interface to offload policy Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 4/8] xfrm: add TX datapath support for IPsec full offload mode Leon Romanovsky 2022-09-25 9:16 ` Steffen Klassert 2022-09-26 6:06 ` Leon Romanovsky 2022-09-27 5:04 ` Steffen Klassert 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 5/8] xfrm: add RX datapath protection " Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 6/8] xfrm: enforce separation between priorities of HW/SW policies Leon Romanovsky 2022-09-25 9:34 ` Steffen Klassert 2022-09-26 6:38 ` Leon Romanovsky 2022-09-27 5:48 ` Steffen Klassert 2022-09-27 10:21 ` Leon Romanovsky 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 7/8] xfrm: add support to HW update soft and hard limits Leon Romanovsky 2022-09-25 9:20 ` Steffen Klassert 2022-09-26 6:07 ` Leon Romanovsky 2022-09-27 5:49 ` Steffen Klassert 2022-09-04 13:15 ` [PATCH RFC xfrm-next v3 8/8] xfrm: document IPsec full offload mode Leon Romanovsky 2022-09-04 13:19 ` [PATCH RFC xfrm-next v3 0/8] Extend XFRM core to allow full offload configuration Leon Romanovsky 2022-09-08 9:56 ` Leon Romanovsky 2022-09-21 14:59 ` Jakub Kicinski 2022-09-21 17:37 ` Leon Romanovsky 2022-09-25 9:40 ` Steffen Klassert 2022-09-26 6:55 ` Leon Romanovsky [this message] 2022-09-27 5:59 ` Steffen Klassert 2022-09-27 10:02 ` Leon Romanovsky 2022-09-19 9:31 ` Leon Romanovsky 2022-09-22 7:17 ` Leon Romanovsky 2022-09-22 7:35 ` Steffen Klassert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YzFM8RF0suHc4cKI@unreal \
--to=leon@kernel.org \
--cc=bbhushan2@marvell.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=herbert@gondor.apana.org.au \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=raeds@nvidia.com \
--cc=saeedm@nvidia.com \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).