From: "Kiyanovski, Arthur" <akiyano@amazon.com>
To: "Caleb Sander Mateos" <csander@purestorage.com>,
"Andrew Lunn" <andrew+netdev@lunn.ch>,
"AngeloGioacchino Del Regno"
<angelogioacchino.delregno@collabora.com>,
"Brett Creeley" <brett.creeley@amd.com>,
"Broadcom internal kernel review list"
<bcm-kernel-feedback-list@broadcom.com>,
"Christophe Leroy" <christophe.leroy@csgroup.eu>,
"Claudiu Manoil" <claudiu.manoil@nxp.com>,
"Arinzon, David" <darinzon@amazon.com>,
"David S. Miller" <davem@davemloft.net>,
"Doug Berger" <opendmb@gmail.com>,
"Eric Dumazet" <edumazet@google.com>,
"Eugenio Pérez" <eperezma@redhat.com>,
"Felix Fietkau" <nbd@nbd.name>,
"Florian Fainelli" <florian.fainelli@broadcom.com>,
"Geetha sowjanya" <gakula@marvell.com>,
hariprasad <hkelam@marvell.com>,
"Jakub Kicinski" <kuba@kernel.org>,
"Jason Wang" <jasowang@redhat.com>,
"Jonathan Corbet" <corbet@lwn.net>,
"Leon Romanovsky" <leon@kernel.org>,
"Lorenzo Bianconi" <lorenzo@kernel.org>,
"Louis Peens" <louis.peens@corigine.com>,
"Mark Lee" <Mark-MC.Lee@mediatek.com>,
"Matthias Brugger" <matthias.bgg@gmail.com>,
"Michael Chan" <michael.chan@broadcom.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Dagan, Noam" <ndagan@amazon.com>,
"Paolo Abeni" <pabeni@redhat.com>,
"Przemek Kitszel" <przemyslaw.kitszel@intel.com>,
"Roy Pledge" <Roy.Pledge@nxp.com>,
"Bshara, Saeed" <saeedb@amazon.com>,
"Saeed Mahameed" <saeedm@nvidia.com>,
"Sean Wang" <sean.wang@mediatek.com>,
"Shannon Nelson" <shannon.nelson@amd.com>,
"Agroskin, Shay" <shayagr@amazon.com>,
"Simon Horman" <horms@kernel.org>,
"Subbaraya Sundeep" <sbhatta@marvell.com>,
"Sunil Goutham" <sgoutham@marvell.com>,
"Tal Gilboa" <talgi@nvidia.com>,
"Tariq Toukan" <tariqt@nvidia.com>,
"Tony Nguyen" <anthony.l.nguyen@intel.com>,
"Vladimir Oltean" <vladimir.oltean@nxp.com>,
"Xuan Zhuo" <xuanzhuo@linux.alibaba.com>
Cc: "intel-wired-lan@lists.osuosl.org"
<intel-wired-lan@lists.osuosl.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-mediatek@lists.infradead.org"
<linux-mediatek@lists.infradead.org>,
"linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"oss-drivers@corigine.com" <oss-drivers@corigine.com>,
"virtualization@lists.linux.dev" <virtualization@lists.linux.dev>
Subject: Re: [Intel-wired-lan] [resend PATCH 2/2] dim: pass dim_sample to net_dim() by reference
Date: Thu, 31 Oct 2024 18:28:19 +0000 [thread overview]
Message-ID: <7f494c4ae5a041fbafa4059e85431857@amazon.com> (raw)
In-Reply-To: <20241031002326.3426181-2-csander@purestorage.com>
> -----Original Message-----
> From: Caleb Sander Mateos <csander@purestorage.com>
> Sent: Wednesday, October 30, 2024 5:23 PM
>
> net_dim() is currently passed a struct dim_sample argument by value.
> struct dim_sample is 24 bytes. Since this is greater 16 bytes, x86-64 passes it
> on the stack. All callers have already initialized dim_sample on the stack, so
> passing it by value requires pushing a duplicated copy to the stack. Either
> witing to the stack and immediately reading it, or perhaps dereferencing
> addresses relative to the stack pointer in a chain of push instructions, seems
> to perform quite poorly.
>
> In a heavy TCP workload, mlx5e_handle_rx_dim() consumes 3% of CPU time,
> 94% of which is attributed to the first push instruction to copy dim_sample on
> the stack for the call to net_dim():
> // Call ktime_get()
> 0.26 |4ead2: call 4ead7 <mlx5e_handle_rx_dim+0x47>
> // Pass the address of struct dim in %rdi
> |4ead7: lea 0x3d0(%rbx),%rdi
> // Set dim_sample.pkt_ctr
> |4eade: mov %r13d,0x8(%rsp)
> // Set dim_sample.byte_ctr
> |4eae3: mov %r12d,0xc(%rsp)
> // Set dim_sample.event_ctr
> 0.15 |4eae8: mov %bp,0x10(%rsp)
> // Duplicate dim_sample on the stack
> 94.16 |4eaed: push 0x10(%rsp)
> 2.79 |4eaf1: push 0x10(%rsp)
> 0.07 |4eaf5: push %rax
> // Call net_dim()
> 0.21 |4eaf6: call 4eafb <mlx5e_handle_rx_dim+0x6b>
>
> To allow the caller to reuse the struct dim_sample already on the stack, pass
> the struct dim_sample by reference to net_dim().
>
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> ---
Thank you for this patch.
For the ENA part:
Reviewed-by: Arthur Kiyanovski <akiyano@amazon.com>
Thanks,
Arthur
WARNING: multiple messages have this Message-ID (diff)
From: "Kiyanovski, Arthur" <akiyano@amazon.com>
To: "Caleb Sander Mateos" <csander@purestorage.com>,
"Andrew Lunn" <andrew+netdev@lunn.ch>,
"AngeloGioacchino Del Regno"
<angelogioacchino.delregno@collabora.com>,
"Brett Creeley" <brett.creeley@amd.com>,
"Broadcom internal kernel review list"
<bcm-kernel-feedback-list@broadcom.com>,
"Christophe Leroy" <christophe.leroy@csgroup.eu>,
"Claudiu Manoil" <claudiu.manoil@nxp.com>,
"Arinzon, David" <darinzon@amazon.com>,
"David S. Miller" <davem@davemloft.net>,
"Doug Berger" <opendmb@gmail.com>,
"Eric Dumazet" <edumazet@google.com>,
"Eugenio Pérez" <eperezma@redhat.com>,
"Felix Fietkau" <nbd@nbd.name>,
"Florian Fainelli" <florian.fainelli@broadcom.com>,
"Geetha sowjanya" <gakula@marvell.com>,
hariprasad <hkelam@marvell.com>,
"Jakub Kicinski" <kuba@kernel.org>,
"Jason Wang" <jasowang@redhat.com>,
"Jonathan Corbet" <corbet@lwn.net>,
"Leon Romanovsky" <leon@kernel.org>,
"Lorenzo Bianconi" <lorenzo@kernel.org>,
"Louis Peens" <louis.peens@corigine.com>,
"Mark Lee" <Mark-MC.Lee@mediatek.com>,
"Matthias Brugger" <matthias.bgg@gmail.com>,
"Michael Chan" <michael.chan@broadcom.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Dagan, Noam" <ndagan@amazon.com>,
"Paolo Abeni" <pabeni@redhat.com>,
"Przemek Kitszel" <przemyslaw.kitszel@intel.com>,
"Roy Pledge" <Roy.Pledge@nxp.com>,
"Bshara, Saeed" <saeedb@amazon.com>,
"Saeed Mahameed" <saeedm@nvidia.com>,
"Sean Wang" <sean.wang@mediatek.com>,
"Shannon Nelson" <shannon.nelson@amd.com>,
"Agroskin, Shay" <shayagr@amazon.com>,
"Simon Horman" <horms@kernel.org>,
"Subbaraya Sundeep" <sbhatta@marvell.com>,
"Sunil Goutham" <sgoutham@marvell.com>,
"Tal Gilboa" <talgi@nvidia.com>,
"Tariq Toukan" <tariqt@nvidia.com>,
"Tony Nguyen" <anthony.l.nguyen@intel.com>,
"Vladimir Oltean" <vladimir.oltean@nxp.com>,
"Xuan Zhuo" <xuanzhuo@linux.alibaba.com>
Cc: "intel-wired-lan@lists.osuosl.org"
<intel-wired-lan@lists.osuosl.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-mediatek@lists.infradead.org"
<linux-mediatek@lists.infradead.org>,
"linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"oss-drivers@corigine.com" <oss-drivers@corigine.com>,
"virtualization@lists.linux.dev" <virtualization@lists.linux.dev>
Subject: RE: [resend PATCH 2/2] dim: pass dim_sample to net_dim() by reference
Date: Thu, 31 Oct 2024 18:28:19 +0000 [thread overview]
Message-ID: <7f494c4ae5a041fbafa4059e85431857@amazon.com> (raw)
In-Reply-To: <20241031002326.3426181-2-csander@purestorage.com>
> -----Original Message-----
> From: Caleb Sander Mateos <csander@purestorage.com>
> Sent: Wednesday, October 30, 2024 5:23 PM
>
> net_dim() is currently passed a struct dim_sample argument by value.
> struct dim_sample is 24 bytes. Since this is greater 16 bytes, x86-64 passes it
> on the stack. All callers have already initialized dim_sample on the stack, so
> passing it by value requires pushing a duplicated copy to the stack. Either
> witing to the stack and immediately reading it, or perhaps dereferencing
> addresses relative to the stack pointer in a chain of push instructions, seems
> to perform quite poorly.
>
> In a heavy TCP workload, mlx5e_handle_rx_dim() consumes 3% of CPU time,
> 94% of which is attributed to the first push instruction to copy dim_sample on
> the stack for the call to net_dim():
> // Call ktime_get()
> 0.26 |4ead2: call 4ead7 <mlx5e_handle_rx_dim+0x47>
> // Pass the address of struct dim in %rdi
> |4ead7: lea 0x3d0(%rbx),%rdi
> // Set dim_sample.pkt_ctr
> |4eade: mov %r13d,0x8(%rsp)
> // Set dim_sample.byte_ctr
> |4eae3: mov %r12d,0xc(%rsp)
> // Set dim_sample.event_ctr
> 0.15 |4eae8: mov %bp,0x10(%rsp)
> // Duplicate dim_sample on the stack
> 94.16 |4eaed: push 0x10(%rsp)
> 2.79 |4eaf1: push 0x10(%rsp)
> 0.07 |4eaf5: push %rax
> // Call net_dim()
> 0.21 |4eaf6: call 4eafb <mlx5e_handle_rx_dim+0x6b>
>
> To allow the caller to reuse the struct dim_sample already on the stack, pass
> the struct dim_sample by reference to net_dim().
>
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> ---
Thank you for this patch.
For the ENA part:
Reviewed-by: Arthur Kiyanovski <akiyano@amazon.com>
Thanks,
Arthur
next prev parent reply other threads:[~2024-11-01 15:30 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-10-31 0:23 [Intel-wired-lan] [resend PATCH 1/2] dim: make dim_calc_stats() inputs const pointers Caleb Sander Mateos
2024-10-31 0:23 ` Caleb Sander Mateos
2024-10-31 0:23 ` [Intel-wired-lan] [resend PATCH 2/2] dim: pass dim_sample to net_dim() by reference Caleb Sander Mateos
2024-10-31 0:23 ` Caleb Sander Mateos
2024-10-31 12:49 ` [Intel-wired-lan] " Louis Peens
2024-10-31 12:49 ` Louis Peens
2024-10-31 17:19 ` [Intel-wired-lan] " Caleb Sander
2024-10-31 17:19 ` Caleb Sander
2024-11-01 8:54 ` [Intel-wired-lan] " Louis Peens
2024-11-01 8:54 ` Louis Peens
2024-11-01 14:11 ` [Intel-wired-lan] " Vladimir Oltean
2024-11-01 14:11 ` Vladimir Oltean
2024-10-31 16:48 ` [Intel-wired-lan] " Vladimir Oltean
2024-10-31 16:48 ` Vladimir Oltean
2024-10-31 16:57 ` [Intel-wired-lan] " Nelson, Shannon
2024-10-31 16:57 ` Nelson, Shannon
2024-10-31 17:17 ` [Intel-wired-lan] " Florian Fainelli
2024-10-31 17:17 ` Florian Fainelli
2024-10-31 18:28 ` Kiyanovski, Arthur [this message]
2024-10-31 18:28 ` Kiyanovski, Arthur
2024-11-03 20:21 ` [Intel-wired-lan] " Jakub Kicinski
2024-11-03 20:21 ` Jakub Kicinski
2024-11-03 22:50 ` [Intel-wired-lan] " Caleb Sander
2024-11-03 22:50 ` Caleb Sander
2024-11-04 2:52 ` [Intel-wired-lan] " Xuan Zhuo
2024-11-04 2:52 ` Xuan Zhuo
2024-10-31 16:44 ` [Intel-wired-lan] [resend PATCH 1/2] dim: make dim_calc_stats() inputs const pointers Vladimir Oltean
2024-10-31 16:44 ` Vladimir Oltean
2024-10-31 17:15 ` [Intel-wired-lan] " Florian Fainelli
2024-10-31 17:15 ` Florian Fainelli
2024-10-31 18:22 ` [Intel-wired-lan] " Kiyanovski, Arthur
2024-10-31 18:22 ` Kiyanovski, Arthur
2024-11-04 2:45 ` [Intel-wired-lan] " Xuan Zhuo
2024-11-04 2:45 ` Xuan Zhuo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7f494c4ae5a041fbafa4059e85431857@amazon.com \
--to=akiyano@amazon.com \
--cc=Mark-MC.Lee@mediatek.com \
--cc=Roy.Pledge@nxp.com \
--cc=andrew+netdev@lunn.ch \
--cc=angelogioacchino.delregno@collabora.com \
--cc=anthony.l.nguyen@intel.com \
--cc=bcm-kernel-feedback-list@broadcom.com \
--cc=brett.creeley@amd.com \
--cc=christophe.leroy@csgroup.eu \
--cc=claudiu.manoil@nxp.com \
--cc=corbet@lwn.net \
--cc=csander@purestorage.com \
--cc=darinzon@amazon.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=eperezma@redhat.com \
--cc=florian.fainelli@broadcom.com \
--cc=gakula@marvell.com \
--cc=hkelam@marvell.com \
--cc=horms@kernel.org \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jasowang@redhat.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=linux-rdma@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=lorenzo@kernel.org \
--cc=louis.peens@corigine.com \
--cc=matthias.bgg@gmail.com \
--cc=michael.chan@broadcom.com \
--cc=mst@redhat.com \
--cc=nbd@nbd.name \
--cc=ndagan@amazon.com \
--cc=netdev@vger.kernel.org \
--cc=opendmb@gmail.com \
--cc=oss-drivers@corigine.com \
--cc=pabeni@redhat.com \
--cc=przemyslaw.kitszel@intel.com \
--cc=saeedb@amazon.com \
--cc=saeedm@nvidia.com \
--cc=sbhatta@marvell.com \
--cc=sean.wang@mediatek.com \
--cc=sgoutham@marvell.com \
--cc=shannon.nelson@amd.com \
--cc=shayagr@amazon.com \
--cc=talgi@nvidia.com \
--cc=tariqt@nvidia.com \
--cc=virtualization@lists.linux.dev \
--cc=vladimir.oltean@nxp.com \
--cc=xuanzhuo@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.