* [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net
@ 2013-01-19 2:23 Craig Hada
2013-01-19 2:23 ` Craig Hada
0 siblings, 1 reply; 5+ messages in thread
From: Craig Hada @ 2013-01-19 2:23 UTC (permalink / raw)
To: netdev, craig.hada
Cc: sathya.perla, subbu.seetharaman, ajit.khaparde, linux-kernel
This patch sets the coherent DMA mask to 64-bit after the be2net driver
has been acknowledged that the system is 64-bit DMA capable. The coherent
DMA mask is examined by the Intel IOMMU driver to determine whether to
allow pass through context mapping for all devices. With this patch, the
be2net driver combined with be2net compatible hardware provides
comparable performance to the case where vt-d is disabled. The main use
case for this change is to decrease the time necessary to copy virtual
machine memory during KVM live migration instantiations.
This patch was tested on a system that enables the IOMMU in non-coherent
mode. Two DMA remapper issues were encountered in the previous version and
both patches have been committed.
commit ea2447f700cab264019b52e2b417d689e052dcfd
commit 2e12bc29fc5a12242d68e11875db3dd58efad9ff
The performance of this patch was measured with netperf with vt-d
enabled and disabled along with kernel boot parameters intel_iommu
and iommu. The command and parameters used in the netperf runs along
with results are as follows:
# netperf -c -p 12865 -H 10.10.0.2 -t TCP_STREAM -l 60
vt-d enabled and intel-iommu=on
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % U us/KB us/KB
87380 16384 16384 60.00 4636.58 16.76 -1.00 1.777 -1.000
87380 16384 16384 60.00 4718.79 17.30 -1.00 1.802 -1.000
87380 16384 16384 60.00 4566.91 18.04 -1.00 1.942 -1.000
vt-d enabled
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % U us/KB us/KB
87380 16384 16384 60.00 6837.12 7.21 -1.00 0.518 -1.000
87380 16384 16384 60.00 6956.20 6.38 -1.00 0.450 -1.000
87380 16384 16384 60.00 6903.63 6.75 -1.00 0.481 -1.000
vt-d disabled
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % U us/KB us/KB
87380 16384 16384 60.00 6909.70 6.57 -1.00 0.467 -1.000
87380 16384 16384 60.00 6811.79 6.90 -1.00 0.498 -1.000
87380 16384 16384 60.00 6861.32 6.71 -1.00 0.481 -1.000
vt-d enabled and intel_iommu=on and iommu=pt
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % U us/KB us/KB
87380 16384 16384 60.00 6768.45 7.30 -1.00 0.530 -1.000
87380 16384 16384 60.00 6841.40 7.39 -1.00 0.531 -1.000
87380 16384 16384 60.00 6861.78 5.68 -1.00 0.407 -1.000
Craig Hada (1):
drivers/net: Enable IOMMU pass through for be2net
drivers/net/ethernet/emulex/benet/be_main.c | 6 ++++++
1 files changed, 6 insertions(+), 0 deletions(-)
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net
2013-01-19 2:23 [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net Craig Hada
@ 2013-01-19 2:23 ` Craig Hada
2013-04-05 7:51 ` Ivan Vecera
0 siblings, 1 reply; 5+ messages in thread
From: Craig Hada @ 2013-01-19 2:23 UTC (permalink / raw)
To: netdev, craig.hada
Cc: sathya.perla, subbu.seetharaman, ajit.khaparde, linux-kernel
This patch sets the coherent DMA mask to 64-bit after the be2net driver
has been acknowledged that the system is 64-bit DMA capable. The coherent
DMA mask is examined by the Intel IOMMU driver to determine whether to
allow pass through context mapping for all devices. With this patch, the
be2net driver combined with be2net compatible hardware provides
comparable performance to the case where vt-d is disabled. The main use
case for this change is to decrease the time necessary to copy virtual
machine memory during KVM live migration instantiations.
This patch was tested on a system that enables the IOMMU in non-coherent
mode. Two DMA remapper issues were encountered in the previous version and
both patches have been committed.
commit ea2447f700cab264019b52e2b417d689e052dcfd
commit 2e12bc29fc5a12242d68e11875db3dd58efad9ff
Signed-off-by: Craig Hada <craig.hada@hp.com>
---
drivers/net/ethernet/emulex/benet/be_main.c | 6 ++++++
1 files changed, 6 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index 9dca22b..b507e99 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -4052,6 +4052,12 @@ static int be_probe(struct pci_dev *pdev, const struct pci_device_id *pdev_id)
status = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
if (!status) {
+ status = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
+ if (status < 0) {
+ dev_err(&pdev->dev,
+ "dma_set_coherent_mask failed, aborting\n");
+ goto free_netdev;
+ }
netdev->features |= NETIF_F_HIGHDMA;
} else {
status = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
--
1.7.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net
2013-01-19 2:23 ` Craig Hada
@ 2013-04-05 7:51 ` Ivan Vecera
2013-04-05 8:24 ` David Miller
0 siblings, 1 reply; 5+ messages in thread
From: Ivan Vecera @ 2013-04-05 7:51 UTC (permalink / raw)
To: David Miller
Cc: Craig Hada, netdev, sathya.perla, subbu.seetharaman,
ajit.khaparde, linux-kernel
On 01/19/2013 03:23 AM, Craig Hada wrote:
> This patch sets the coherent DMA mask to 64-bit after the be2net driver
> has been acknowledged that the system is 64-bit DMA capable. The coherent
> DMA mask is examined by the Intel IOMMU driver to determine whether to
> allow pass through context mapping for all devices. With this patch, the
> be2net driver combined with be2net compatible hardware provides
> comparable performance to the case where vt-d is disabled. The main use
> case for this change is to decrease the time necessary to copy virtual
> machine memory during KVM live migration instantiations.
>
> This patch was tested on a system that enables the IOMMU in non-coherent
> mode. Two DMA remapper issues were encountered in the previous version and
> both patches have been committed.
> commit ea2447f700cab264019b52e2b417d689e052dcfd
> commit 2e12bc29fc5a12242d68e11875db3dd58efad9ff
>
> Signed-off-by: Craig Hada <craig.hada@hp.com>
> ---
> drivers/net/ethernet/emulex/benet/be_main.c | 6 ++++++
> 1 files changed, 6 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
> index 9dca22b..b507e99 100644
> --- a/drivers/net/ethernet/emulex/benet/be_main.c
> +++ b/drivers/net/ethernet/emulex/benet/be_main.c
> @@ -4052,6 +4052,12 @@ static int be_probe(struct pci_dev *pdev, const struct pci_device_id *pdev_id)
>
> status = dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
> if (!status) {
> + status = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
> + if (status < 0) {
> + dev_err(&pdev->dev,
> + "dma_set_coherent_mask failed, aborting\n");
> + goto free_netdev;
> + }
> netdev->features |= NETIF_F_HIGHDMA;
> } else {
> status = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
>
Dave, I'd like to ask when do you plan to apply this patch?
Thanks,
Ivan
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net
2013-04-05 7:51 ` Ivan Vecera
@ 2013-04-05 8:24 ` David Miller
2013-04-05 8:37 ` Perla, Sathya
0 siblings, 1 reply; 5+ messages in thread
From: David Miller @ 2013-04-05 8:24 UTC (permalink / raw)
To: ivecera
Cc: craig.hada, netdev, sathya.perla, subbu.seetharaman,
ajit.khaparde, linux-kernel
From: Ivan Vecera <ivecera@redhat.com>
Date: Fri, 05 Apr 2013 09:51:24 +0200
> Dave, I'd like to ask when do you plan to apply this patch?
I'm waiting for the be2net driver maintainers to integrate it and
resend it to me, they're usually very good about that.
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net
2013-04-05 8:24 ` David Miller
@ 2013-04-05 8:37 ` Perla, Sathya
0 siblings, 0 replies; 5+ messages in thread
From: Perla, Sathya @ 2013-04-05 8:37 UTC (permalink / raw)
To: David Miller, ivecera@redhat.com
Cc: craig.hada@hp.com, netdev@vger.kernel.org,
Seetharaman, Subramanian, Khaparde, Ajit,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: David Miller [mailto:davem@davemloft.net]
>
> From: Ivan Vecera <ivecera@redhat.com>
> Date: Fri, 05 Apr 2013 09:51:24 +0200
>
> > Dave, I'd like to ask when do you plan to apply this patch?
>
> I'm waiting for the be2net driver maintainers to integrate it and resend it to
> me, they're usually very good about that.
Ok, I'll do it and send out a patch soon...thanks!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-04-05 8:37 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-01-19 2:23 [PATCH v2 net-next] drivers/net: Enable IOMMU pass through for be2net Craig Hada
2013-01-19 2:23 ` Craig Hada
2013-04-05 7:51 ` Ivan Vecera
2013-04-05 8:24 ` David Miller
2013-04-05 8:37 ` Perla, Sathya
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).