All of lore.kernel.org
 help / color / mirror / Atom feed
From: Shaohua Li <shli-b10kYP2dOMg@public.gmane.org>
To: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
Cc: Kernel-team-b10kYP2dOMg@public.gmane.org
Subject: [PATCH 0/4] iommu: remove lock contention in iova allocation
Date: Tue, 24 Nov 2015 13:54:06 -0800	[thread overview]
Message-ID: <cover.1448401089.git.shli@fb.com> (raw)

Hi,

The lock contention in iova allocation is very significant. In my iperf test
with intel 10Gbps NIC card with 48 threads, I observed 99% cpu time is spending
on the lock contention. In the test:

CPU utilization 100%, iperf reports ~1Gbps

The patches introduce a bitmap based approach for dma address allocation. It
completetly avoids the lock contention. Run the same test:

CPU utilization 1%, iperf reports 9.3Gbps

I also tried the test with the patch, but disable bitmap. The result is the
same as that without the patch.

The patches only work for DMA less than 8k, which is the most comman case we
have highest request(DMA) per second. Bigger size DMA still fallback to rbtree
based allocation. But if required and DAC is enabled by default, it's easy to
support bigger size DMA in the bitmap allocation.

Thanks,
Shaohua


Shaohua Li (4):
  iommu: alloc_iova returns a pfn
  iommu: add a bitmap based dma address allocator
  iommu: enable bitmap allocation for intel iommu
  iommu: free_iova doesn't need lock twice

 drivers/iommu/dma-iommu.c   |  32 ++++-----
 drivers/iommu/intel-iommu.c |  96 ++++++++++++++++-----------
 drivers/iommu/iova.c        | 157 +++++++++++++++++++++++++++++++++-----------
 include/linux/iova.h        |  21 ++++--
 4 files changed, 210 insertions(+), 96 deletions(-)

-- 
2.4.6

             reply	other threads:[~2015-11-24 21:54 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-24 21:54 Shaohua Li [this message]
     [not found] ` <cover.1448401089.git.shli-b10kYP2dOMg@public.gmane.org>
2015-11-24 21:54   ` [PATCH 1/4] iommu: alloc_iova returns a pfn Shaohua Li
2015-11-24 21:54   ` [PATCH 2/4] iommu: add a bitmap based dma address allocator Shaohua Li
     [not found]     ` <4c0804c85404be81acfe81fcd402f1af484e9d5f.1448401089.git.shli-b10kYP2dOMg@public.gmane.org>
2015-11-24 21:56       ` Woodhouse, David
     [not found]         ` <1448402209.44976.109.camel-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2015-11-24 22:05           ` Shaohua Li
     [not found]             ` <20151124220511.GA2106568-tb7CFzD8y5b7E6g3fPdp/g2O0Ztt9esIQQ4Iyu8u01E@public.gmane.org>
2015-11-25 14:46               ` jroedel-l3A5Bk7waGM
     [not found]                 ` <20151125144613.GA3763-l3A5Bk7waGM@public.gmane.org>
2015-11-25 16:40                   ` Woodhouse, David
     [not found]                     ` <1448469599.44976.134.camel-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2015-11-25 16:43                       ` Jens Axboe
     [not found]                         ` <5655E545.1010509-b10kYP2dOMg@public.gmane.org>
2015-11-25 17:33                           ` Shaohua Li
     [not found]                             ` <20151125173320.GA92074-tb7CFzD8y5b7E6g3fPdp/g2O0Ztt9esIQQ4Iyu8u01E@public.gmane.org>
2015-11-25 17:38                               ` Jens Axboe
     [not found]                                 ` <5655F1F8.9070203-b10kYP2dOMg@public.gmane.org>
2015-11-25 18:57                                   ` Shaohua Li
2015-11-24 21:54   ` [PATCH 3/4] iommu: enable bitmap allocation for intel iommu Shaohua Li
2015-11-24 21:54   ` [PATCH 4/4] iommu: free_iova doesn't need lock twice Shaohua Li
2015-12-07 17:52   ` [PATCH 0/4] iommu: remove lock contention in iova allocation Shaohua Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1448401089.git.shli@fb.com \
    --to=shli-b10kyp2domg@public.gmane.org \
    --cc=Kernel-team-b10kYP2dOMg@public.gmane.org \
    --cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.