All of lore.kernel.org
 help / color / mirror / Atom feed
From: Lu Baolu <baolu.lu@linux.intel.com>
To: John Garry <john.garry@huawei.com>,
	joro@8bytes.org, will@kernel.org, robin.murphy@arm.com
Cc: linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
	linuxarm@huawei.com
Subject: Re: [PATCH v3 0/7] iommu: Allow IOVA rcache range be configured
Date: Thu, 3 Jun 2021 08:39:29 +0800	[thread overview]
Message-ID: <5e6ff4d6-cd67-d4ab-814c-e10a258191b1@linux.intel.com> (raw)
In-Reply-To: <1cbf8cc2-8cee-0579-2514-56f664baa9cd@huawei.com>

On 6/2/21 3:48 PM, John Garry wrote:
> On 02/06/2021 05:37, Lu Baolu wrote:
>> On 6/1/21 10:29 PM, John Garry wrote:
>>> For streaming DMA mappings involving an IOMMU and whose IOVA len 
>>> regularly
>>> exceeds the IOVA rcache upper limit (meaning that they are not cached),
>>> performance can be reduced.
>>>
>>> This is much more pronounced from commit 4e89dce72521 ("iommu/iova: 
>>> Retry
>>> from last rb tree node if iova search fails"), as discussed at [0].
>>>
>>> IOVAs which cannot be cached are highly involved in the IOVA ageing 
>>> issue,
>>> as discussed at [1].
>>>
>>> This series allows the IOVA rcache range be configured, so that we may
>>> cache all IOVAs per domain, thus improving performance.
>>>
>>> A new IOMMU group sysfs file is added - max_opt_dma_size - which is used
>>> indirectly to configure the IOVA rcache range:
>>> /sys/kernel/iommu_groups/X/max_opt_dma_size
>>>
>>> This file is updated same as how the IOMMU group default domain type is
>>> updated, i.e. must unbind the only device in the group first.
>>
>> Could you explain why it requires singleton group and driver unbinding
>> if the user only wants to increase the upper limit? I haven't dived into
>> the details yet, sorry if this is a silly question.
> 
> Hi Baolu,
> 
> I did actually try increasing the range for a 'live' domain in the v1 
> series, but it turned out too messy. First problem is reallocating the 
> memory to hold the rcaches. Second problem is that we need to deal with 
> the issue that all IOVAs in the rcache need to be a pow-of-2, which is 
> difficult to enforce for IOVAs which weren't being cached before, but 
> now would be.
> 
> So now I changed to work similar to how we change the default domain 
> type, i.e. don't operate on a 'live' domain.

If these hard restrictions on users are just to walk around the messy
code in kernel, I would rather solve those messy code to achieve a
better user experience. :-)

Best regards,
baolu


_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

WARNING: multiple messages have this Message-ID (diff)
From: Lu Baolu <baolu.lu@linux.intel.com>
To: John Garry <john.garry@huawei.com>,
	joro@8bytes.org, will@kernel.org, robin.murphy@arm.com
Cc: baolu.lu@linux.intel.com, iommu@lists.linux-foundation.org,
	linuxarm@huawei.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 0/7] iommu: Allow IOVA rcache range be configured
Date: Thu, 3 Jun 2021 08:39:29 +0800	[thread overview]
Message-ID: <5e6ff4d6-cd67-d4ab-814c-e10a258191b1@linux.intel.com> (raw)
In-Reply-To: <1cbf8cc2-8cee-0579-2514-56f664baa9cd@huawei.com>

On 6/2/21 3:48 PM, John Garry wrote:
> On 02/06/2021 05:37, Lu Baolu wrote:
>> On 6/1/21 10:29 PM, John Garry wrote:
>>> For streaming DMA mappings involving an IOMMU and whose IOVA len 
>>> regularly
>>> exceeds the IOVA rcache upper limit (meaning that they are not cached),
>>> performance can be reduced.
>>>
>>> This is much more pronounced from commit 4e89dce72521 ("iommu/iova: 
>>> Retry
>>> from last rb tree node if iova search fails"), as discussed at [0].
>>>
>>> IOVAs which cannot be cached are highly involved in the IOVA ageing 
>>> issue,
>>> as discussed at [1].
>>>
>>> This series allows the IOVA rcache range be configured, so that we may
>>> cache all IOVAs per domain, thus improving performance.
>>>
>>> A new IOMMU group sysfs file is added - max_opt_dma_size - which is used
>>> indirectly to configure the IOVA rcache range:
>>> /sys/kernel/iommu_groups/X/max_opt_dma_size
>>>
>>> This file is updated same as how the IOMMU group default domain type is
>>> updated, i.e. must unbind the only device in the group first.
>>
>> Could you explain why it requires singleton group and driver unbinding
>> if the user only wants to increase the upper limit? I haven't dived into
>> the details yet, sorry if this is a silly question.
> 
> Hi Baolu,
> 
> I did actually try increasing the range for a 'live' domain in the v1 
> series, but it turned out too messy. First problem is reallocating the 
> memory to hold the rcaches. Second problem is that we need to deal with 
> the issue that all IOVAs in the rcache need to be a pow-of-2, which is 
> difficult to enforce for IOVAs which weren't being cached before, but 
> now would be.
> 
> So now I changed to work similar to how we change the default domain 
> type, i.e. don't operate on a 'live' domain.

If these hard restrictions on users are just to walk around the messy
code in kernel, I would rather solve those messy code to achieve a
better user experience. :-)

Best regards,
baolu



  reply	other threads:[~2021-06-03  0:40 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-01 14:29 [PATCH v3 0/7] iommu: Allow IOVA rcache range be configured John Garry
2021-06-01 14:29 ` John Garry
2021-06-01 14:29 ` [PATCH v3 1/7] iommu: Reactor iommu_group_store_type() John Garry
2021-06-01 14:29   ` John Garry
2021-06-01 14:29 ` [PATCH v3 2/7] iova: Allow rcache range upper limit to be flexible John Garry
2021-06-01 14:29   ` John Garry
2021-06-01 14:29 ` [PATCH v3 3/7] iommu: Allow iommu_change_dev_def_domain() realloc default domain for same type John Garry
2021-06-01 14:29   ` John Garry
2021-06-01 14:29 ` [PATCH v3 4/7] iova: Add iova_domain_len_is_cached() John Garry
2021-06-01 14:29   ` John Garry
2021-06-01 14:29 ` [PATCH v3 5/7] iova: Add init_iova_domain_ext() John Garry
2021-06-01 14:29   ` John Garry
2021-06-01 14:29 ` [PATCH v3 6/7] iommu: Allow max opt DMA len be set for a group via sysfs John Garry
2021-06-01 14:29   ` John Garry
2021-06-01 14:29 ` [PATCH v3 7/7] dma-iommu: Use init_iova_domain_ext() for IOVA domain init John Garry
2021-06-01 14:29   ` John Garry
2021-06-02  4:37 ` [PATCH v3 0/7] iommu: Allow IOVA rcache range be configured Lu Baolu
2021-06-02  4:37   ` Lu Baolu
2021-06-02  7:48   ` John Garry
2021-06-02  7:48     ` John Garry
2021-06-03  0:39     ` Lu Baolu [this message]
2021-06-03  0:39       ` Lu Baolu
2021-06-03  8:00       ` John Garry
2021-06-03  8:00         ` John Garry

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5e6ff4d6-cd67-d4ab-814c-e10a258191b1@linux.intel.com \
    --to=baolu.lu@linux.intel.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=john.garry@huawei.com \
    --cc=joro@8bytes.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxarm@huawei.com \
    --cc=robin.murphy@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.