[PATCH] mm/hmm/test: destroy xa_array instead of looping

[PATCH] mm/hmm/test: destroy xa_array instead of looping

[Ralph Campbell]

The test driver uses an xa_array to store virtual to physical address
translations for a simulated hardware device. The MMU notifier
invalidation callback is used to keep the table consistent with the CPU
page table and is frequently called only for a page or two. However, if
the test process exits unexpectedly or is killed, the range can be
[0..ULONG_MAX] in which case calling xa_erase() for every possible PFN
results in CPU timeouts. Munmap() can result in a large range being
invalidated but in that case, the xa_array is likely to contain entries
that need to be invalidated.
Check for [0..ULONG_MAX] explicitly and just destroy the whole table.

Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
---

This patch is based on Jason Gunthorpe's hmm tree and should be folded
into the ("mm/hmm/test: add selftest driver for HMM") patch once this
patch is reviewed, etc.

 lib/test_hmm.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/test_hmm.c b/lib/test_hmm.c
index 8b36c26b717b..b89852ec3c29 100644
--- a/lib/test_hmm.c
+++ b/lib/test_hmm.c
@@ -201,7 +201,13 @@ static void dmirror_do_update(struct dmirror *dmirror, unsigned long start,
 	 * The XArray doesn't hold references to pages since it relies on
 	 * the mmu notifier to clear page pointers when they become stale.
 	 * Therefore, it is OK to just clear the entry.
+	 * However, if the entire address space is being invalidated, it
+	 * takes too long to clear them one at a time so destroy the array.
 	 */
+	if (start == 0 && end == ULONG_MAX) {
+		xa_destroy(&dmirror->pt);
+		return;
+	}
 	for (pfn = start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++)
 		xa_erase(&dmirror->pt, pfn);
 }
-- 
2.20.1

Re: [PATCH] mm/hmm/test: destroy xa_array instead of looping

[Jason Gunthorpe]

On Wed, May 13, 2020 at 02:45:07PM -0700, Ralph Campbell wrote:
> The test driver uses an xa_array to store virtual to physical address
> translations for a simulated hardware device. The MMU notifier
> invalidation callback is used to keep the table consistent with the CPU
> page table and is frequently called only for a page or two. However, if
> the test process exits unexpectedly or is killed, the range can be
> [0..ULONG_MAX] in which case calling xa_erase() for every possible PFN
> results in CPU timeouts. Munmap() can result in a large range being
> invalidated but in that case, the xa_array is likely to contain entries
> that need to be invalidated.
> Check for [0..ULONG_MAX] explicitly and just destroy the whole table.
> 
> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
> 
> This patch is based on Jason Gunthorpe's hmm tree and should be folded
> into the ("mm/hmm/test: add selftest driver for HMM") patch once this
> patch is reviewed, etc.
> 
>  lib/test_hmm.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/lib/test_hmm.c b/lib/test_hmm.c
> index 8b36c26b717b..b89852ec3c29 100644
> +++ b/lib/test_hmm.c
> @@ -201,7 +201,13 @@ static void dmirror_do_update(struct dmirror *dmirror, unsigned long start,
>  	 * The XArray doesn't hold references to pages since it relies on
>  	 * the mmu notifier to clear page pointers when they become stale.
>  	 * Therefore, it is OK to just clear the entry.
> +	 * However, if the entire address space is being invalidated, it
> +	 * takes too long to clear them one at a time so destroy the array.
>  	 */
> +	if (start == 0 && end == ULONG_MAX) {
> +		xa_destroy(&dmirror->pt);
> +		return;
> +	}
>  	for (pfn = start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++)
>  		xa_erase(&dmirror->pt, pfn);
>  }

Just use xa_for_each_range() instead of the naive loop, it already
optimizes against membership and avoids the need for the xa_destroy
hack

Jason

Re: [PATCH] mm/hmm/test: destroy xa_array instead of looping

[Ralph Campbell]

On 5/15/20 4:15 PM, Jason Gunthorpe wrote:
> On Wed, May 13, 2020 at 02:45:07PM -0700, Ralph Campbell wrote:
>> The test driver uses an xa_array to store virtual to physical address
>> translations for a simulated hardware device. The MMU notifier
>> invalidation callback is used to keep the table consistent with the CPU
>> page table and is frequently called only for a page or two. However, if
>> the test process exits unexpectedly or is killed, the range can be
>> [0..ULONG_MAX] in which case calling xa_erase() for every possible PFN
>> results in CPU timeouts. Munmap() can result in a large range being
>> invalidated but in that case, the xa_array is likely to contain entries
>> that need to be invalidated.
>> Check for [0..ULONG_MAX] explicitly and just destroy the whole table.
>>
>> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
>>
>> This patch is based on Jason Gunthorpe's hmm tree and should be folded
>> into the ("mm/hmm/test: add selftest driver for HMM") patch once this
>> patch is reviewed, etc.
>>
>>   lib/test_hmm.c | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/lib/test_hmm.c b/lib/test_hmm.c
>> index 8b36c26b717b..b89852ec3c29 100644
>> +++ b/lib/test_hmm.c
>> @@ -201,7 +201,13 @@ static void dmirror_do_update(struct dmirror *dmirror, unsigned long start,
>>   	 * The XArray doesn't hold references to pages since it relies on
>>   	 * the mmu notifier to clear page pointers when they become stale.
>>   	 * Therefore, it is OK to just clear the entry.
>> +	 * However, if the entire address space is being invalidated, it
>> +	 * takes too long to clear them one at a time so destroy the array.
>>   	 */
>> +	if (start == 0 && end == ULONG_MAX) {
>> +		xa_destroy(&dmirror->pt);
>> +		return;
>> +	}
>>   	for (pfn = start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++)
>>   		xa_erase(&dmirror->pt, pfn);
>>   }
> 
> Just use xa_for_each_range() instead of the naive loop, it already
> optimizes against membership and avoids the need for the xa_destroy
> hack
> 
> Jason
> 

For some reason I had looked at that and rejected it but of course, it works
fine. :-)
Thanks!