All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4 0/1] Risc-V Svinval support
@ 2023-06-21 12:41 Mayuresh Chitale
  2023-06-21 12:41 ` [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma Mayuresh Chitale
  0 siblings, 1 reply; 6+ messages in thread
From: Mayuresh Chitale @ 2023-06-21 12:41 UTC (permalink / raw)
  To: Palmer Dabbelt, Paul Walmsley, Albert Ou
  Cc: Mayuresh Chitale, =Atish Patra, Anup Patel, linux-riscv

This patch adds support for the Svinval extension as defined in the
Risc V Privileged specification. 

Changes in v4:
- Rebase and refactor as per latest changes on torvalds/master
- Drop patch 1 in the series

Changes in v3:
- Fix incorrect vma used for sinval instructions
- Use unified static key mechanism for svinval
- Rebased on torvalds/master

Changes in v2:
- Rebased on 5.18-rc3
- update riscv_fill_hwcap to probe Svinval extension


Mayuresh Chitale (1):
  riscv: mm: use svinval instructions instead of sfence.vma

 arch/riscv/mm/tlbflush.c | 62 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 56 insertions(+), 6 deletions(-)

-- 
2.34.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 6+ messages in thread
* [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma
  2023-06-21 12:41 [PATCH v4 0/1] Risc-V Svinval support Mayuresh Chitale
@ 2023-06-21 12:41 ` Mayuresh Chitale
  2023-06-21 14:14   ` Andrew Jones
  2023-06-21 15:44   ` Alexandre Ghiti
  0 siblings, 2 replies; 6+ messages in thread
From: Mayuresh Chitale @ 2023-06-21 12:41 UTC (permalink / raw)
  To: Palmer Dabbelt, Paul Walmsley, Albert Ou
  Cc: Mayuresh Chitale, =Atish Patra, Anup Patel, linux-riscv

When svinval is supported the local_flush_tlb_page*
functions would prefer to use the following sequence
to optimize the tlb flushes instead of a simple sfence.vma:

sfence.w.inval
svinval.vma
  .
  .
svinval.vma
sfence.inval.ir

The maximum number of consecutive svinval.vma instructions
that can be executed in local_flush_tlb_page* functions is
limited to PTRS_PER_PTE. This is required to avoid soft
lockups and the approach is similar to that used in arm64.

Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
---
 arch/riscv/mm/tlbflush.c | 62 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 56 insertions(+), 6 deletions(-)

diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
index 77be59aadc73..ade0b5cf8b47 100644
--- a/arch/riscv/mm/tlbflush.c
+++ b/arch/riscv/mm/tlbflush.c
@@ -5,6 +5,12 @@
 #include <linux/sched.h>
 #include <asm/sbi.h>
 #include <asm/mmu_context.h>
+#include <asm/hwcap.h>
+#include <asm/insn-def.h>
+
+#define has_svinval()	riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)
+
+static unsigned long tlb_flush_all_threshold __read_mostly = PTRS_PER_PTE;
 
 static inline void local_flush_tlb_all_asid(unsigned long asid)
 {
@@ -26,19 +32,63 @@ static inline void local_flush_tlb_page_asid(unsigned long addr,
 static inline void local_flush_tlb_range(unsigned long start,
 		unsigned long size, unsigned long stride)
 {
-	if (size <= stride)
-		local_flush_tlb_page(start);
-	else
+	if ((size / stride) <= tlb_flush_all_threshold) {
+		if (has_svinval()) {
+			asm volatile(SFENCE_W_INVAL() ::: "memory");
+			while (size) {
+				asm volatile(SINVAL_VMA(%0, zero)
+					     : : "r" (start) : "memory");
+				start += stride;
+				if (size > stride)
+					size -= stride;
+				else
+					size = 0;
+			}
+			asm volatile(SFENCE_INVAL_IR() ::: "memory");
+		} else {
+			while (size) {
+				local_flush_tlb_page(start);
+				start += stride;
+				if (size > stride)
+					size -= stride;
+				else
+					size = 0;
+			}
+		}
+	} else {
 		local_flush_tlb_all();
+	}
 }
 
 static inline void local_flush_tlb_range_asid(unsigned long start,
 		unsigned long size, unsigned long stride, unsigned long asid)
 {
-	if (size <= stride)
-		local_flush_tlb_page_asid(start, asid);
-	else
+	if ((size / stride) <= tlb_flush_all_threshold) {
+		if (has_svinval()) {
+			asm volatile(SFENCE_W_INVAL() ::: "memory");
+			while (size) {
+				asm volatile(SINVAL_VMA(%0, %1) : : "r" (start),
+					     "r" (asid) : "memory");
+				start += stride;
+				if (size > stride)
+					size -= stride;
+				else
+					size = 0;
+			}
+			asm volatile(SFENCE_INVAL_IR() ::: "memory");
+		} else {
+			while (size) {
+				local_flush_tlb_page_asid(start, asid);
+				start += stride;
+				if (size > stride)
+					size -= stride;
+				else
+					size = 0;
+			}
+		}
+	} else {
 		local_flush_tlb_all_asid(asid);
+	}
 }
 
 static void __ipi_flush_tlb_all(void *info)
-- 
2.34.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma
  2023-06-21 12:41 ` [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma Mayuresh Chitale
@ 2023-06-21 14:14   ` Andrew Jones
  2023-06-22 14:13     ` Mayuresh Chitale
  2023-06-21 15:44   ` Alexandre Ghiti
  1 sibling, 1 reply; 6+ messages in thread
From: Andrew Jones @ 2023-06-21 14:14 UTC (permalink / raw)
  To: Mayuresh Chitale
  Cc: Palmer Dabbelt, Paul Walmsley, Albert Ou, =Atish Patra,
	Anup Patel, linux-riscv

On Wed, Jun 21, 2023 at 06:11:33PM +0530, Mayuresh Chitale wrote:
> When svinval is supported the local_flush_tlb_page*
> functions would prefer to use the following sequence
> to optimize the tlb flushes instead of a simple sfence.vma:
> 
> sfence.w.inval
> svinval.vma
>   .
>   .
> svinval.vma
> sfence.inval.ir
> 
> The maximum number of consecutive svinval.vma instructions
> that can be executed in local_flush_tlb_page* functions is
> limited to PTRS_PER_PTE. This is required to avoid soft
> lockups and the approach is similar to that used in arm64.
> 
> Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
> ---
>  arch/riscv/mm/tlbflush.c | 62 ++++++++++++++++++++++++++++++++++++----
>  1 file changed, 56 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
> index 77be59aadc73..ade0b5cf8b47 100644
> --- a/arch/riscv/mm/tlbflush.c
> +++ b/arch/riscv/mm/tlbflush.c
> @@ -5,6 +5,12 @@
>  #include <linux/sched.h>
>  #include <asm/sbi.h>
>  #include <asm/mmu_context.h>
> +#include <asm/hwcap.h>
> +#include <asm/insn-def.h>
> +
> +#define has_svinval()	riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)
> +
> +static unsigned long tlb_flush_all_threshold __read_mostly = PTRS_PER_PTE;
>  
>  static inline void local_flush_tlb_all_asid(unsigned long asid)
>  {
> @@ -26,19 +32,63 @@ static inline void local_flush_tlb_page_asid(unsigned long addr,
>  static inline void local_flush_tlb_range(unsigned long start,
>  		unsigned long size, unsigned long stride)
>  {
> -	if (size <= stride)
> -		local_flush_tlb_page(start);
> -	else
> +	if ((size / stride) <= tlb_flush_all_threshold) {
> +		if (has_svinval()) {
> +			asm volatile(SFENCE_W_INVAL() ::: "memory");
> +			while (size) {
> +				asm volatile(SINVAL_VMA(%0, zero)
> +					     : : "r" (start) : "memory");
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}

nit: The four while loops added by this patch could all be written more
concisely as

  unsigned long end = start + size;

  while (start < end) {
    /* flush one */
    start += stride;
  }

And we could shift everything one level of indentation left with

  if ((size / stride) > tlb_flush_all_threshold) {
    local_flush_tlb_all();
    return;
  }

  if (has_svinval()) {
  ...

Thanks,
drew

> +			asm volatile(SFENCE_INVAL_IR() ::: "memory");
> +		} else {
> +			while (size) {
> +				local_flush_tlb_page(start);
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +		}
> +	} else {
>  		local_flush_tlb_all();
> +	}
>  }
>  
>  static inline void local_flush_tlb_range_asid(unsigned long start,
>  		unsigned long size, unsigned long stride, unsigned long asid)
>  {
> -	if (size <= stride)
> -		local_flush_tlb_page_asid(start, asid);
> -	else
> +	if ((size / stride) <= tlb_flush_all_threshold) {
> +		if (has_svinval()) {
> +			asm volatile(SFENCE_W_INVAL() ::: "memory");
> +			while (size) {
> +				asm volatile(SINVAL_VMA(%0, %1) : : "r" (start),
> +					     "r" (asid) : "memory");
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +			asm volatile(SFENCE_INVAL_IR() ::: "memory");
> +		} else {
> +			while (size) {
> +				local_flush_tlb_page_asid(start, asid);
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +		}
> +	} else {
>  		local_flush_tlb_all_asid(asid);
> +	}
>  }
>  
>  static void __ipi_flush_tlb_all(void *info)
> -- 
> 2.34.1
> 
> 
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma
  2023-06-21 12:41 ` [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma Mayuresh Chitale
  2023-06-21 14:14   ` Andrew Jones
@ 2023-06-21 15:44   ` Alexandre Ghiti
  2023-06-23  6:58     ` Mayuresh Chitale
  1 sibling, 1 reply; 6+ messages in thread
From: Alexandre Ghiti @ 2023-06-21 15:44 UTC (permalink / raw)
  To: Mayuresh Chitale, Palmer Dabbelt, Paul Walmsley, Albert Ou
  Cc: =Atish Patra, Anup Patel, linux-riscv

Hi Mayuresh,

On 21/06/2023 14:41, Mayuresh Chitale wrote:
> When svinval is supported the local_flush_tlb_page*
> functions would prefer to use the following sequence
> to optimize the tlb flushes instead of a simple sfence.vma:
>
> sfence.w.inval
> svinval.vma
>    .
>    .
> svinval.vma
> sfence.inval.ir
>
> The maximum number of consecutive svinval.vma instructions
> that can be executed in local_flush_tlb_page* functions is
> limited to PTRS_PER_PTE. This is required to avoid soft
> lockups and the approach is similar to that used in arm64.
>
> Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
> ---
>   arch/riscv/mm/tlbflush.c | 62 ++++++++++++++++++++++++++++++++++++----
>   1 file changed, 56 insertions(+), 6 deletions(-)
>
> diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
> index 77be59aadc73..ade0b5cf8b47 100644
> --- a/arch/riscv/mm/tlbflush.c
> +++ b/arch/riscv/mm/tlbflush.c
> @@ -5,6 +5,12 @@
>   #include <linux/sched.h>
>   #include <asm/sbi.h>
>   #include <asm/mmu_context.h>
> +#include <asm/hwcap.h>
> +#include <asm/insn-def.h>
> +
> +#define has_svinval()	riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)
> +
> +static unsigned long tlb_flush_all_threshold __read_mostly = PTRS_PER_PTE;


The threshold is quite high to me: internally, we computed that 
something like 50 would be a good bet, because to me, 512 * sfence.vma 
(or svinval equivalent) takes way more time than just flushing the whole 
tlb (even with the whole refill I'd say). How did you get this number? 
And this value is micro-architecture dependent, so we need to find a 
consensus or a mechanism to allow a vendor to change it.

FYI, x86 threshold is 33 (can't find right now the pointer sorry).


>   
>   static inline void local_flush_tlb_all_asid(unsigned long asid)
>   {
> @@ -26,19 +32,63 @@ static inline void local_flush_tlb_page_asid(unsigned long addr,
>   static inline void local_flush_tlb_range(unsigned long start,
>   		unsigned long size, unsigned long stride)
>   {
> -	if (size <= stride)
> -		local_flush_tlb_page(start);
> -	else
> +	if ((size / stride) <= tlb_flush_all_threshold) {


If size is not aligned on stride, you could get another page to flush so 
you do not respect the threshold. In my patchset, I used DIV_ROUND_UP.


> +		if (has_svinval()) {
> +			asm volatile(SFENCE_W_INVAL() ::: "memory");
> +			while (size) {
> +				asm volatile(SINVAL_VMA(%0, zero)
> +					     : : "r" (start) : "memory");
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +			asm volatile(SFENCE_INVAL_IR() ::: "memory");
> +		} else {
> +			while (size) {
> +				local_flush_tlb_page(start);
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +		}
> +	} else {
>   		local_flush_tlb_all();
> +	}
>   }
>   
>   static inline void local_flush_tlb_range_asid(unsigned long start,
>   		unsigned long size, unsigned long stride, unsigned long asid)
>   {
> -	if (size <= stride)
> -		local_flush_tlb_page_asid(start, asid);
> -	else
> +	if ((size / stride) <= tlb_flush_all_threshold) {
> +		if (has_svinval()) {
> +			asm volatile(SFENCE_W_INVAL() ::: "memory");
> +			while (size) {
> +				asm volatile(SINVAL_VMA(%0, %1) : : "r" (start),
> +					     "r" (asid) : "memory");
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +			asm volatile(SFENCE_INVAL_IR() ::: "memory");
> +		} else {
> +			while (size) {
> +				local_flush_tlb_page_asid(start, asid);
> +				start += stride;
> +				if (size > stride)
> +					size -= stride;
> +				else
> +					size = 0;
> +			}
> +		}
> +	} else {
>   		local_flush_tlb_all_asid(asid);
> +	}
>   }
>   
>   static void __ipi_flush_tlb_all(void *info)

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma
  2023-06-21 14:14   ` Andrew Jones
@ 2023-06-22 14:13     ` Mayuresh Chitale
  0 siblings, 0 replies; 6+ messages in thread
From: Mayuresh Chitale @ 2023-06-22 14:13 UTC (permalink / raw)
  To: Andrew Jones
  Cc: Palmer Dabbelt, Paul Walmsley, Albert Ou, =Atish Patra,
	Anup Patel, linux-riscv

On Wed, Jun 21, 2023 at 7:44 PM Andrew Jones <ajones@ventanamicro.com> wrote:
>
> On Wed, Jun 21, 2023 at 06:11:33PM +0530, Mayuresh Chitale wrote:
> > When svinval is supported the local_flush_tlb_page*
> > functions would prefer to use the following sequence
> > to optimize the tlb flushes instead of a simple sfence.vma:
> >
> > sfence.w.inval
> > svinval.vma
> >   .
> >   .
> > svinval.vma
> > sfence.inval.ir
> >
> > The maximum number of consecutive svinval.vma instructions
> > that can be executed in local_flush_tlb_page* functions is
> > limited to PTRS_PER_PTE. This is required to avoid soft
> > lockups and the approach is similar to that used in arm64.
> >
> > Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
> > ---
> >  arch/riscv/mm/tlbflush.c | 62 ++++++++++++++++++++++++++++++++++++----
> >  1 file changed, 56 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
> > index 77be59aadc73..ade0b5cf8b47 100644
> > --- a/arch/riscv/mm/tlbflush.c
> > +++ b/arch/riscv/mm/tlbflush.c
> > @@ -5,6 +5,12 @@
> >  #include <linux/sched.h>
> >  #include <asm/sbi.h>
> >  #include <asm/mmu_context.h>
> > +#include <asm/hwcap.h>
> > +#include <asm/insn-def.h>
> > +
> > +#define has_svinval()        riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)
> > +
> > +static unsigned long tlb_flush_all_threshold __read_mostly = PTRS_PER_PTE;
> >
> >  static inline void local_flush_tlb_all_asid(unsigned long asid)
> >  {
> > @@ -26,19 +32,63 @@ static inline void local_flush_tlb_page_asid(unsigned long addr,
> >  static inline void local_flush_tlb_range(unsigned long start,
> >               unsigned long size, unsigned long stride)
> >  {
> > -     if (size <= stride)
> > -             local_flush_tlb_page(start);
> > -     else
> > +     if ((size / stride) <= tlb_flush_all_threshold) {
> > +             if (has_svinval()) {
> > +                     asm volatile(SFENCE_W_INVAL() ::: "memory");
> > +                     while (size) {
> > +                             asm volatile(SINVAL_VMA(%0, zero)
> > +                                          : : "r" (start) : "memory");
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
>
> nit: The four while loops added by this patch could all be written more
> concisely as
>
>   unsigned long end = start + size;
>
>   while (start < end) {
>     /* flush one */
>     start += stride;
>   }
>
> And we could shift everything one level of indentation left with
>
>   if ((size / stride) > tlb_flush_all_threshold) {
>     local_flush_tlb_all();
>     return;
>   }
>
>   if (has_svinval()) {

Thx Drew. I will update in the next revision
>   ...
>
> Thanks,
> drew
>
> > +                     asm volatile(SFENCE_INVAL_IR() ::: "memory");
> > +             } else {
> > +                     while (size) {
> > +                             local_flush_tlb_page(start);
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +             }
> > +     } else {
> >               local_flush_tlb_all();
> > +     }
> >  }
> >
> >  static inline void local_flush_tlb_range_asid(unsigned long start,
> >               unsigned long size, unsigned long stride, unsigned long asid)
> >  {
> > -     if (size <= stride)
> > -             local_flush_tlb_page_asid(start, asid);
> > -     else
> > +     if ((size / stride) <= tlb_flush_all_threshold) {
> > +             if (has_svinval()) {
> > +                     asm volatile(SFENCE_W_INVAL() ::: "memory");
> > +                     while (size) {
> > +                             asm volatile(SINVAL_VMA(%0, %1) : : "r" (start),
> > +                                          "r" (asid) : "memory");
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +                     asm volatile(SFENCE_INVAL_IR() ::: "memory");
> > +             } else {
> > +                     while (size) {
> > +                             local_flush_tlb_page_asid(start, asid);
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +             }
> > +     } else {
> >               local_flush_tlb_all_asid(asid);
> > +     }
> >  }
> >
> >  static void __ipi_flush_tlb_all(void *info)
> > --
> > 2.34.1
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > linux-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma
  2023-06-21 15:44   ` Alexandre Ghiti
@ 2023-06-23  6:58     ` Mayuresh Chitale
  0 siblings, 0 replies; 6+ messages in thread
From: Mayuresh Chitale @ 2023-06-23  6:58 UTC (permalink / raw)
  To: Alexandre Ghiti
  Cc: Palmer Dabbelt, Paul Walmsley, Albert Ou, =Atish Patra,
	Anup Patel, linux-riscv

Hi Alex,

On Wed, Jun 21, 2023 at 9:14 PM Alexandre Ghiti <alex@ghiti.fr> wrote:
>
> Hi Mayuresh,
>
> On 21/06/2023 14:41, Mayuresh Chitale wrote:
> > When svinval is supported the local_flush_tlb_page*
> > functions would prefer to use the following sequence
> > to optimize the tlb flushes instead of a simple sfence.vma:
> >
> > sfence.w.inval
> > svinval.vma
> >    .
> >    .
> > svinval.vma
> > sfence.inval.ir
> >
> > The maximum number of consecutive svinval.vma instructions
> > that can be executed in local_flush_tlb_page* functions is
> > limited to PTRS_PER_PTE. This is required to avoid soft
> > lockups and the approach is similar to that used in arm64.
> >
> > Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
> > ---
> >   arch/riscv/mm/tlbflush.c | 62 ++++++++++++++++++++++++++++++++++++----
> >   1 file changed, 56 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
> > index 77be59aadc73..ade0b5cf8b47 100644
> > --- a/arch/riscv/mm/tlbflush.c
> > +++ b/arch/riscv/mm/tlbflush.c
> > @@ -5,6 +5,12 @@
> >   #include <linux/sched.h>
> >   #include <asm/sbi.h>
> >   #include <asm/mmu_context.h>
> > +#include <asm/hwcap.h>
> > +#include <asm/insn-def.h>
> > +
> > +#define has_svinval()        riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL)
> > +
> > +static unsigned long tlb_flush_all_threshold __read_mostly = PTRS_PER_PTE;
>
>
> The threshold is quite high to me: internally, we computed that
> something like 50 would be a good bet, because to me, 512 * sfence.vma
> (or svinval equivalent) takes way more time than just flushing the whole
> tlb (even with the whole refill I'd say). How did you get this number?
> And this value is micro-architecture dependent, so we need to find a
> consensus or a mechanism to allow a vendor to change it.
I had borrowed this limit from ARM64 but I agree that this is
micro-architecture specific.
So how about we make this a global variable and let the platform override it?
>
> FYI, x86 threshold is 33 (can't find right now the pointer sorry).
>
>
> >
> >   static inline void local_flush_tlb_all_asid(unsigned long asid)
> >   {
> > @@ -26,19 +32,63 @@ static inline void local_flush_tlb_page_asid(unsigned long addr,
> >   static inline void local_flush_tlb_range(unsigned long start,
> >               unsigned long size, unsigned long stride)
> >   {
> > -     if (size <= stride)
> > -             local_flush_tlb_page(start);
> > -     else
> > +     if ((size / stride) <= tlb_flush_all_threshold) {
>
>
> If size is not aligned on stride, you could get another page to flush so
> you do not respect the threshold. In my patchset, I used DIV_ROUND_UP.
Ok. Will change it in the next revision.
>
>
> > +             if (has_svinval()) {
> > +                     asm volatile(SFENCE_W_INVAL() ::: "memory");
> > +                     while (size) {
> > +                             asm volatile(SINVAL_VMA(%0, zero)
> > +                                          : : "r" (start) : "memory");
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +                     asm volatile(SFENCE_INVAL_IR() ::: "memory");
> > +             } else {
> > +                     while (size) {
> > +                             local_flush_tlb_page(start);
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +             }
> > +     } else {
> >               local_flush_tlb_all();
> > +     }
> >   }
> >
> >   static inline void local_flush_tlb_range_asid(unsigned long start,
> >               unsigned long size, unsigned long stride, unsigned long asid)
> >   {
> > -     if (size <= stride)
> > -             local_flush_tlb_page_asid(start, asid);
> > -     else
> > +     if ((size / stride) <= tlb_flush_all_threshold) {
> > +             if (has_svinval()) {
> > +                     asm volatile(SFENCE_W_INVAL() ::: "memory");
> > +                     while (size) {
> > +                             asm volatile(SINVAL_VMA(%0, %1) : : "r" (start),
> > +                                          "r" (asid) : "memory");
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +                     asm volatile(SFENCE_INVAL_IR() ::: "memory");
> > +             } else {
> > +                     while (size) {
> > +                             local_flush_tlb_page_asid(start, asid);
> > +                             start += stride;
> > +                             if (size > stride)
> > +                                     size -= stride;
> > +                             else
> > +                                     size = 0;
> > +                     }
> > +             }
> > +     } else {
> >               local_flush_tlb_all_asid(asid);
> > +     }
> >   }
> >
> >   static void __ipi_flush_tlb_all(void *info)

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-06-23  6:59 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-06-21 12:41 [PATCH v4 0/1] Risc-V Svinval support Mayuresh Chitale
2023-06-21 12:41 ` [PATCH v4 1/1] riscv: mm: use svinval instructions instead of sfence.vma Mayuresh Chitale
2023-06-21 14:14   ` Andrew Jones
2023-06-22 14:13     ` Mayuresh Chitale
2023-06-21 15:44   ` Alexandre Ghiti
2023-06-23  6:58     ` Mayuresh Chitale

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.