Re: [PATCH 3/3] riscv: rewrite tlb flush for performance improvement

[Christoph Hellwig <hch@infradead.org>]

> +menu "Virtual memory management"
> +
> +config RISCV_TLBI_IPI
> +	bool "Use IPI instead of SBI for remote TLB shootdown"
> +	default n
> +	help
> +	  Instead of using remote TLB shootdown interfaces provided by SBI,
> +	  use IPI to handle remote TLB shootdown within Linux kernel.
> +
> +	  BBL/OpenSBI are currently ignoring ASID and address range provided
> +	  by SBI call argument, and do a full TLB flush instead. This may
> +	  negatively impact performance on implementations with page-level
> +	  sfence.vma support.
> +
> +	  If you don't know what to do here, say N.

Requiring a kconfig here is rather sad.  For now I would just
switch entirely to your non-SBI version as doing SBI calls for this
is rather pointless to start with.  Either we get real architectural
hardware acceleration, or we might as well use IPIs ourselves.

That being said if there are strong arguments to keep the old code
I'd still prefer that to be runtime selectable.

> +
> +config RISCV_TLBI_MAX_OPS
> +	int "Max number of page-level sfence.vma per range TLB flush"
> +	range 1 511
> +	default 1
> +	help
> +	  This config specifies how many page-level sfence.vma can the Linux
> +	  kernel issue when the kernel needs to flush a range from the TLB.
> +	  If the required number of page-level sfence.vma exceeds this limit,
> +	  a full sfence.vma is issued.
> +
> +	  Increase this number can negatively impact performance on
> +	  implemntations where sfence.vma's address operand is ignored and
> +	  always perform a global TLB flush.
> +
> +	  On the other hand, implementations with page-level TLB flush support
> +	  can benefit from a larger number.
> +
> +	  If you don't know what to do here, keep the default value 1.

Again, I don't think hardcoding this makes any sense.  To make the
setting it needs to be overridable, and preferably provided by the
SBI code in some form (DT entry?).

> index 54fee0cadb1e..f254237a3bda 100644
> --- a/arch/riscv/include/asm/tlbflush.h
> +++ b/arch/riscv/include/asm/tlbflush.h
> @@ -1,6 +1,5 @@
>  /*
> - * Copyright (C) 2009 Chen Liqin <liqin.chen@sunplusct.com>
> - * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2019 Gary Guo, University of Cambridge

Unless you complete rewrite the file it is rather rude to remove
the existing copyright.  There still seem to be a decent amount
of at least comments left from the old codebase.

> +static inline void local_flush_tlb_page(struct vm_area_struct *vma,
> +		unsigned long addr)
> +{
> +	__asm__ __volatile__ ("sfence.vma %0, %1" : : "r" (addr), "r" (0) : "memory");
> +}

Please avoid lines over 80 chars.  Also I find inline assembly much
easier to read if each argument has its own line.

> +#ifndef CONFIG_SMP

If you rewrite this anyway can you switch the order around and use
an ifdef instead of ifndef?

> +extern void flush_tlb_all(void);
> +extern void flush_tlb_mm(struct mm_struct *mm);
> +extern void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr);
> +extern void flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
> +		unsigned long end);
> +extern void flush_tlb_kernel_range(unsigned long start, unsigned long end);

No need for all the externs.

> diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
> new file mode 100644
> index 000000000000..76cea33aa9c7
> --- /dev/null
> +++ b/arch/riscv/mm/tlbflush.c
> @@ -0,0 +1,144 @@
> +/*
> + * Copyright (C) 2019 Gary Guo, University of Cambridge
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <http://www.gnu.org/licenses/>.
> + */

Please use an SPDX tag instead of the license boilerplate.

> +#include <linux/mm.h>
> +#include <asm/sbi.h>
> +
> +/*
> + * BBL/OpenSBI are currently ignoring ASID and address range provided
> + * by SBI call argument, and do a full TLB flush instead.

This is really information for the changelog, not really for the code.

> +static void ipi_remote_sfence_vma(void *info)
> +{
> +	struct tlbi *data = (struct tlbi *) info;

No need for the cast.

> +	if (size == (unsigned long) -1) {

Maybe add a symbolic RISCV_FLUSH_ALL macros for the magic -1?

> +void flush_tlb_page(struct vm_area_struct *vma,
> +		unsigned long addr)

The second argument easily fits onto the first line.

> +void flush_tlb_range(struct vm_area_struct *vma,
> +		unsigned long start, unsigned long end)

Same here.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv