[RFC 0/8] ARM64 TLB logic revision for more control and enhanced diagnostics

[RFC 0/8] ARM64 TLB logic revision for more control and enhanced diagnostics

[Christoph Lameter]

WARNING: First draft of the patchset, tested with kernel compiles, may
corrupt your memory.

This patchset intends to aid in help debugging and scaling TLB operations
on ARM64. This is in particular desirable for ARM architectures with
large numbers of cores. What is often seen is that the mesh is flooded with
snoop traffic that is related to TLBI broadcasts.

The patchset adds the following features:

- Allow diagnostics via /proc/vmstat like already possible on X86 with the
	CONFIG_DEBUG_TLB  option.

  Some sample output:

  cat /proc/vmstat
	...
	nr_tlb_remote_flush 554104		Flushes converted to IPIs with local flushing
	nr_tlb_remote_flush_received 1312243	IPIs received to perform local flushing
	nr_tlb_local_flush_all 141837		Local flush alls
	nr_tlb_local_flush_range 571784		Local flush range
	nr_tlb_local_flush_one 8011880		Local individual page flushes
	nr_tlb_flush_all 28239			Flush alls through the mesh
	nr_tlb_flush_range 13003		Flush range through the mesh
	nr_tlb_flush_one 54764			Flush one through the mesh
	nr_tlb_skipped 0			Suppressed flush

- Tracks the cores that have used an address space. With that we can
	compute the weight of cpus that have used this address space
	to decide on how to optimally do the flushing when such an action
	is required.

- Control the TLB flushing behavior via the kernel command line and also on a running system.

    New Kernel parameter  tlb_mode=<tlb_mode>
    New sysfs setting     /sys/kernel/debug/tlb_mode

	tlb_mode is comprised of a set of flags starting at bit 10. Bit 0-9
	are used to set a boundary as to what cpu weight will lead to a mesh
	flush. If the cpu weight is lower then IPIs are send avoiding the mesh.

	Feature flags:

	Bit 10 = If the current cpu is the only one that has ever used an
			address space then perform local invalidation.
			This catches the majority of flushes on boot and
			activities of typical single threaded Unixy processes.

	Bit 11 = Enable TLB range. Various hardware has problems with TLB range.
			This allows the kernel to recognize that TLB range
			should not be used and an alternate method is to be
			used to do the flushing.

	Bit 12 = Suppress TLB flushes if the address space is unused.
			If this bit is set and a flush is requested in an
			unused address space then no flush will be performed
			since there cannot any TLB entries. If this is
			not set then perform mesh flush (just to be sure).

- Autotunes the feature flags on bootup if the user has not specified tlb_mode.
	Calculates an optimal balance between IPIs and mesh flushing
	based on the number of cpus in the system. Enables local validation
	always and tlb range flushing if the processor features indicate
	that the processor supports it.


We need a more detailed description but I hope this is enough to get started.

These issues have been discussed before in an patchset that contains a similar feature 
in 2019:

https://lore.kernel.org/linux-arm-kernel/20190617143255.10462-1-indou.takao@jp.fujitsu.com/


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC 3/8] ARM64: Base TLB functions

[Catalin Marinas]

Hi Christoph,

I don't think any of these patches made it to the list for some reason,
only the cover letter. I'll add some general comments below in case you
plan to repost.

On Wed, Dec 06, 2023 at 07:57:06PM -0800, Christoph Lameter wrote:
> Add a series of TLB function that allow TLB flushing in a variety of
> ways depending on the mode encoded in enum tlb_state:
> 
> TLB_BROADCOAST	-> Use single TLBI propagated to all units via the mesh
> TLB_LOCAL	-> Use TLBI that only perform local flushes
> TLB_IPI		-> Use TLBIs that perform local flushes on multiple cpus
> TLB_NONE	-> Suppress TLBI because there are no users of the address space

TLB_IPI - I really don't want this on arm64, irrespective of the
performance improvement on specific hardware. IPIs need additional care
to avoid deadlocking, e.g. don't do a TLBI with interrupts disabled. I'd
rather not have to care about this.

TLB_LOCAL and TLB_NONE - these are potentially useful _but_ they don't
take the SMMU with SVA into account (another potential issue below). The
mm_cpumask() does not track other non-CPU contexts but they do need the
broadcast TLBI.

> +static enum tlb_state tlbstat(struct cpumask *mask)
> +{
> +	unsigned int weight = cpumask_weight(mask);
> +	bool present = cpumask_test_cpu(smp_processor_id(), mask);

Is this guaranteed to be called in a non-preemptible context? My biggest
concern with the cpumask tracking is that the thread is migrated in the
middle of a TLB flush and it would wrongly assume that only a local TLBI
is needed. On arm64 we don't flush the TLBs on context switch. I don't
think we can solve this without additional locking around mm_cpumask()
or at least preemption disabling.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC 3/8] ARM64: Base TLB functions

[Christoph Lameter (Ampere)]

On Mon, 4 Mar 2024, Catalin Marinas wrote:

> I don't think any of these patches made it to the list for some reason,
> only the cover letter. I'll add some general comments below in case you
> plan to repost.

Correct. I believe I have fixed the isue but did not get around to repost 
the patchset.

>
> On Wed, Dec 06, 2023 at 07:57:06PM -0800, Christoph Lameter wrote:
>> Add a series of TLB function that allow TLB flushing in a variety of
>> ways depending on the mode encoded in enum tlb_state:
>>
>> TLB_BROADCOAST	-> Use single TLBI propagated to all units via the mesh
>> TLB_LOCAL	-> Use TLBI that only perform local flushes
>> TLB_IPI		-> Use TLBIs that perform local flushes on multiple cpus
>> TLB_NONE	-> Suppress TLBI because there are no users of the address space
>
> TLB_IPI - I really don't want this on arm64, irrespective of the
> performance improvement on specific hardware. IPIs need additional care
> to avoid deadlocking, e.g. don't do a TLBI with interrupts disabled. I'd
> rather not have to care about this.

These are commonly used on other platforms in these paths. So I assumed it 
would be safe.

> TLB_LOCAL and TLB_NONE - these are potentially useful _but_ they don't
> take the SMMU with SVA into account (another potential issue below). The
> mm_cpumask() does not track other non-CPU contexts but they do need the
> broadcast TLBI.

Hmm....I assume there is no way to determine if a TLB entry is in use by 
non-cpu contexts? There are MMU notifiers typically used for devices that 
require context outside of a cpu and that also flush the information. 
There are already callbacks in the ARM64 tlb flushing code to support MMU 
notifications.

>> +static enum tlb_state tlbstat(struct cpumask *mask)
>> +{
>> +	unsigned int weight = cpumask_weight(mask);
>> +	bool present = cpumask_test_cpu(smp_processor_id(), mask);
>
> Is this guaranteed to be called in a non-preemptible context? My biggest
> concern with the cpumask tracking is that the thread is migrated in the
> middle of a TLB flush and it would wrongly assume that only a local TLBI
> is needed. On arm64 we don't flush the TLBs on context switch. I don't
> think we can solve this without additional locking around mm_cpumask()
> or at least preemption disabling.

This would mean that the scheduler moves the process while the kernel 
executes a tlb flush operation.

TLB operations are related to page table modifications and those require 
to take a write lock on the mmap lock. This would prevent issues here.



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel