Re: [PATCH v7 4/4] arm64/mm: Elide tlbi in contpte_convert() under BBML2

[Ryan Roberts <ryan.roberts@arm.com>]

On 25/06/2025 14:37, Jason Gunthorpe wrote:
> On Wed, Jun 25, 2025 at 02:07:23PM +0100, Ryan Roberts wrote:
>> I think what you are saying is that despite going via invalid, the HW may see
>> this direct transition:
>>
>> +----+----+----+----+
>> |RO,n|RO,n|RO,n|RW,n|
>> +----+----+----+----+
>> to:
>> +----+----+----+----+
>> |RO,c|RO,c|RO,c|RO,c|
>> +----+----+----+----+
>>
>> There are 2 logical operations here. The first is changing the permissions of
>> the last entry. The second is changing the size of the entry.
>>
>> As I understand it, it's permissible in the architecture to update the
>> permissions of the a PTE without break-before-make and without issuing a tlbi
>> afterwards; in that case the HW may apply either the old permissions or the new
>> permissions up until a future tlbi (after which the new permissions are
>> guarranteed). That's the first logical operation.
>>
>> The second logical operation is permitted by FEAT_BBM level 2.
>>
>> So both logical operations are permitted and the Arm ARM doesn't mention any
>> requirement to "separate" these operations with a tlbi or anything else, as far
>> as I can see.
>>
>> So I would interpret that combining these 2 in the way we have should be safe.
>> RNGLXZ and RJQQTC give further insight into the spirit of the spec. But I agree
>> this isn't spelled out super clearly.
>>
>> Perhaps we can move forwards based on this understanding, and I will seek some
>> clarifying words to be added to the Arm ARM?
> 
> FWIW this matches my understanding as well.
> 
> AFAIK the actual physical issue for BBM < 2 is the TLB lookup HW
> cannot tolerate having two entries that overlap in address because of
> different sizes. Every lookup must return exactly one result. If more
> results are returned the HW fails in some way.
> 
> For BBM 2 if the HW lookup gets more than one result the HW selects
> one unpredictably and uses that.
> 
> Permissions shouldn't have any impact because they do not effect how
> the lookup is performed, they are the result of the lookup.

Totally agree with all of this! But ideally we want the Arm ARM to be
unambiguous so that we can implement against that rather than against our
understanding of the HW issues and what the Arm ARM "almost certainly intended".

> 
> I've been looking at implementing BBM 2 support on the iommu side and
> I didn't see anything that said we cannot do arbitary transformations
> under BBM 2? For instance when degrading from a contiguous range of
> block descriptors down to single page descriptors I'm expecting to go
> like:
>     16x2M -> 2M -> 4k -> FLUSH

If this is just changing sizes, it's definitely, explcitly safe in the Arm ARM
(I haven't read the equivalent SMMU text). And there isn't a requirement to
flush at the end. You only need to flush the (sub-)ranges where permissions or
whatever have changed.

> 
> Jason