Re: [RFC PATCH v1 04/38] tsm: Support DMA Allocation from private memory

Alexey Kardashevskiy
2025-08-05T10:22:21Z

On 28/7/25 20:03, Jason Gunthorpe wrote:
> On Mon, Jul 28, 2025 at 07:21:41PM +0530, Aneesh Kumar K.V (Arm) wrote:
>> @@ -48,3 +49,12 @@ int set_memory_decrypted(unsigned long addr, int numpages)
>> return crypt_ops->decrypt(addr, numpages);
>> }
>> EXPORT_SYMBOL_GPL(set_memory_decrypted);
>> +
>> +bool force_dma_unencrypted(struct device *dev)
>> +{
>> + if (dev->tdi_enabled)
>> + return false;
>
> Is this OK? I see code like this:
>
> static inline dma_addr_t phys_to_dma_direct(struct device *dev,
> phys_addr_t phys)
> {
> if (force_dma_unencrypted(dev))
> return phys_to_dma_unencrypted(dev, phys);
> return phys_to_dma(dev, phys);

I did write this in the first place so I'll comment :)

>
> What are the ARM rules for generating dma addreses?
>
> 1) Device is T=0, memory is unencrypted, call dma_addr_unencrypted()
> and do "top bit IBA set"
>
> 2) Device is T=1, memory is encrypted, use the phys_to_dma() normally
>
> 3) Device it T=1, memory is uncrypted, use the phys_to_dma()
> normally??? Seems odd, I would have guessed the DMA address sould
> be the same as case #1?
>
> Can you document this in a comment?

On AMD, T=1 only encrypts the PCIe trafic, when a DMA request hits the IOMMU, the IOMMU decrypts it and then decides whether to encrypt it with a memory key: if there is secure vIOMMU - it will do what Cbit says in the guest IOMMU table (this is in the works) oooor just always set Cbit without guest vIOMMU (which is a big knob per a device and this is what my patches do now).

And with vIOMMU, I'd expect phys_to_dma_direct() not to be called as this one is in a direct map path.

>
>> diff --git a/include/linux/device.h b/include/linux/device.h
>> index 4940db137fff..d62e0dd9d8ee 100644
>> --- a/include/linux/device.h
>> +++ b/include/linux/device.h
>> @@ -688,6 +688,7 @@ struct device {
>> #ifdef CONFIG_IOMMU_DMA
>> bool dma_iommu:1;
>> #endif
>> + bool tdi_enabled:1;
>> };
>
> I would give the dev->tdi_enabled a clearer name, maybe
> dev->encrypted_dma_supported ?

May be but "_enabled", not "_supported". And, ideally, with vIOMMU, at least AMD won't be needing it.

>
> Also need to think carefully of a bitfield is OK here, we can't
> locklessly change a bitfield so need to audit that all members are set
> under, probably, the device lock or some other single threaded hand
> waving. It seems believable it is like that but should be checked out,
> and add a lockdep if it relies on the device lock.

True.

>
> Jason
>

--
Alexey