Re: Decrypting tt maps in ttm

[Zack Rusin <zackr@vmware.com>]

On Mon, 2023-09-18 at 16:21 -0400, Alex Deucher wrote:
> !! External Email
>
> On Mon, Sep 18, 2023 at 3:06 PM Thomas Hellström
> <thomas.hellstrom@linux.intel.com> wrote:
> >
> >
> > On 9/18/23 17:52, Zack Rusin wrote:
> > > On Mon, 2023-09-18 at 17:13 +0200, Thomas Hellström wrote:
> > > > Hi,
> > > >
> > > > On 9/18/23 16:56, Thomas Hellström wrote:
> > > > > Hi Zack, Christian
> > > > >
> > > > > On 9/18/23 13:36, Christian König wrote:
> > > > > > Hi Zack,
> > > > > >
> > > > > > adding Thomas and Daniel.
> > > > > >
> > > > > > I briefly remember that I talked with Thomas and some other people
> > > > > > about that quite a while ago as well, but I don't fully remember the
> > > > > > outcome.
> > > > > Found one old thread, but didn't read it:
> > > > >
> > > > > https://lists.freedesktop.org/archives/dri-devel/2019-September/234100.html
> > > > >
> > > > >
> > > > > /Thomas
> > > > >
> > > > >
> > > > Ugh. Now starting to read that thread I have a vague recollection it all
> > > > ended with not supporting mapping any device pages whatsoever when SEV
> > > > was enabled, but rather resorting to llvmpipe and VM-local bos.
> > > Hi, Thomas.
> > >
> > > Thanks for finding this! I'd (of course) like to solve it properly and get
> > > vmwgfx
> > > running with 3d support with SEV-ES active instead of essentially disabling
> > > the
> > > driver when SEV-ES is active.
> > >
> > > I think there are two separate discussions there, the non-controversial one
> > > and the
> > > controversial one:
> > > 1) The non-controversial: is there a case where drivers would want encrypted
> > > memory
> > > for TT pages but not for io mem mappings? Because if not then as Christian
> > > pointed
> > > out we could just add pgprot_decrypted to ttm_io_prot and be essentially done.
> > > The
> > > current method of decrypting io mem but leaving sys mem mappings encrypted is
> > > a bit
> > > weird anyway.
> > >
> > > If the answer to that question is "yes, some driver does want the TT mappings
> > > to be
> > > encrypted" then your "[PATCH v2 3/4] drm/ttm, drm/vmwgfx: Correctly support
> > > support
> > > AMD memory encryption" solves that. I think getting one of those two in makes
> > > sense
> > > regardless of everything else, agreed?
> >
> > Well, there is more to it I think.
> >
> > IIRC, the AMD SME encryption mode has a way for a device to have the
> > memory controller (?) encrypt / decrypt device traffic by using an
> > address range alias, so in theory it supports encrypted TT pages, and
> > the dma-layer may indeed hand encrypted DMA pages to TTM on such systems
> > depending on the device's DMA mask. That's why I think that
> > force_dma_unencrypted() export was needed, and If the amdgpu driver
> > accesses TT memory in SME mode *without* pgprot_decrypted() and it still
> > works, then I think that mode is actually used. How could it otherwise work?
>
> For SME, as long as the encrypted bit is set in the physical address
> used for DMA, the memory controller will handle the encrypt/decrypt
> for the device.  For devices with a limited dma mask, you need to use
> the IOMMU so that the encrypted bit is retained when the address hits
> the memory controller.

How does that work on systems with swiotlb, e.g. swiotlb=force, or i.e. what would
decrypt the ttm tt mappings when copying between system and vram when iommu is
disabled/absent?

z