Re: [PATCH v3 0/3] Enable QEMU NVMe userspace driver on s390x

[Niklas Schnelle <schnelle@linux.ibm.com>]

On Fri, 2025-04-11 at 16:28 -0600, Alex Williamson wrote:
> > > 
--- snip ---
> > > Cc: folks related to that commit.
> > > 
> > > The original issue that brought us ram_device was a very obscure
> > > alignment of a memory region versus a device quirk only seen with
> > > assignment of specific RTL NICs.
> > > 
> > > The description for commit 4a2e242bbb30 ("memory: Don't use memcpy for
> > > ram_device regions") also addresses unaligned accesses, we don't expect
> > > drivers to use them and we don't want them to work differently in a VM
> > > than they might on bare metal.  We can debate whether that's valid or
> > > not, but that was the intent.
> > > 
> > > Have we re-introduced the chance that we're using optimized
> > > instructions only meant to target RAM here or is __builtin_memcpy
> > > implicitly safe for MMIO?  Thanks,
> > > 
> > > Alex  
> > 
> > 
> > Hi Stefan, Alex
> > 
> > 
> > Polite ping. Following up to understand how we should proceed with this 
> > series. Please let me know if there are any concerns that i haven't 
> > addressed?
> 
> I disassembled the current implementation using ldn_he_p/stn_he_p on
> x86_64 and it doesn't appear to introduce any of the mmx/sse optimized
> optimized code that we were trying to get away from in introducing the
> ram_device MemoryRegion and getting away from memcpy.  I wish I had
> some assurance that __builtin_memcpy won't invoke such operations, but
> it seems unlikely that it would for the discrete, fundamental size
> operations we're asking of it.  Therefore, maybe it is advisable to use
> the ld*_he_p/st*_he_p helpers rather than open code the memory derefs.
> 
> It's unfortunate that s390x needs to specifically restrict this access
> to PCI memory, but maybe that means that PCI specific version of these
> helpers are only created for s390x and elsewhere #define'd to the
> generic ld/st helpers, which maybe means the main interface should be a
> host_pci_{ld,st}n_he_p (maybe "le" given the implementation) type
> function.  I don't know if we'd then create a ram_pci_device variant
> memory region ops for use in vfio-pci, but it should probably be coded
> with that in mind.  Thanks,
> 
> Alex
> 

Hi Alex,

Just as a clarification because it was unclear earlier in the thread.
While it is true that the PCI instructions are restricted to PCI, there
is also no other kind of MMIO on s390x. This might not really help here
though because the PCI instructions can not be used to access
main/normal memory. So it's really a distinction between normal memory
and MMIO where MMIO would, if they were available, also include things
like GPU VRAM.

Thanks,
Niklas