mmap2 is probably still broken in linux-user & other things

mmap2 is probably still broken in linux-user & other things

[Catherine A. Frederick]

Hi, I submitted a patch a while ago and then dropped off the face of the 
planet like most people do. In my journey to fix DRM/radeonsi in 
user-mode emulation I discovered a few bugs. I don't really have the 
time to write the infrastructure to make wrapping DRM IOCTLs remotely 
not-ugly, but I do want to get patches in for the last bug I encountered 
as it's kinda sneaky.

Currently(as of 6 months ago, and I doubt anyone noticed) mmap2 is 
completely broken for it's intended use(mapping pages outside the 
address limit). The value of the address input to mmap is clipped to an 
abi_ulong(4 bytes of 32 bit platforms afaik?) and because of this the 
value in pages when converted to addresses and passed to mmap on the 
host is invalid and this breaks radeonsi(and probably any other GPU 
driver in linux-user.

I don't really know how to fix this in a sane way so I'm looking for ideas.

Another thing I'm looking for ideas wrt is I'm thinking about is some 
form of automagic(opt-in, through a new type) pointer/union translation 
in IOCTLs over the guest barrier. Currently it's really ugly to wrap 
most of the IOCTLs in DRM and amdgpu because the DRM ioctls frequently 
pass pointers to the kernel and because of this I don't feel comfortable 
merging my patches(manually translating the struct to the types of the 
host sucks and feels like redundant work). On top of that AMDGPU uses a 
lot of structs where currently having one type for the call would 
corrupt the data in the return union.

Last thing: I never really got any performance benefit out of my 
scheduler, but I think that's mostly from bad implementation. Is there 
any information on TCG's performance characteristics? Thanks.

Re: mmap2 is probably still broken in linux-user & other things

[Philippe Mathieu-Daudé]

Cc'ing user-mode team.

On 11/8/20 4:53 PM, Catherine A. Frederick wrote:
> Hi, I submitted a patch a while ago and then dropped off the face of the
> planet like most people do. In my journey to fix DRM/radeonsi in
> user-mode emulation I discovered a few bugs. I don't really have the
> time to write the infrastructure to make wrapping DRM IOCTLs remotely
> not-ugly, but I do want to get patches in for the last bug I encountered
> as it's kinda sneaky.
> 
> Currently(as of 6 months ago, and I doubt anyone noticed) mmap2 is
> completely broken for it's intended use(mapping pages outside the
> address limit). The value of the address input to mmap is clipped to an
> abi_ulong(4 bytes of 32 bit platforms afaik?) and because of this the
> value in pages when converted to addresses and passed to mmap on the
> host is invalid and this breaks radeonsi(and probably any other GPU
> driver in linux-user.
> 
> I don't really know how to fix this in a sane way so I'm looking for ideas.
> 
> Another thing I'm looking for ideas wrt is I'm thinking about is some
> form of automagic(opt-in, through a new type) pointer/union translation
> in IOCTLs over the guest barrier. Currently it's really ugly to wrap
> most of the IOCTLs in DRM and amdgpu because the DRM ioctls frequently
> pass pointers to the kernel and because of this I don't feel comfortable
> merging my patches(manually translating the struct to the types of the
> host sucks and feels like redundant work). On top of that AMDGPU uses a
> lot of structs where currently having one type for the call would
> corrupt the data in the return union.
> 
> Last thing: I never really got any performance benefit out of my
> scheduler, but I think that's mostly from bad implementation. Is there
> any information on TCG's performance characteristics? Thanks.
> 
>