[Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[Peter Maydell]

It turns out that SPARC hosts impose a requirement that
if you mmap MAP_SHARED|MAP_FIXED then the address must be
aligned to SHMLBA alignment, which for SPARC is 16K and
larger than the 8K page size:
 http://lxr.linux.no/#linux+v4.10.1/arch/sparc/kernel/sys_sparc_64.c#L158

QEMU doesn't really cope with this. In particular, ivmshm
doesn't work:

$ ppc64-softmmu/qemu-system-ppc64 -display none -device
ivshmem,shm=/qtest-20273-1334572184,size=0x1
qemu-system-ppc64: System page size 0x2000 is not enabled in
page_size_mask (0x11000). Performance may be slow
qemu-system-ppc64: -device
ivshmem,shm=/qtest-20273-1334572184,size=0x1: ivshmem is deprecated,
please use ivshmem-plain or ivshmem-doorbell instead
Unexpected error in file_ram_alloc() at /home/pm215/qemu/exec.c:1599:
qemu-system-ppc64: -device
ivshmem,shm=/qtest-20273-1334572184,size=0x1: unable to map backing
store for guest RAM: Invalid argument
Aborted

(noticed because this is a make check failure).

exec.c calls qemu_fd_getpagesize() to figure out the alignment
here (on a file in /dev/shm/) and ends up in its default codepath
that returns getpagesize(), which is too small.

What's the best way to fix this? As far as I can tell
the restriction applies to any mapping, not just /dev/shm files.
QEMU's assumptions that pagesize is the alignment requirement
seem fairly widespread :-(

thanks
-- PMM

Re: [Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[Laszlo Ersek]

On 06/15/17 20:17, Peter Maydell wrote:
> It turns out that SPARC hosts impose a requirement that
> if you mmap MAP_SHARED|MAP_FIXED then the address must be
> aligned to SHMLBA alignment, which for SPARC is 16K and
> larger than the 8K page size:
>  http://lxr.linux.no/#linux+v4.10.1/arch/sparc/kernel/sys_sparc_64.c#L158
> 
> QEMU doesn't really cope with this. In particular, ivmshm
> doesn't work:
> 
> $ ppc64-softmmu/qemu-system-ppc64 -display none -device
> ivshmem,shm=/qtest-20273-1334572184,size=0x1
> qemu-system-ppc64: System page size 0x2000 is not enabled in
> page_size_mask (0x11000). Performance may be slow
> qemu-system-ppc64: -device
> ivshmem,shm=/qtest-20273-1334572184,size=0x1: ivshmem is deprecated,
> please use ivshmem-plain or ivshmem-doorbell instead
> Unexpected error in file_ram_alloc() at /home/pm215/qemu/exec.c:1599:
> qemu-system-ppc64: -device
> ivshmem,shm=/qtest-20273-1334572184,size=0x1: unable to map backing
> store for guest RAM: Invalid argument
> Aborted
> 
> (noticed because this is a make check failure).
> 
> exec.c calls qemu_fd_getpagesize() to figure out the alignment
> here (on a file in /dev/shm/) and ends up in its default codepath
> that returns getpagesize(), which is too small.
> 
> What's the best way to fix this? As far as I can tell
> the restriction applies to any mapping, not just /dev/shm files.
> QEMU's assumptions that pagesize is the alignment requirement
> seem fairly widespread :-(

In POSIX, SHMLBA is specified for shmat(), and not specified for mmap().

http://pubs.opengroup.org/onlinepubs/9699919799/functions/shmat.html
http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html

Apparently, /dev/shm (tmpfs) stands for the same sysv memory sharing
that shmat() stands for. Perhaps we can collect the /dev/shm mappings
and apply the shmat() restriction(s) to them explicitly -- but only to them.

If that's not the case, i.e. Linux on SPARC enforces SHMLBA for all
kinds of mmap() -- which I would call a POSIX conformance bug --, then
we can perhaps generally round up to SHMLBA within qemu_fd_getpagesize()
and qemu_mempath_getpagesize(), on SPARC hosts.

Laszlo

Re: [Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[Peter Maydell]

On 15 June 2017 at 20:13, Laszlo Ersek <lersek@redhat.com> wrote:
> If that's not the case, i.e. Linux on SPARC enforces SHMLBA for all
> kinds of mmap() -- which I would call a POSIX conformance bug --,

I'm pretty sure it does: the intention is to avoid violating
h/w cache aliasing constraints so the code enforces the
16k alignment restriction for any MAP_SHARED mapping.
(This is only awkward for MAP_FIXED since otherwise it can
just pick the address to suit its purposes anyway.)

(As an aside I wonder whether any still-used hardware has
the cache aliasing constraints in question :-))

thanks
-- PMM

Re: [Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[Richard Henderson]

On 06/16/2017 02:18 AM, Peter Maydell wrote:
> (As an aside I wonder whether any still-used hardware has
> the cache aliasing constraints in question :-))

I strongly doubt it.  Certainly none of the 64-bit chips have this.

Amusingly, while we require a 64-bit chip in TCG, we do allow elf32 via 
sparcv8plus, which still runs afoul of this check.  In hindsight, sparcv8plus 
probably should have relaxed this, but it's 20 years too late for that.


r~

Re: [Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[Peter Maydell]

On 19 June 2017 at 23:49, Richard Henderson <rth@twiddle.net> wrote:
> On 06/16/2017 02:18 AM, Peter Maydell wrote:
>>
>> (As an aside I wonder whether any still-used hardware has
>> the cache aliasing constraints in question :-))
>
>
> I strongly doubt it.  Certainly none of the 64-bit chips have this.

If so then it would be possible to make the kernel less strict
here, I think (without breaking back compat), though for QEMU's
purposes we probably ought to deal with the kernel as it stands.

thanks
-- PMM

Re: [Qemu-devel] QEMU issues on SPARC hosts due to more-than-page-alignment requirement for MAP_SHARED|MAP_FIXED...

[John Paul Adrian Glaubitz]

Hi Peter!

On Thu, Jun 15, 2017 at 07:17:14PM +0100, Peter Maydell wrote:
> What's the best way to fix this? As far as I can tell
> the restriction applies to any mapping, not just /dev/shm files.
> QEMU's assumptions that pagesize is the alignment requirement
> seem fairly widespread :-(

I suggest posting your question to the sparclinux Linux kernel mailing
list [1]. There are many knowledgable SPARC people on the list,
including lots of folk from Oracle and David Miller who is the
original creator of the SPARC port of Linux.

Adrian

> [1] http://vger.kernel.org/vger-lists.html#sparclinux

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913