[Qemu-arm] recvfrom error with linux-user emulating armhf on aarch64

[Qemu-arm] recvfrom error with linux-user emulating armhf on aarch64

[Guido Günther]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

Hi,
Running the attached program⁰ in a armhf chroot on a arm64 host¹ like:

    chroot . /usr/bin/qemu-arm-static tmp/nl-bad-addr

fails with "Bad address" when invoking audit_log_acct_message. strace looks like:

  [..snip..]
  31572 socket(AF_NETLINK, SOCK_RAW, NETLINK_AUDIT) = 3
  ...
  31572 sendto(3, {{len=124, type=0x454 /* NLMSG_??? */, flags=NLM_F_REQUEST|NLM_F_ACK, seq=1, pid=0}, "op=test:message acct=\"?\" exe=\"/tmp/nl-bad-addr\" hostname=localhost addr=? terminal=/dev/pts/2 res=success\0\0\0"}, 124, 0, 0xfffffa3897d0, 0) = 124
  31572 ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=500000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=499993180})
  31572 recvfrom(3, 0x112a50eb4, 8988, MSG_PEEK|MSG_DONTWAIT, 0xfffffa3897e0, 0x42) = -1 EFAULT (Bad address)
  [..snip..]

And it fails in

   ret = get_errno(safe_recvfrom(fd, host_msg, len, flags,
                                      addr, &addrlen));

in linux-user/syscall.c:do_recvfrom but I have no idea yet what causes this.

However the same works without problems when the host is a x86_64 box.
Any pointers in debugging this further would be appreciated.

The issue is also present in qemu master. The overall goal is to get

   qemu-debootstrap --arch armhf buster chroot

to succeed on an aarch64 host as it does on x86_64.
Cheers,
 -- Guido

P.S.: Please cc: me on replies since I'm currently not subscribed to qemu-arm.

⁹: which is a stripped down testcase for /usr/bin/chfn failing
¹: the scaleway server this runs on doesn't support armhf.

[-- Attachment #2: audit.c --]
[-- Type: text/x-csrc, Size: 555 bytes --]

// gcc -Wall -O2 audit.c -laudit

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

#include <libaudit.h>


void err(const char* str)
{
  fprintf(stderr, "%s\n", str);
  exit(1);
}

int main()
{
  int audit_fd, rc;
  audit_fd = audit_open();

  if (audit_fd < 0)
      err("audit_open");
    
  rc = audit_log_acct_message (audit_fd, AUDIT_USER_CHAUTHTOK, NULL, "test:message",
                              "?", -1, "localhost", NULL, "/dev/pts/2", 1);
  if (rc < 0)
     err("audit_log_acct_message");

  audit_close(audit_fd);
  return 0;
}

[Qemu-arm] recvfrom error with linux-user emulating armhf on aarch64

[Guido Günther]

[-- Attachment #1: Type: text/plain, Size: 1514 bytes --]

Hi,
Running the attached program⁰ in a armhf chroot on a arm64 host¹ like:

    chroot . /usr/bin/qemu-arm-static tmp/nl-bad-addr

fails with "Bad address" when invoking audit_log_acct_message. strace looks like:

  [..snip..]
  31572 socket(AF_NETLINK, SOCK_RAW, NETLINK_AUDIT) = 3
  ...
  31572 sendto(3, {{len=124, type=0x454 /* NLMSG_??? */, flags=NLM_F_REQUEST|NLM_F_ACK, seq=1, pid=0}, "op=test:message acct=\"?\" exe=\"/tmp/nl-bad-addr\" hostname=localhost addr=? terminal=/dev/pts/2 res=success\0\0\0"}, 124, 0, 0xfffffa3897d0, 0) = 124
  31572 ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=500000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=499993180})
  31572 recvfrom(3, 0x112a50eb4, 8988, MSG_PEEK|MSG_DONTWAIT, 0xfffffa3897e0, 0x42) = -1 EFAULT (Bad address)
  [..snip..]

And it fails in

   ret = get_errno(safe_recvfrom(fd, host_msg, len, flags,
                                      addr, &addrlen));

in linux-user/syscall.c:do_recvfrom but I have no idea yet what causes this.

However the same works without problems when the host is a x86_64 box.
Any pointers in debugging this further would be appreciated.

The issue is also present in qemu master. The overall goal is to get

   qemu-debootstrap --arch armhf buster chroot

to succeed on an aarch64 host as it does on x86_64.
Cheers,
 -- Guido

⁹: which is a stripped down testcase for /usr/bin/chfn failing
¹: the scaleway server this runs on doesn't support armhf.

[-- Attachment #2: audit.c --]
[-- Type: text/x-csrc, Size: 555 bytes --]

// gcc -Wall -O2 audit.c -laudit

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

#include <libaudit.h>


void err(const char* str)
{
  fprintf(stderr, "%s\n", str);
  exit(1);
}

int main()
{
  int audit_fd, rc;
  audit_fd = audit_open();

  if (audit_fd < 0)
      err("audit_open");
    
  rc = audit_log_acct_message (audit_fd, AUDIT_USER_CHAUTHTOK, NULL, "test:message",
                              "?", -1, "localhost", NULL, "/dev/pts/2", 1);
  if (rc < 0)
     err("audit_log_acct_message");

  audit_close(audit_fd);
  return 0;
}

Re: [Qemu-arm] recvfrom error with linux-user emulating armhf on aarch64

[Philippe Mathieu-Daudé]

[-- Attachment #1.1: Type: text/plain, Size: 2503 bytes --]

Hi Guido, Laurent,

On 01/23/2018 09:05 AM, Guido Günther wrote:
> Hi,
> Running the attached program⁰ in a armhf chroot on a arm64 host¹ like:
> 
>     chroot . /usr/bin/qemu-arm-static tmp/nl-bad-addr
> 
> fails with "Bad address" when invoking audit_log_acct_message. strace looks like:
> 
>   [..snip..]
>   31572 socket(AF_NETLINK, SOCK_RAW, NETLINK_AUDIT) = 3
>   ...
>   31572 sendto(3, {{len=124, type=0x454 /* NLMSG_??? */, flags=NLM_F_REQUEST|NLM_F_ACK, seq=1, pid=0}, "op=test:message acct=\"?\" exe=\"/tmp/nl-bad-addr\" hostname=localhost addr=? terminal=/dev/pts/2 res=success\0\0\0"}, 124, 0, 0xfffffa3897d0, 0) = 124

the do_sendto() seems to correctly handle the u64 target_addr

>   31572 ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=500000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=499993180})
>   31572 recvfrom(3, 0x112a50eb4, 8988, MSG_PEEK|MSG_DONTWAIT, 0xfffffa3897e0, 0x42) = -1 EFAULT (Bad address)

but the recvfrom() code is:

static abi_long do_recvfrom(int fd, abi_ulong msg, size_t len,
                            int flags,
                            abi_ulong target_addr,
                            abi_ulong target_addrlen)
{
    [...]
    if (target_addr) {
        if (get_user_u32(addrlen, target_addrlen)) {
            ret = -TARGET_EFAULT;
            goto fail;
        }

so current code won't work on 64-bit host, and we are probably lucky it
works on x86_64.

>   [..snip..]
> 
> And it fails in
> 
>    ret = get_errno(safe_recvfrom(fd, host_msg, len, flags,
>                                       addr, &addrlen));
> 
> in linux-user/syscall.c:do_recvfrom but I have no idea yet what causes this.
> 
> However the same works without problems when the host is a x86_64 box.
> Any pointers in debugging this further would be appreciated.

Laurent, in do_syscall() should we swap the pid back?

@@ -10628,4 +10628,5 @@ abi_long do_syscall(...
         target_header->version = tswap32(header.version);
+        target_header->pid = tswap32(header.pid);
         unlock_user_struct(target_header, arg1, 1);

> 
> The issue is also present in qemu master. The overall goal is to get
> 
>    qemu-debootstrap --arch armhf buster chroot
> 
> to succeed on an aarch64 host as it does on x86_64.
> Cheers,
>  -- Guido
> 
> ⁹: which is a stripped down testcase for /usr/bin/chfn failing
> ¹: the scaleway server this runs on doesn't support armhf.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-arm] recvfrom error with linux-user emulating armhf on aarch64

[Philippe Mathieu-Daudé]

[-- Attachment #1.1: Type: text/plain, Size: 2814 bytes --]

Hi Guido,

On 01/23/2018 11:42 AM, Philippe Mathieu-Daudé wrote:
> Hi Guido, Laurent,
> 
> On 01/23/2018 09:05 AM, Guido Günther wrote:
>> Hi,
>> Running the attached program⁰ in a armhf chroot on a arm64 host¹ like:
>>
>>     chroot . /usr/bin/qemu-arm-static tmp/nl-bad-addr
>>
>> fails with "Bad address" when invoking audit_log_acct_message. strace looks like:
>>
>>   [..snip..]
>>   31572 socket(AF_NETLINK, SOCK_RAW, NETLINK_AUDIT) = 3
>>   ...
>>   31572 sendto(3, {{len=124, type=0x454 /* NLMSG_??? */, flags=NLM_F_REQUEST|NLM_F_ACK, seq=1, pid=0}, "op=test:message acct=\"?\" exe=\"/tmp/nl-bad-addr\" hostname=localhost addr=? terminal=/dev/pts/2 res=success\0\0\0"}, 124, 0, 0xfffffa3897d0, 0) = 124
> 
> the do_sendto() seems to correctly handle the u64 target_addr
> 
>>   31572 ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=0, tv_nsec=500000000}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=0, tv_nsec=499993180})
>>   31572 recvfrom(3, 0x112a50eb4, 8988, MSG_PEEK|MSG_DONTWAIT, 0xfffffa3897e0, 0x42) = -1 EFAULT (Bad address)
> 
> but the recvfrom() code is:
> 
> static abi_long do_recvfrom(int fd, abi_ulong msg, size_t len,
>                             int flags,
>                             abi_ulong target_addr,
>                             abi_ulong target_addrlen)
> {
>     [...]
>     if (target_addr) {
>         if (get_user_u32(addrlen, target_addrlen)) {
>             ret = -TARGET_EFAULT;
>             goto fail;
>         }
> 
> so current code won't work on 64-bit host, and we are probably lucky it
> works on x86_64.
> 
>>   [..snip..]
>>
>> And it fails in
>>
>>    ret = get_errno(safe_recvfrom(fd, host_msg, len, flags,
>>                                       addr, &addrlen));
>>
>> in linux-user/syscall.c:do_recvfrom but I have no idea yet what causes this.
>>
>> However the same works without problems when the host is a x86_64 box.
>> Any pointers in debugging this further would be appreciated.

I proposed a fix for this, do you mind testing it?

http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg05670.html

Regards,

Phil.

> 
> Laurent, in do_syscall() should we swap the pid back?
> 
> @@ -10628,4 +10628,5 @@ abi_long do_syscall(...
>          target_header->version = tswap32(header.version);
> +        target_header->pid = tswap32(header.pid);
>          unlock_user_struct(target_header, arg1, 1);
> 
>>
>> The issue is also present in qemu master. The overall goal is to get
>>
>>    qemu-debootstrap --arch armhf buster chroot
>>
>> to succeed on an aarch64 host as it does on x86_64.
>> Cheers,
>>  -- Guido
>>
>> ⁹: which is a stripped down testcase for /usr/bin/chfn failing
>> ¹: the scaleway server this runs on doesn't support armhf.
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Qemu-arm] recvfrom error with linux-user emulating armhf on aarch64

[Laurent Vivier]

[-- Attachment #1.1: Type: text/plain, Size: 490 bytes --]

Le 23/01/2018 à 15:42, Philippe Mathieu-Daudé a écrit :
...
> Laurent, in do_syscall() should we swap the pid back?
> 
> @@ -10628,4 +10628,5 @@ abi_long do_syscall(...
>          target_header->version = tswap32(header.version);
> +        target_header->pid = tswap32(header.pid);
>          unlock_user_struct(target_header, arg1, 1);
> 

The kernel doesn't change it, so we don't need to update it.

(CC: Peter as he wrote the capget/capset code)

Thanks,
Laurent


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]