[Qemu-devel] [PATCH 2/2] linux-user: correct core dump format

[Qemu-devel] [PATCH 2/2] linux-user: correct core dump format

[Laurent Vivier]

This patch allows to really use the core dumped by qemu with guest
architecture tools.

- it adds a missing bswap_phdr() for the program headers
  of memory regions.

  "objdump -x" sample:

BEFORE:

0x1000000 off    0x00200000 vaddr 0x00000400 paddr 0x00000000 align 2**21
         filesz 0x00000000 memsz 0x00100000 flags ---
0x1000000 off    0x00200000 vaddr 0x00100400 paddr 0x00000000 align 2**21
         filesz 0x00000000 memsz 0x00080000 flags --- 6000000

AFTER:

    LOAD off    0x00002000 vaddr 0x00040000 paddr 0x00000000 align 2**13
         filesz 0x00000000 memsz 0x00001000 flags ---
    LOAD off    0x00002000 vaddr 0x00041000 paddr 0x00000000 align 2**13
         filesz 0x00000000 memsz 0x00000800 flags rw-

- it doesn't pad the note size to sizeof(int32_t).
  On m68k the NT_PRSTATUS note size is 154 and
  must not be rounded up to 156, because this value is checked by
  objdump and gdb.

  "gdb" symptoms:

      "warning: Couldn't find general-purpose registers in core file."

  "objdump -x" sample:

BEFORE:

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 note0         000001c4  00000000  00000000  000003b4  2**0
                  CONTENTS, READONLY
  1 .auxv         00000070  00000000  00000000  00000508  2**2
                  CONTENTS
  2 proc1         00100000  00000400  00000000  00200000  2**10
                  READONLY

AFTER:

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 note0         000001c4  00000000  00000000  000003b4  2**0
                  CONTENTS, READONLY
  1 .reg/19022    00000050  00000000  00000000  0000040e  2**2
                  CONTENTS
  2 .reg          00000050  00000000  00000000  0000040e  2**2
                  CONTENTS
  3 .auxv         00000070  00000000  00000000  00000508  2**2
                  CONTENTS
  4 load1         00000000  00040000  00000000  00002000  2**13
                  ALLOC, READONLY

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/elfload.c |   16 +++++++++-------
 1 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 33d776d..61f65d6 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -1688,6 +1688,7 @@ struct memelfnote {
     size_t     namesz_rounded;
     int        type;
     size_t     datasz;
+    size_t     datasz_rounded;
     void       *data;
     size_t     notesz;
 };
@@ -1713,7 +1714,7 @@ struct target_elf_prstatus {
     struct target_timeval pr_cstime; /* XXX Cumulative system time */
     target_elf_gregset_t      pr_reg;       /* GP registers */
     int                pr_fpvalid;   /* XXX */
-};
+} __attribute__((__aligned__(TARGET_ALIGNMENT))) __attribute__((packed));
 
 #define ELF_PRARGSZ     (80) /* Number of chars for args */
 
@@ -1963,7 +1964,9 @@ static void fill_note(struct memelfnote *note, const char *name, int type,
     note->namesz = namesz;
     note->namesz_rounded = roundup(namesz, sizeof (int32_t));
     note->type = type;
-    note->datasz = roundup(sz, sizeof (int32_t));;
+    note->datasz = sz;
+    note->datasz_rounded = roundup(sz, sizeof (int32_t));
+
     note->data = data;
 
     /*
@@ -1971,7 +1974,7 @@ static void fill_note(struct memelfnote *note, const char *name, int type,
      * ELF document.
      */
     note->notesz = sizeof (struct elf_note) +
-        note->namesz_rounded + note->datasz;
+        note->namesz_rounded + note->datasz_rounded;
 }
 
 static void fill_elf_header(struct elfhdr *elf, int segs, uint16_t machine,
@@ -2191,7 +2194,7 @@ static int write_note(struct memelfnote *men, int fd)
         return (-1);
     if (dump_write(fd, men->name, men->namesz_rounded) != 0)
         return (-1);
-    if (dump_write(fd, men->data, men->datasz) != 0)
+    if (dump_write(fd, men->data, men->datasz_rounded) != 0)
         return (-1);
 
     return (0);
@@ -2407,7 +2410,7 @@ static int elf_core_dump(int signr, const CPUState *env)
      * ELF specification wants data to start at page boundary so
      * we align it here.
      */
-    offset = roundup(offset, ELF_EXEC_PAGESIZE);
+    data_offset = offset = roundup(offset, ELF_EXEC_PAGESIZE);
 
     /*
      * Write program headers for memory regions mapped in
@@ -2430,6 +2433,7 @@ static int elf_core_dump(int signr, const CPUState *env)
             phdr.p_flags |= PF_X;
         phdr.p_align = ELF_EXEC_PAGESIZE;
 
+        bswap_phdr(&phdr, 1);
         dump_write(fd, &phdr, sizeof (phdr));
     }
 
@@ -2441,8 +2445,6 @@ static int elf_core_dump(int signr, const CPUState *env)
         goto out;
 
     /* align data to page boundary */
-    data_offset = lseek(fd, 0, SEEK_CUR);
-    data_offset = TARGET_PAGE_ALIGN(data_offset);
     if (lseek(fd, data_offset, SEEK_SET) != data_offset)
         goto out;
 
-- 
1.7.1

[Qemu-devel] [PATCH 0/2][v3] correct core dump format

[Laurent Vivier]

This is the v3 of my patch correcting the core dump format.

It introduces a new parameter of the target: the datatype alignment size.

Targets like i386, mips or ppc align (short, int, long, long long) on
(2, 4, 4, 8), target like x86_64 aligns on (2, 4, 8, 8)

but arm aligns on (2, 4, 4, 4) and m68k (680x0) on (2, 2, 2, 2).

And this knowledge is needed to correctly generate a core dump.

For other targets, please update the patch with your favorite one.

[PATCH 1/2] linux-user: Define target alignment size
[PATCH 2/2] linux-user: correct core dump format

[Qemu-devel] [PATCH 1/2] linux-user: Define target alignment size

[Laurent Vivier]

Datatype alignment can be found using following application:

int main(void)
{
	printf("alignof(short) %ld\n", __alignof__(short));
	printf("alignof(int) %ld\n", __alignof__(int));
	printf("alignof(long) %ld\n", __alignof__(long));
	printf("alignof(long long) %ld\n", __alignof__(long long));
}

This patch includes following alignments:

i386

   alignof(short) 2
   alignof(int) 4
   alignof(long) 4
   alignof(long long) 8

 x86_64

   alignof(short) 2
   alignof(int) 4
   alignof(long) 8
   alignof(long long) 8

 arm

   alignof(short) 2
   alignof(int) 4
   alignof(long) 4
   alignof(long long) 4

 m68k (680x0)

   alignof(short) 2
   alignof(int) 2
   alignof(long) 2
   alignof(long long) 2

 mips

   alignof(short) 2
   alignof(int) 4
   alignof(long) 4
   alignof(long long) 8

 ppc

   alignof(short) 2
   alignof(int) 4
   alignof(long) 4
   alignof(long long) 8

for other targets, use by default (2,4,4,8).

Please, update for your favorite target...

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
v2: compute align size for each basic datatype

 configure  |   13 +++++++++++++
 cpu-defs.h |   14 ++++++++++----
 2 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/configure b/configure
index 25381e5..efc6fa4 100755
--- a/configure
+++ b/configure
@@ -2919,6 +2919,10 @@ target_nptl="no"
 interp_prefix1=`echo "$interp_prefix" | sed "s/%M/$target_arch2/g"`
 echo "CONFIG_QEMU_INTERP_PREFIX=\"$interp_prefix1\"" >> $config_target_mak
 gdb_xml_files=""
+target_short_alignment=2
+target_int_alignment=4
+target_long_alignment=4
+target_long_long_alignment=8
 
 TARGET_ARCH="$target_arch2"
 TARGET_BASE_ARCH=""
@@ -2931,6 +2935,7 @@ case "$target_arch2" in
   x86_64)
     TARGET_BASE_ARCH=i386
     target_phys_bits=64
+    target_long_alignment=8
   ;;
   alpha)
     target_phys_bits=64
@@ -2942,6 +2947,7 @@ case "$target_arch2" in
     target_nptl="yes"
     gdb_xml_files="arm-core.xml arm-vfp.xml arm-vfp3.xml arm-neon.xml"
     target_phys_bits=32
+    target_long_long_alignment=4
   ;;
   cris)
     target_nptl="yes"
@@ -2951,6 +2957,9 @@ case "$target_arch2" in
     bflt="yes"
     gdb_xml_files="cf-core.xml cf-fp.xml"
     target_phys_bits=32
+    target_int_alignment=2
+    target_long_alignment=2
+    target_long_long_alignment=2
   ;;
   microblaze)
     bflt="yes"
@@ -3029,6 +3038,10 @@ case "$target_arch2" in
     exit 1
   ;;
 esac
+echo "TARGET_SHORT_ALIGNMENT=$target_short_alignment" >> $config_target_mak
+echo "TARGET_INT_ALIGNMENT=$target_int_alignment" >> $config_target_mak
+echo "TARGET_LONG_ALIGNMENT=$target_long_alignment" >> $config_target_mak
+echo "TARGET_LONG_LONG_ALIGNMENT=$target_long_long_alignment" >> $config_target_mak
 echo "TARGET_ARCH=$TARGET_ARCH" >> $config_target_mak
 target_arch_name="`echo $TARGET_ARCH | tr '[:lower:]' '[:upper:]'`"
 echo "TARGET_$target_arch_name=y" >> $config_target_mak
diff --git a/cpu-defs.h b/cpu-defs.h
index 8d4bf86..98c6718 100644
--- a/cpu-defs.h
+++ b/cpu-defs.h
@@ -37,16 +37,22 @@
 
 #define TARGET_LONG_SIZE (TARGET_LONG_BITS / 8)
 
+typedef int16_t target_short __attribute__ ((aligned(TARGET_SHORT_ALIGNMENT)));
+typedef uint16_t target_ushort __attribute__((aligned(TARGET_SHORT_ALIGNMENT)));
+typedef int32_t target_int __attribute__((aligned(TARGET_INT_ALIGNMENT)));
+typedef uint32_t target_uint __attribute__((aligned(TARGET_INT_ALIGNMENT)));
+typedef int64_t target_long_long __attribute__((aligned(TARGET_LONG_LONG_ALIGNMENT)));
+typedef uint64_t target_ulong_long __attribute__((aligned(TARGET_LONG_LONG_ALIGNMENT)));
 /* target_ulong is the type of a virtual address */
 #if TARGET_LONG_SIZE == 4
-typedef int32_t target_long;
-typedef uint32_t target_ulong;
+typedef int32_t target_long __attribute__((aligned(TARGET_LONG_ALIGNMENT)));
+typedef uint32_t target_ulong __attribute__((aligned(TARGET_LONG_ALIGNMENT)));
 #define TARGET_FMT_lx "%08x"
 #define TARGET_FMT_ld "%d"
 #define TARGET_FMT_lu "%u"
 #elif TARGET_LONG_SIZE == 8
-typedef int64_t target_long;
-typedef uint64_t target_ulong;
+typedef int64_t target_long __attribute__((aligned(TARGET_LONG_ALIGNMENT)));
+typedef uint64_t target_ulong __attribute__((aligned(TARGET_LONG_ALIGNMENT)));
 #define TARGET_FMT_lx "%016" PRIx64
 #define TARGET_FMT_ld "%" PRId64
 #define TARGET_FMT_lu "%" PRIu64
-- 
1.7.1

[Qemu-devel] [PATCH 2/2] linux-user: correct core dump format

[Laurent Vivier]

This patch allows to really use the core dumped by qemu with guest
architecture tools.

- it adds a missing bswap_phdr() for the program headers
  of memory regions.

  "objdump -x" sample:

BEFORE:

0x1000000 off    0x00200000 vaddr 0x00000400 paddr 0x00000000 align 2**21
         filesz 0x00000000 memsz 0x00100000 flags ---
0x1000000 off    0x00200000 vaddr 0x00100400 paddr 0x00000000 align 2**21
         filesz 0x00000000 memsz 0x00080000 flags --- 6000000

AFTER:

    LOAD off    0x00002000 vaddr 0x00040000 paddr 0x00000000 align 2**13
         filesz 0x00000000 memsz 0x00001000 flags ---
    LOAD off    0x00002000 vaddr 0x00041000 paddr 0x00000000 align 2**13
         filesz 0x00000000 memsz 0x00000800 flags rw-

- it doesn't pad the note size to sizeof(int32_t).
  On m68k the NT_PRSTATUS note size is 154 and
  must not be rounded up to 156, because this value is checked by
  objdump and gdb.

  "gdb" symptoms:

      "warning: Couldn't find general-purpose registers in core file."

  "objdump -x" sample:

BEFORE:

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 note0         000001c4  00000000  00000000  000003b4  2**0
                  CONTENTS, READONLY
  1 .auxv         00000070  00000000  00000000  00000508  2**2
                  CONTENTS
  2 proc1         00100000  00000400  00000000  00200000  2**10
                  READONLY

AFTER:

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 note0         000001c4  00000000  00000000  000003b4  2**0
                  CONTENTS, READONLY
  1 .reg/19022    00000050  00000000  00000000  0000040e  2**2
                  CONTENTS
  2 .reg          00000050  00000000  00000000  0000040e  2**2
                  CONTENTS
  3 .auxv         00000070  00000000  00000000  00000508  2**2
                  CONTENTS
  4 load1         00000000  00040000  00000000  00002000  2**13
                  ALLOC, READONLY

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
v2: use a predefined alignment size for target_elf_prstatus
v3: use target_<type> aligned according target properties

 linux-user/elfload.c |   34 ++++++++++++++++++----------------
 1 files changed, 18 insertions(+), 16 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 2de83e4..fe5410e 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -103,13 +103,13 @@ enum {
 
 typedef target_ulong    target_elf_greg_t;
 #ifdef USE_UID16
-typedef uint16_t        target_uid_t;
-typedef uint16_t        target_gid_t;
+typedef target_ushort   target_uid_t;
+typedef target_ushort   target_gid_t;
 #else
-typedef uint32_t        target_uid_t;
-typedef uint32_t        target_gid_t;
+typedef target_uint     target_uid_t;
+typedef target_uint     target_gid_t;
 #endif
-typedef int32_t         target_pid_t;
+typedef target_int      target_pid_t;
 
 #ifdef TARGET_I386
 
@@ -1761,19 +1761,20 @@ struct memelfnote {
     size_t     namesz_rounded;
     int        type;
     size_t     datasz;
+    size_t     datasz_rounded;
     void       *data;
     size_t     notesz;
 };
 
 struct target_elf_siginfo {
-    int  si_signo; /* signal number */
-    int  si_code;  /* extra code */
-    int  si_errno; /* errno */
+    target_int  si_signo; /* signal number */
+    target_int  si_code;  /* extra code */
+    target_int  si_errno; /* errno */
 };
 
 struct target_elf_prstatus {
     struct target_elf_siginfo pr_info;      /* Info associated with signal */
-    short              pr_cursig;    /* Current signal */
+    target_short       pr_cursig;    /* Current signal */
     target_ulong       pr_sigpend;   /* XXX */
     target_ulong       pr_sighold;   /* XXX */
     target_pid_t       pr_pid;
@@ -1785,7 +1786,7 @@ struct target_elf_prstatus {
     struct target_timeval pr_cutime; /* XXX Cumulative user time */
     struct target_timeval pr_cstime; /* XXX Cumulative system time */
     target_elf_gregset_t      pr_reg;       /* GP registers */
-    int                pr_fpvalid;   /* XXX */
+    target_int         pr_fpvalid;   /* XXX */
 };
 
 #define ELF_PRARGSZ     (80) /* Number of chars for args */
@@ -2036,7 +2037,9 @@ static void fill_note(struct memelfnote *note, const char *name, int type,
     note->namesz = namesz;
     note->namesz_rounded = roundup(namesz, sizeof (int32_t));
     note->type = type;
-    note->datasz = roundup(sz, sizeof (int32_t));;
+    note->datasz = sz;
+    note->datasz_rounded = roundup(sz, sizeof (int32_t));
+
     note->data = data;
 
     /*
@@ -2044,7 +2047,7 @@ static void fill_note(struct memelfnote *note, const char *name, int type,
      * ELF document.
      */
     note->notesz = sizeof (struct elf_note) +
-        note->namesz_rounded + note->datasz;
+        note->namesz_rounded + note->datasz_rounded;
 }
 
 static void fill_elf_header(struct elfhdr *elf, int segs, uint16_t machine,
@@ -2264,7 +2267,7 @@ static int write_note(struct memelfnote *men, int fd)
         return (-1);
     if (dump_write(fd, men->name, men->namesz_rounded) != 0)
         return (-1);
-    if (dump_write(fd, men->data, men->datasz) != 0)
+    if (dump_write(fd, men->data, men->datasz_rounded) != 0)
         return (-1);
 
     return (0);
@@ -2480,7 +2483,7 @@ static int elf_core_dump(int signr, const CPUState *env)
      * ELF specification wants data to start at page boundary so
      * we align it here.
      */
-    offset = roundup(offset, ELF_EXEC_PAGESIZE);
+    data_offset = offset = roundup(offset, ELF_EXEC_PAGESIZE);
 
     /*
      * Write program headers for memory regions mapped in
@@ -2503,6 +2506,7 @@ static int elf_core_dump(int signr, const CPUState *env)
             phdr.p_flags |= PF_X;
         phdr.p_align = ELF_EXEC_PAGESIZE;
 
+        bswap_phdr(&phdr, 1);
         dump_write(fd, &phdr, sizeof (phdr));
     }
 
@@ -2514,8 +2518,6 @@ static int elf_core_dump(int signr, const CPUState *env)
         goto out;
 
     /* align data to page boundary */
-    data_offset = lseek(fd, 0, SEEK_CUR);
-    data_offset = TARGET_PAGE_ALIGN(data_offset);
     if (lseek(fd, data_offset, SEEK_SET) != data_offset)
         goto out;
 
-- 
1.7.1

Re: [Qemu-devel] [PATCH 1/2] linux-user: Define target alignment size

[Blue Swirl]

On Sun, Feb 13, 2011 at 4:22 AM, Laurent Vivier <laurent@vivier.eu> wrote:
> Datatype alignment can be found using following application:
>
> int main(void)
> {
>        printf("alignof(short) %ld\n", __alignof__(short));
>        printf("alignof(int) %ld\n", __alignof__(int));
>        printf("alignof(long) %ld\n", __alignof__(long));
>        printf("alignof(long long) %ld\n", __alignof__(long long));
> }
>
> This patch includes following alignments:
>
> i386
>
>   alignof(short) 2
>   alignof(int) 4
>   alignof(long) 4
>   alignof(long long) 8
>
>  x86_64
>
>   alignof(short) 2
>   alignof(int) 4
>   alignof(long) 8
>   alignof(long long) 8
>
>  arm
>
>   alignof(short) 2
>   alignof(int) 4
>   alignof(long) 4
>   alignof(long long) 4
>
>  m68k (680x0)
>
>   alignof(short) 2
>   alignof(int) 2
>   alignof(long) 2
>   alignof(long long) 2
>
>  mips
>
>   alignof(short) 2
>   alignof(int) 4
>   alignof(long) 4
>   alignof(long long) 8
>
>  ppc
>
>   alignof(short) 2
>   alignof(int) 4
>   alignof(long) 4
>   alignof(long long) 8
>
> for other targets, use by default (2,4,4,8).
>
> Please, update for your favorite target...

For Sparc32 (I think also sparc32plus), the default is OK.

For Sparc64, please use 2, 4, 8, 8. I'd guess other 64 bit platforms
(Alpha, MIPS64, PPC64 etc) should use the same.

Does GCC produce correct code using the attributes on strictly aligned
host, when the target is less strictly aligned?

Should the alignment of floating point variables be specified as well?
The strict alignment required for doubles is 4, but recommended
alignment is 8, I'm not sure which one is used for structures
containing doubles.

Re: [Qemu-devel] [PATCH 1/2] linux-user: Define target alignment size

[Laurent Vivier]

Le dimanche 13 février 2011 à 10:24 +0200, Blue Swirl a écrit :
> On Sun, Feb 13, 2011 at 4:22 AM, Laurent Vivier <laurent@vivier.eu> wrote:
> > Datatype alignment can be found using following application:
> >
> > int main(void)
> > {
> >        printf("alignof(short) %ld\n", __alignof__(short));
> >        printf("alignof(int) %ld\n", __alignof__(int));
> >        printf("alignof(long) %ld\n", __alignof__(long));
> >        printf("alignof(long long) %ld\n", __alignof__(long long));
> > }
> >
> > This patch includes following alignments:
> >
> > i386
> >
> >   alignof(short) 2
> >   alignof(int) 4
> >   alignof(long) 4
> >   alignof(long long) 8
> >
> >  x86_64
> >
> >   alignof(short) 2
> >   alignof(int) 4
> >   alignof(long) 8
> >   alignof(long long) 8
> >
> >  arm
> >
> >   alignof(short) 2
> >   alignof(int) 4
> >   alignof(long) 4
> >   alignof(long long) 4
> >
> >  m68k (680x0)
> >
> >   alignof(short) 2
> >   alignof(int) 2
> >   alignof(long) 2
> >   alignof(long long) 2
> >
> >  mips
> >
> >   alignof(short) 2
> >   alignof(int) 4
> >   alignof(long) 4
> >   alignof(long long) 8
> >
> >  ppc
> >
> >   alignof(short) 2
> >   alignof(int) 4
> >   alignof(long) 4
> >   alignof(long long) 8
> >
> > for other targets, use by default (2,4,4,8).
> >
> > Please, update for your favorite target...
> 
> For Sparc32 (I think also sparc32plus), the default is OK.
> 
> For Sparc64, please use 2, 4, 8, 8. I'd guess other 64 bit platforms
> (Alpha, MIPS64, PPC64 etc) should use the same.

OK, I update my patch.

> Does GCC produce correct code using the attributes on strictly aligned
> host, when the target is less strictly aligned?

It seems it is OK. I did some tests into a mips-linux-user chroot (sparc
one is broken ;-) ) :

mips is a strictly aligned host, from gcc/config/mips/mips.h:

#define STRICT_ALIGNMENT 1

aligments are:

   alignof(short) 2
   alignof(int) 4
   alignof(long) 4
   alignof(long long) 8

We try to align int on 2.

#include <stdio.h>

typedef int target_int __attribute__((aligned(2)));

struct {
        char Z;
        target_int A;
} B;

int main(void)
{
        B.A = 0xdeadbeaf;
        printf("%d: %x\n", __alignof__(B.A), B.A);
}

./test_align
2: deadbeaf

disass:

        lw      $2,%got(B)($28)
        li      $3,-559087616                   # 0xffffffffdead0000
        ori     $3,$3,0xbeaf
        swl     $3,2($2)
        swr     $3,5($2)

normal case:

        lw      $3,%got(B)($28)
        li      $2,-559087616                   # 0xffffffffdead0000
        ori     $2,$2,0xbeaf
        sw      $2,4($3)

So, gcc seems smart enough to split the memory access in several ones
compatible with the strict alignment rules.

> Should the alignment of floating point variables be specified as well?

At the moment it seems useless.

> The strict alignment required for doubles is 4, but recommended
> alignment is 8, I'm not sure which one is used for structures
> containing doubles.

if necessary, some tests will be helpfull.

Thank you for your comments.

Regards,
Laurent 

-- 
--------------------- laurent@vivier.eu ----------------------
"Tout ce qui est impossible reste à accomplir"    Jules Verne
"Things are only impossible until they're not" Jean-Luc Picard