Re: [RFC PATCH 4/4] perf-probe: Support probing on offline cross-arch binary

Masami Hiramatsu
2016-08-25T07:06:16Z

On Wed, 24 Aug 2016 10:03:08 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Wed, Aug 24, 2016 at 02:58:41PM +0900, Masami Hiramatsu escreveu:
> > Support probing on offline cross-architecture binary by
> > adding getting the target machine arch from ELF and
> > choose correct register string for the machine.
> >
> > Here is an example:
> > -----
> > $ mkdir tracing
>
> No need for the sudo here, right?
>
> > $ sudo perf probe --outdir=./tracing --vmlinux=./vmlinux-arm \
> > do_sys_open '$vars'

Ah, actually, there might be something wrong processing path around
kernel_get_ref_reloc_sym(), it calls map__load() to load symbols in the
kernel, but it actually loads /proc/kallsyms, and if kptr_restrict is
enabled fails to find reloc_sym, because all addresses are 0.
I think it should not load the kallsyms if symbol_conf.vmlinux_name is
given. But I'm not sure how.
(Moreover, it is very strange why we could get the correct ref_reloc_sym
address(_text) from kallsyms, I guess map__load() may load both of the
kallsyms and given vmlinux)
BTW, actually, the dso already loaded given vmlinux, so I would like
to know how to load a map from dso.

> Going with the --definition (or any other name that spits out what needs
> to go to kprobes_events, this could be further short circuited, ie.:
>
> $ perf probe --vmlinux=./vmlinux-arm do_sys_open '$vars'
> p:probe/do_sys_open _text+1282292 dfd=%r5:s32 filename=%r1:u32 flags=%r6:s32 mode=%r3:u16 op=-60(%sp) fd=%r4:s32 tmp=%r7:u32
> $
>
> Because in this case clearly the probe is not for the running kernel,
> thus we could imply that this is to generate probes for running on
> some other place.

Hmm, but if user gives a file what he think it is running kernel vmlinux to
--vmlinux, he might be upset. I would like to see the Buildid error in that case.

If he gives --definition, we may be sure that user is done with his intention.

Thanks,

> Also perhaps even this could be made shorter:
>
> $ perf probe vmlinux-arm do_sys_open '$vars'
> p:probe/do_sys_open _text+1282292 dfd=%r5:s32 filename=%r1:u32 flags=%r6:s32 mode=%r3:u16 op=-60(%sp) fd=%r4:s32 tmp=%r7:u32
> $
>
> As it can easily figure out the first argument _is_ a vmlinux :-)
>
> - Arnaldo
>
>
> > Added new event:
> > probe:do_sys_open (on do_sys_open with $vars)
> >
> > You can now use it in all perf tools, such as:
> >
> > perf record -e probe:do_sys_open -aR sleep 1
> >
> > $ cat tracing/kprobe_events
> > p:probe/do_sys_open _text+1282292 dfd=%r5:s32 filename=%r1:u32
> > flags=%r6:s32 mode=%r3:u16 op=-60(%sp) fd=%r4:s32 tmp=%r7:u32
> > -----
> > Here, we can get probe/do_sys_open event by "copy & paste" from
> > ./tracing/kprobe_events to target-machine's debugfs/tracing/kprobe_events
> >
> > To make sure it is correct:
> > -----
> > $ nm vmlinux-arm | grep "T _text"
> > 80008000 T _text
> > $ nm vmlinux-arm | grep "T do_sys_open"
> > 801410f4 T do_sys_open
> > $ expr `printf "%d + %d" 0x80008000 1282292`
> > 2148798708
> > $ printf "%x\n" 2148798708
> > 801410f4
> > -----
> > So "_text+12882292" indicates do_sys_open on the target binary correctly.
> >
> > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > ---
> > tools/perf/arch/arm/include/dwarf-regs-table.h | 9 +++
> > tools/perf/arch/arm64/include/dwarf-regs-table.h | 13 +++++
> > tools/perf/arch/powerpc/include/dwarf-regs-table.h | 27 ++++++++++
> > tools/perf/arch/s390/include/dwarf-regs-table.h | 8 +++
> > tools/perf/arch/sh/include/dwarf-regs-table.h | 25 +++++++++
> > tools/perf/arch/sparc/include/dwarf-regs-table.h | 18 +++++++
> > tools/perf/arch/x86/include/dwarf-regs-table.h | 14 +++++
> > tools/perf/arch/xtensa/include/dwarf-regs-table.h | 8 +++
> > tools/perf/util/Build | 1
> > tools/perf/util/dwarf-regs.c | 55 ++++++++++++++++++++
> > tools/perf/util/include/dwarf-regs.h | 6 ++
> > tools/perf/util/probe-finder.c | 27 ++++++----
> > tools/perf/util/probe-finder.h | 1
> > 13 files changed, 201 insertions(+), 11 deletions(-)
> > create mode 100644 tools/perf/arch/arm/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/arm64/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/powerpc/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/s390/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/sh/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/sparc/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/x86/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/arch/xtensa/include/dwarf-regs-table.h
> > create mode 100644 tools/perf/util/dwarf-regs.c
> >
> > diff --git a/tools/perf/arch/arm/include/dwarf-regs-table.h b/tools/perf/arch/arm/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..f298d03
> > --- /dev/null
> > +++ b/tools/perf/arch/arm/include/dwarf-regs-table.h
> > @@ -0,0 +1,9 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +static const char * const arm_regstr_tbl[] = {
> > + "%r0", "%r1", "%r2", "%r3", "%r4",
> > + "%r5", "%r6", "%r7", "%r8", "%r9", "%r10",
> > + "%fp", "%ip", "%sp", "%lr", "%pc",
> > +};
> > +#endif
> > diff --git a/tools/perf/arch/arm64/include/dwarf-regs-table.h b/tools/perf/arch/arm64/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..2675936
> > --- /dev/null
> > +++ b/tools/perf/arch/arm64/include/dwarf-regs-table.h
> > @@ -0,0 +1,13 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +static const char * const aarch64_regstr_tbl[] = {
> > + "%r0", "%r1", "%r2", "%r3", "%r4",
> > + "%r5", "%r6", "%r7", "%r8", "%r9",
> > + "%r10", "%r11", "%r12", "%r13", "%r14",
> > + "%r15", "%r16", "%r17", "%r18", "%r19",
> > + "%r20", "%r21", "%r22", "%r23", "%r24",
> > + "%r25", "%r26", "%r27", "%r28", "%r29",
> > + "%lr", "%sp",
> > +};
> > +#endif
> > diff --git a/tools/perf/arch/powerpc/include/dwarf-regs-table.h b/tools/perf/arch/powerpc/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..db4730f
> > --- /dev/null
> > +++ b/tools/perf/arch/powerpc/include/dwarf-regs-table.h
> > @@ -0,0 +1,27 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +/*
> > + * Reference:
> > + * http://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi-1.9.html
> > + * http://refspecs.linux-foundation.org/elf/elfspec_ppc.pdf
> > + */
> > +#define REG_DWARFNUM_NAME(reg, idx) [idx] = "%" #reg
> > +
> > +static const char * const powerpc_regstr_tbl[] = {
> > + "%gpr0", "%gpr1", "%gpr2", "%gpr3", "%gpr4",
> > + "%gpr5", "%gpr6", "%gpr7", "%gpr8", "%gpr9",
> > + "%gpr10", "%gpr11", "%gpr12", "%gpr13", "%gpr14",
> > + "%gpr15", "%gpr16", "%gpr17", "%gpr18", "%gpr19",
> > + "%gpr20", "%gpr21", "%gpr22", "%gpr23", "%gpr24",
> > + "%gpr25", "%gpr26", "%gpr27", "%gpr28", "%gpr29",
> > + "%gpr30", "%gpr31",
> > + REG_DWARFNUM_NAME(msr, 66),
> > + REG_DWARFNUM_NAME(ctr, 109),
> > + REG_DWARFNUM_NAME(link, 108),
> > + REG_DWARFNUM_NAME(xer, 101),
> > + REG_DWARFNUM_NAME(dar, 119),
> > + REG_DWARFNUM_NAME(dsisr, 118),
> > +};
> > +
> > +#endif
> > diff --git a/tools/perf/arch/s390/include/dwarf-regs-table.h b/tools/perf/arch/s390/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..9da74a9
> > --- /dev/null
> > +++ b/tools/perf/arch/s390/include/dwarf-regs-table.h
> > @@ -0,0 +1,8 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +static const char * const s390_regstr_tbl[] = {
> > + "%r0", "%r1", "%r2", "%r3", "%r4", "%r5", "%r6", "%r7",
> > + "%r8", "%r9", "%r10", "%r11", "%r12", "%r13", "%r14", "%r15",
> > +};
> > +#endif
> > diff --git a/tools/perf/arch/sh/include/dwarf-regs-table.h b/tools/perf/arch/sh/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..3a2deaf
> > --- /dev/null
> > +++ b/tools/perf/arch/sh/include/dwarf-regs-table.h
> > @@ -0,0 +1,25 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +const char * const sh_regstr_tbl[] = {
> > + "r0",
> > + "r1",
> > + "r2",
> > + "r3",
> > + "r4",
> > + "r5",
> > + "r6",
> > + "r7",
> > + "r8",
> > + "r9",
> > + "r10",
> > + "r11",
> > + "r12",
> > + "r13",
> > + "r14",
> > + "r15",
> > + "pc",
> > + "pr",
> > +};
> > +
> > +#endif
> > diff --git a/tools/perf/arch/sparc/include/dwarf-regs-table.h b/tools/perf/arch/sparc/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..12c0761
> > --- /dev/null
> > +++ b/tools/perf/arch/sparc/include/dwarf-regs-table.h
> > @@ -0,0 +1,18 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +static const char * const sparc_regstr_tbl[] = {
> > + "%g0", "%g1", "%g2", "%g3", "%g4", "%g5", "%g6", "%g7",
> > + "%o0", "%o1", "%o2", "%o3", "%o4", "%o5", "%sp", "%o7",
> > + "%l0", "%l1", "%l2", "%l3", "%l4", "%l5", "%l6", "%l7",
> > + "%i0", "%i1", "%i2", "%i3", "%i4", "%i5", "%fp", "%i7",
> > + "%f0", "%f1", "%f2", "%f3", "%f4", "%f5", "%f6", "%f7",
> > + "%f8", "%f9", "%f10", "%f11", "%f12", "%f13", "%f14", "%f15",
> > + "%f16", "%f17", "%f18", "%f19", "%f20", "%f21", "%f22", "%f23",
> > + "%f24", "%f25", "%f26", "%f27", "%f28", "%f29", "%f30", "%f31",
> > + "%f32", "%f33", "%f34", "%f35", "%f36", "%f37", "%f38", "%f39",
> > + "%f40", "%f41", "%f42", "%f43", "%f44", "%f45", "%f46", "%f47",
> > + "%f48", "%f49", "%f50", "%f51", "%f52", "%f53", "%f54", "%f55",
> > + "%f56", "%f57", "%f58", "%f59", "%f60", "%f61", "%f62", "%f63",
> > +};
> > +#endif
> > diff --git a/tools/perf/arch/x86/include/dwarf-regs-table.h b/tools/perf/arch/x86/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..39ac7cb
> > --- /dev/null
> > +++ b/tools/perf/arch/x86/include/dwarf-regs-table.h
> > @@ -0,0 +1,14 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +static const char * const x86_32_regstr_tbl[] = {
> > + "%ax", "%cx", "%dx", "%bx", "$stack",/* Stack address instead of %sp */
> > + "%bp", "%si", "%di",
> > +};
> > +
> > +static const char * const x86_64_regstr_tbl[] = {
> > + "%ax", "dx", "%cx", "%bx", "%si", "%di",
> > + "%bp", "%sp", "%r8", "%r9", "%r10", "%r11",
> > + "%r12", "%r13", "%r14", "%r15",
> > +};
> > +#endif
> > diff --git a/tools/perf/arch/xtensa/include/dwarf-regs-table.h b/tools/perf/arch/xtensa/include/dwarf-regs-table.h
> > new file mode 100644
> > index 0000000..aa0444a
> > --- /dev/null
> > +++ b/tools/perf/arch/xtensa/include/dwarf-regs-table.h
> > @@ -0,0 +1,8 @@
> > +#ifdef DEFINE_DWARF_REGSTR_TABLE
> > +/* This is included in perf/util/dwarf-regs.c */
> > +
> > +static const char * const xtensa_regstr_tbl[] = {
> > + "a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7",
> > + "a8", "a9", "a10", "a11", "a12", "a13", "a14", "a15",
> > +};
> > +#endif
> > diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> > index 91c5f6e..f1a6d17 100644
> > --- a/tools/perf/util/Build
> > +++ b/tools/perf/util/Build
> > @@ -98,6 +98,7 @@ endif
> >
> > libperf-$(CONFIG_DWARF) += probe-finder.o
> > libperf-$(CONFIG_DWARF) += dwarf-aux.o
> > +libperf-$(CONFIG_DWARF) += dwarf-regs.o
> >
> > libperf-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
> > libperf-$(CONFIG_LOCAL_LIBUNWIND) += unwind-libunwind-local.o
> > diff --git a/tools/perf/util/dwarf-regs.c b/tools/perf/util/dwarf-regs.c
> > new file mode 100644
> > index 0000000..ce15816
> > --- /dev/null
> > +++ b/tools/perf/util/dwarf-regs.c
> > @@ -0,0 +1,55 @@
> > +/*
> > + * dwarf-regs.c : Mapping of DWARF debug register numbers into register names.
> > + *
> > + * Written by: Masami Hiramatsu <mhiramat@kernel.org>
> > + */
> > +
> > +#include <util.h>
> > +#include <debug.h>
> > +#include <dwarf-regs.h>
> > +#include <elf.h>
> > +
> > +/* Define const char * {arch}_register_tbl[] */
> > +#define DEFINE_DWARF_REGSTR_TABLE
> > +#include "../arch/x86/include/dwarf-regs-table.h"
> > +#include "../arch/arm/include/dwarf-regs-table.h"
> > +#include "../arch/arm64/include/dwarf-regs-table.h"
> > +#include "../arch/sh/include/dwarf-regs-table.h"
> > +#include "../arch/powerpc/include/dwarf-regs-table.h"
> > +#include "../arch/s390/include/dwarf-regs-table.h"
> > +#include "../arch/sparc/include/dwarf-regs-table.h"
> > +#include "../arch/xtensa/include/dwarf-regs-table.h"
> > +
> > +#define __get_dwarf_regstr(tbl, n) (((n) < ARRAY_SIZE(tbl)) ? (tbl)[(n)] : NULL)
> > +
> > +/* Return architecture dependent register string (for kprobe-tracer) */
> > +const char *get_dwarf_regstr(unsigned int n, unsigned int machine)
> > +{
> > + switch (machine) {
> > + case EM_NONE: /* Generic arch - use host arch */
> > + return get_arch_regstr(n);
> > + case EM_386:
> > + return __get_dwarf_regstr(x86_32_regstr_tbl, n);
> > + case EM_X86_64:
> > + return __get_dwarf_regstr(x86_64_regstr_tbl, n);
> > + case EM_ARM:
> > + return __get_dwarf_regstr(arm_regstr_tbl, n);
> > + case EM_AARCH64:
> > + return __get_dwarf_regstr(aarch64_regstr_tbl, n);
> > + case EM_SH:
> > + return __get_dwarf_regstr(sh_regstr_tbl, n);
> > + case EM_S390:
> > + return __get_dwarf_regstr(s390_regstr_tbl, n);
> > + case EM_PPC:
> > + case EM_PPC64:
> > + return __get_dwarf_regstr(powerpc_regstr_tbl, n);
> > + case EM_SPARC:
> > + case EM_SPARCV9:
> > + return __get_dwarf_regstr(sparc_regstr_tbl, n);
> > + case EM_XTENSA:
> > + return __get_dwarf_regstr(xtensa_regstr_tbl, n);
> > + default:
> > + pr_err("ELF MACHINE %x is not supported.\n", machine);
> > + }
> > + return NULL;
> > +}
> > diff --git a/tools/perf/util/include/dwarf-regs.h b/tools/perf/util/include/dwarf-regs.h
> > index 07c644e..43bfd8d 100644
> > --- a/tools/perf/util/include/dwarf-regs.h
> > +++ b/tools/perf/util/include/dwarf-regs.h
> > @@ -3,6 +3,12 @@
> >
> > #ifdef HAVE_DWARF_SUPPORT
> > const char *get_arch_regstr(unsigned int n);
> > +/*
> > + * get_dwarf_regstr - Returns ftrace register string from DWARF regnum
> > + * n: DWARF register number
> > + * machine: ELF machine signature (EM_*)
> > + */
> > +const char *get_dwarf_regstr(unsigned int n, unsigned int machine);
> > #endif
> >
> > #ifdef HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET
> > diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
> > index ac4740f..508b61c 100644
> > --- a/tools/perf/util/probe-finder.c
> > +++ b/tools/perf/util/probe-finder.c
> > @@ -171,6 +171,7 @@ static struct probe_trace_arg_ref *alloc_trace_arg_ref(long offs)
> > */
> > static int convert_variable_location(Dwarf_Die *vr_die, Dwarf_Addr addr,
> > Dwarf_Op *fb_ops, Dwarf_Die *sp_die,
> > + unsigned int machine,
> > struct probe_trace_arg *tvar)
> > {
> > Dwarf_Attribute attr;
> > @@ -266,7 +267,7 @@ static_var:
> > if (!tvar)
> > return ret2;
> >
> > - regs = get_arch_regstr(regn);
> > + regs = get_dwarf_regstr(regn, machine);
> > if (!regs) {
> > /* This should be a bug in DWARF or this tool */
> > pr_warning("Mapping for the register number %u "
> > @@ -543,7 +544,7 @@ static int convert_variable(Dwarf_Die *vr_die, struct probe_finder *pf)
> > dwarf_diename(vr_die));
> >
> > ret = convert_variable_location(vr_die, pf->addr, pf->fb_ops,
> > - &pf->sp_die, pf->tvar);
> > + &pf->sp_die, pf->machine, pf->tvar);
> > if (ret == -ENOENT || ret == -EINVAL) {
> > pr_err("Failed to find the location of the '%s' variable at this address.\n"
> > " Perhaps it has been optimized out.\n"
> > @@ -1106,11 +1107,8 @@ static int debuginfo__find_probes(struct debuginfo *dbg,
> > struct probe_finder *pf)
> > {
> > int ret = 0;
> > -
> > -#if _ELFUTILS_PREREQ(0, 142)
> > Elf *elf;
> > GElf_Ehdr ehdr;
> > - GElf_Shdr shdr;
> >
> > if (pf->cfi_eh || pf->cfi_dbg)
> > return debuginfo__find_probe_location(dbg, pf);
> > @@ -1123,11 +1121,18 @@ static int debuginfo__find_probes(struct debuginfo *dbg,
> > if (gelf_getehdr(elf, &ehdr) == NULL)
> > return -EINVAL;
> >
> > - if (elf_section_by_name(elf, &ehdr, &shdr, ".eh_frame", NULL) &&
> > - shdr.sh_type == SHT_PROGBITS)
> > - pf->cfi_eh = dwarf_getcfi_elf(elf);
> > + pf->machine = ehdr.e_machine;
> > +
> > +#if _ELFUTILS_PREREQ(0, 142)
> > + do {
> > + GElf_Shdr shdr;
> > +
> > + if (elf_section_by_name(elf, &ehdr, &shdr, ".eh_frame", NULL) &&
> > + shdr.sh_type == SHT_PROGBITS)
> > + pf->cfi_eh = dwarf_getcfi_elf(elf);
> >
> > - pf->cfi_dbg = dwarf_getcfi(dbg->dbg);
> > + pf->cfi_dbg = dwarf_getcfi(dbg->dbg);
> > + } while (0);
> > #endif
> >
> > ret = debuginfo__find_probe_location(dbg, pf);
> > @@ -1155,7 +1160,7 @@ static int copy_variables_cb(Dwarf_Die *die_mem, void *data)
> > (tag == DW_TAG_variable && vf->vars)) {
> > if (convert_variable_location(die_mem, vf->pf->addr,
> > vf->pf->fb_ops, &pf->sp_die,
> > - NULL) == 0) {
> > + pf->machine, NULL) == 0) {
> > vf->args[vf->nargs].var = (char *)dwarf_diename(die_mem);
> > if (vf->args[vf->nargs].var == NULL) {
> > vf->ret = -ENOMEM;
> > @@ -1318,7 +1323,7 @@ static int collect_variables_cb(Dwarf_Die *die_mem, void *data)
> > tag == DW_TAG_variable) {
> > ret = convert_variable_location(die_mem, af->pf.addr,
> > af->pf.fb_ops, &af->pf.sp_die,
> > - NULL);
> > + af->pf.machine, NULL);
> > if (ret == 0 || ret == -ERANGE) {
> > int ret2;
> > bool externs = !af->child;
> > diff --git a/tools/perf/util/probe-finder.h b/tools/perf/util/probe-finder.h
> > index 51137fc..f1d8558 100644
> > --- a/tools/perf/util/probe-finder.h
> > +++ b/tools/perf/util/probe-finder.h
> > @@ -80,6 +80,7 @@ struct probe_finder {
> > Dwarf_CFI *cfi_dbg;
> > #endif
> > Dwarf_Op *fb_ops; /* Frame base attribute */
> > + unsigned int machine; /* Target machine arch */
> > struct perf_probe_arg *pvar; /* Current target variable */
> > struct probe_trace_arg *tvar; /* Current result variable */
> > };

--
Masami Hiramatsu <mhiramat@kernel.org>