From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40840) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bX91E-0006EG-RS for qemu-devel@nongnu.org; Tue, 09 Aug 2016 11:32:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bX91A-0005Zq-6V for qemu-devel@nongnu.org; Tue, 09 Aug 2016 11:32:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39832) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bX919-0005Zg-Rb for qemu-devel@nongnu.org; Tue, 09 Aug 2016 11:32:36 -0400 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 64FB04E328 for ; Tue, 9 Aug 2016 15:32:35 +0000 (UTC) From: "Daniel P. Berrange" Date: Tue, 9 Aug 2016 16:31:28 +0100 Message-Id: <1470756748-18933-1-git-send-email-berrange@redhat.com> Subject: [Qemu-devel] [PATCH for-2.8 v1 00/60] Modular build of trace event files List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Stefan Hajnoczi , "Daniel P. Berrange" I previously split the global trace-events file up into one file per-subdirectory to avoid merge conflict hell. This series builds on that to now actually generate the individual trace files per-subdirectory too. The key benefit of doing this is that a change in a trace-events file will no longer cause a rebuild of the entire world. Instead only that one affected sub-directory will have files rebuilt. The key problem faced when creating this patch series was the fact that the trace code had a global assumption that there was a single enum declaring all events in a sequence with no gaps. Most of the hard work in this series involves getting rid of that assumption throughout the code. Instead we now have a notion of trace event groups, each with their own distinct enum of events. As such the individual event IDs are no longer globally unique. Most of the code could be adapted to work with IDs that were unique only within a group. The first sticky part was the simpletrace binary event format which includes the event ID. To deal with this I invented a notion of an ID offset per trace-events file. So simpletrace format will now use "(id-offset << 16) | event-id" in order to get a globally unique ID value for its trace files. This means each trace-events file can only have 65536 distinct events declared, but that ought to be enough for a while :-) The second sticky part was the per-VCPU events, due to the use of a "trace_dstate" bitmap in the CPUState struct. The bitmap inherantly assumes a global ID sequence, and would not work well with the per-file id-offset concept, since it would require a huge mostly-sparse bitmap. Rather than try to solve this problem I took the easy way out and simply specified that all per-VCPU events need to be declared in the top level trace-events file. Since we have almost no per-vCPU events today, this works out just fine. I've build-tested with all targets enabled in both VPATH and non-VPATH setups, on Fedora 24, with all 5 trace event backends enabled at once. I've tested that simpletrace.py can still extract events from generated traces. I've not functionally tested dtrace/ftrace/ust at this time. Daniel P. Berrange (60): trace: move util/buffer.c trace points into correct file trace: move util/qemu-coroutine*.c trace points into correct file trace: move hw/mem/pc-dimm.c trace points into correct file trace: move hw/virtio/virtio-balloon.c trace points into correct file trace: add trace event iterator APIs trace: convert code to use event iterators trace: remove some now unused functions trace: provide mechanism for registering trace events trace: remove use of TRACE_VCPU_EVENT_COUNT in cpu.h trace: remove fixed global event state arrays trace: remove use of event ID enums from APIs trace: use -1 instead of TRACE_VCPU_EVENT_COUNT as magic value trace: remove generated-events.h from many includes trace: get rid of generated-events.h/generated-events.c trace: introduce a formal group name for trace events trace: introduce ID range offsets per trace-events file trace: introduce some Makefile rules for module code gen trace: switch io/ directory to modular trace.h file trace: switch util/ directory to modular trace.h file trace: switch crypto/ directory to modular trace.h file trace: switch migration/ directory to modular trace.h file trace: switch block/ directory to modular trace.h file trace: switch hw/block/ directory to modular trace.h file trace: switch hw/char/ directory to modular trace.h file trace: switch hw/intc/ directory to modular trace.h file trace: switch hw/net/ directory to modular trace.h file trace: switch hw/virtio/ directory to modular trace.h file trace: switch hw/audio/ directory to modular trace.h file trace: switch hw/misc/ directory to modular trace.h file trace: switch hw/usb/ directory to modular trace.h file trace: switch hw/scsi/ directory to modular trace.h file trace: switch hw/nvram/ directory to modular trace.h file trace: switch hw/display/ directory to modular trace.h file trace: switch hw/input/ directory to modular trace.h file trace: switch hw/timer/ directory to modular trace.h file trace: switch hw/dma/ directory to modular trace.h file trace: switch hw/sparc/ directory to modular trace.h file trace: switch hw/sd/ directory to modular trace.h file trace: switch hw/isa/ directory to modular trace.h file trace: switch hw/mem/ directory to modular trace.h file trace: switch hw/i386/ directory to modular trace.h file trace: switch hw/9pfs/ directory to modular trace.h file trace: switch hw/ppc/ directory to modular trace.h file trace: switch hw/pci/ directory to modular trace.h file trace: switch hw/s390x/ directory to modular trace.h file trace: switch hw/vfio/ directory to modular trace.h file trace: switch hw/acpi/ directory to modular trace.h file trace: switch hw/arm/ directory to modular trace.h file trace: switch hw/alpha/ directory to modular trace.h file trace: switch ui/ directory to modular trace.h file trace: switch audio/ directory to modular trace.h file trace: switch net/ directory to modular trace.h file trace: switch target-i386/ directory to modular trace.h file trace: switch target-sparc/ directory to modular trace.h file trace: switch target-s390x/ directory to modular trace.h file trace: switch target-ppc/ directory to modular trace.h file trace: switch qom/ directory to modular trace.h file trace: switch linux-user/ directory to modular trace.h file trace: remove the global include/trace.h file trace: update docs to reflect new code generation approach .gitignore | 15 +- Makefile | 116 +++++++++++--- Makefile.objs | 89 ++++++----- Makefile.target | 7 +- audio/Makefile.objs | 2 + audio/alsaaudio.c | 2 +- audio/ossaudio.c | 2 +- audio/trace-events | 2 + block.c | 2 +- block/Makefile.objs | 2 + block/backup.c | 2 +- block/block-backend.c | 2 +- block/commit.c | 2 +- block/dirty-bitmap.c | 2 +- block/io.c | 2 +- block/mirror.c | 2 +- block/nfs.c | 2 +- block/qcow2-cache.c | 2 +- block/qcow2-cluster.c | 2 +- block/qcow2.c | 2 +- block/qed-l2-cache.c | 2 +- block/qed-table.c | 2 +- block/qed.c | 2 +- block/raw-posix.c | 2 +- block/raw-win32.c | 2 +- block/stream.c | 2 +- block/trace-events | 2 + blockdev.c | 2 +- crypto/Makefile.objs | 2 + crypto/secret.c | 2 +- crypto/tlscreds.c | 2 +- crypto/tlscredsanon.c | 2 +- crypto/tlscredsx509.c | 2 +- crypto/tlssession.c | 2 +- crypto/trace-events | 2 + docs/tracing.txt | 61 ++++++-- hw/9pfs/9p.c | 2 +- hw/9pfs/Makefile.objs | 2 + hw/9pfs/trace-events | 2 + hw/acpi/Makefile.objs | 2 + hw/acpi/cpu.c | 2 +- hw/acpi/memory_hotplug.c | 2 +- hw/acpi/trace-events | 2 + hw/alpha/Makefile.objs | 2 + hw/alpha/pci.c | 2 +- hw/alpha/trace-events | 2 + hw/arm/Makefile.objs | 2 + hw/arm/trace-events | 2 + hw/arm/virt-acpi-build.c | 2 +- hw/audio/Makefile.objs | 3 + hw/audio/cs4231.c | 2 +- hw/audio/milkymist-ac97.c | 2 +- hw/audio/trace-events | 2 + hw/block/Makefile.objs | 2 + hw/block/dataplane/virtio-blk.c | 2 +- hw/block/hd-geometry.c | 2 +- hw/block/trace-events | 2 + hw/block/virtio-blk.c | 2 +- hw/char/Makefile.objs | 2 + hw/char/escc.c | 2 +- hw/char/grlib_apbuart.c | 2 +- hw/char/lm32_juart.c | 2 +- hw/char/lm32_uart.c | 2 +- hw/char/milkymist-uart.c | 2 +- hw/char/trace-events | 2 + hw/char/virtio-console.c | 2 +- hw/char/virtio-serial-bus.c | 2 +- hw/display/Makefile.objs | 3 + hw/display/g364fb.c | 2 +- hw/display/jazz_led.c | 2 +- hw/display/milkymist-tmu2.c | 2 +- hw/display/milkymist-vgafb.c | 2 +- hw/display/qxl-render.c | 2 +- hw/display/qxl.c | 2 +- hw/display/trace-events | 2 + hw/display/vga.c | 2 +- hw/display/virtio-gpu-3d.c | 2 +- hw/display/virtio-gpu.c | 2 +- hw/display/vmware_vga.c | 2 +- hw/display/xenfb.c | 2 +- hw/dma/Makefile.objs | 2 + hw/dma/i8257.c | 2 +- hw/dma/rc4030.c | 2 +- hw/dma/sparc32_dma.c | 2 +- hw/dma/sun4m_iommu.c | 2 +- hw/dma/trace-events | 2 + hw/i386/Makefile.objs | 2 + hw/i386/trace-events | 6 +- hw/i386/x86-iommu.c | 2 +- hw/i386/xen/xen_platform.c | 1 + hw/i386/xen/xen_pvdevice.c | 1 + hw/input/Makefile.objs | 3 + hw/input/hid.c | 2 +- hw/input/milkymist-softusb.c | 2 +- hw/input/ps2.c | 2 +- hw/input/trace-events | 2 + hw/input/virtio-input.c | 2 +- hw/intc/Makefile.objs | 2 + hw/intc/apic.c | 2 +- hw/intc/apic_common.c | 2 +- hw/intc/arm_gic.c | 2 +- hw/intc/arm_gicv3_cpuif.c | 2 +- hw/intc/arm_gicv3_dist.c | 2 +- hw/intc/arm_gicv3_redist.c | 2 +- hw/intc/aspeed_vic.c | 2 +- hw/intc/grlib_irqmp.c | 2 +- hw/intc/lm32_pic.c | 2 +- hw/intc/s390_flic.c | 2 +- hw/intc/s390_flic_kvm.c | 2 +- hw/intc/slavio_intctl.c | 2 +- hw/intc/trace-events | 2 + hw/intc/xics.c | 2 +- hw/intc/xics_kvm.c | 2 +- hw/intc/xics_spapr.c | 2 +- hw/isa/Makefile.objs | 2 + hw/isa/pc87312.c | 2 +- hw/isa/trace-events | 2 + hw/mem/Makefile.objs | 3 + hw/mem/pc-dimm.c | 2 +- hw/mem/trace-events | 7 + hw/misc/Makefile.objs | 3 + hw/misc/aspeed_scu.c | 2 +- hw/misc/eccmemctl.c | 2 +- hw/misc/milkymist-hpdmc.c | 2 +- hw/misc/milkymist-pfpu.c | 2 +- hw/misc/slavio_misc.c | 2 +- hw/misc/trace-events | 2 + hw/net/Makefile.objs | 3 + hw/net/e1000e.c | 2 +- hw/net/e1000e_core.c | 2 +- hw/net/e1000x_common.c | 2 +- hw/net/lance.c | 2 +- hw/net/milkymist-minimac2.c | 2 +- hw/net/mipsnet.c | 2 +- hw/net/net_rx_pkt.c | 2 +- hw/net/opencores_eth.c | 2 +- hw/net/pcnet-pci.c | 2 +- hw/net/pcnet.c | 2 +- hw/net/trace-events | 2 + hw/nvram/Makefile.objs | 2 + hw/nvram/ds1225y.c | 2 +- hw/nvram/fw_cfg.c | 2 +- hw/nvram/trace-events | 2 + hw/pci/Makefile.objs | 2 + hw/pci/pci.c | 2 +- hw/pci/pci_host.c | 2 +- hw/pci/trace-events | 2 + hw/ppc/Makefile.objs | 3 + hw/ppc/ppc.c | 2 +- hw/ppc/prep.c | 2 +- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_hcall.c | 2 +- hw/ppc/spapr_iommu.c | 2 +- hw/ppc/spapr_pci.c | 2 +- hw/ppc/spapr_rtas_ddw.c | 2 +- hw/ppc/trace-events | 2 + hw/s390x/Makefile.objs | 3 + hw/s390x/css.c | 2 +- hw/s390x/trace-events | 2 + hw/s390x/virtio-ccw.c | 2 +- hw/scsi/Makefile.objs | 3 + hw/scsi/esp-pci.c | 2 +- hw/scsi/esp.c | 2 +- hw/scsi/megasas.c | 2 +- hw/scsi/mptconfig.c | 2 +- hw/scsi/mptendian.c | 2 +- hw/scsi/mptsas.c | 2 +- hw/scsi/scsi-bus.c | 2 +- hw/scsi/trace-events | 2 + hw/scsi/vmw_pvscsi.c | 2 +- hw/sd/Makefile.objs | 2 + hw/sd/milkymist-memcard.c | 2 +- hw/sd/trace-events | 2 + hw/sparc/Makefile.objs | 2 + hw/sparc/leon3.c | 2 +- hw/sparc/sun4m.c | 2 +- hw/sparc/trace-events | 2 + hw/timer/Makefile.objs | 2 + hw/timer/aspeed_timer.c | 2 +- hw/timer/grlib_gptimer.c | 2 +- hw/timer/lm32_timer.c | 2 +- hw/timer/milkymist-sysctl.c | 2 +- hw/timer/slavio_timer.c | 2 +- hw/timer/trace-events | 2 + hw/usb/Makefile.objs | 3 + hw/usb/bus.c | 2 +- hw/usb/combined-packet.c | 2 +- hw/usb/core.c | 2 +- hw/usb/desc.c | 2 +- hw/usb/dev-hub.c | 2 +- hw/usb/dev-mtp.c | 2 +- hw/usb/dev-uas.c | 2 +- hw/usb/hcd-ehci.c | 2 +- hw/usb/hcd-ohci.c | 8 +- hw/usb/hcd-uhci.c | 2 +- hw/usb/hcd-xhci.c | 2 +- hw/usb/host-libusb.c | 2 +- hw/usb/trace-events | 2 + hw/vfio/Makefile.objs | 2 + hw/vfio/common.c | 2 +- hw/vfio/pci-quirks.c | 2 +- hw/vfio/pci.c | 2 +- hw/vfio/platform.c | 2 +- hw/vfio/spapr.c | 2 +- hw/vfio/trace-events | 2 + hw/virtio/Makefile.objs | 2 + hw/virtio/trace-events | 7 + hw/virtio/virtio-balloon.c | 2 +- hw/virtio/virtio-rng.c | 2 +- hw/virtio/virtio.c | 2 +- include/qemu/module.h | 2 + include/qom/cpu.h | 10 +- include/trace-tcg.h | 1 - include/trace.h | 7 - io/Makefile.objs | 2 + io/channel-buffer.c | 2 +- io/channel-command.c | 2 +- io/channel-file.c | 2 +- io/channel-socket.c | 2 +- io/channel-tls.c | 2 +- io/channel-websock.c | 2 +- io/task.c | 2 +- io/trace-events | 6 +- linux-user/Makefile.objs | 2 + linux-user/signal.c | 2 +- linux-user/trace-events | 2 + migration/Makefile.objs | 2 + migration/exec.c | 2 +- migration/fd.c | 2 +- migration/migration.c | 2 +- migration/postcopy-ram.c | 2 +- migration/qemu-file.c | 2 +- migration/ram.c | 2 +- migration/rdma.c | 2 +- migration/savevm.c | 2 +- migration/socket.c | 2 +- migration/tls.c | 2 +- migration/trace-events | 2 + migration/vmstate.c | 2 +- monitor.c | 16 +- net/Makefile.objs | 2 + net/filter-mirror.c | 2 +- net/trace-events | 2 + net/vhost-user.c | 2 +- qemu-img.c | 1 + qemu-io.c | 1 + qemu-nbd.c | 1 + qom/Makefile.objs | 2 + qom/cpu.c | 2 +- qom/object.c | 2 +- qom/trace-events | 2 + scripts/simpletrace.py | 7 +- scripts/tracetool.py | 10 +- scripts/tracetool/__init__.py | 59 +++++-- scripts/tracetool/backend/__init__.py | 12 +- scripts/tracetool/backend/dtrace.py | 6 +- scripts/tracetool/backend/ftrace.py | 7 +- scripts/tracetool/backend/log.py | 6 +- scripts/tracetool/backend/simple.py | 13 +- scripts/tracetool/backend/ust.py | 6 +- scripts/tracetool/format/__init__.py | 4 +- scripts/tracetool/format/c.py | 48 +++++- scripts/tracetool/format/d.py | 2 +- scripts/tracetool/format/events_c.py | 44 ------ scripts/tracetool/format/events_h.py | 60 ------- scripts/tracetool/format/h.py | 51 +++++- scripts/tracetool/format/simpletrace_stap.py | 4 +- scripts/tracetool/format/stap.py | 2 +- scripts/tracetool/format/tcg_h.py | 2 +- scripts/tracetool/format/tcg_helper_c.py | 2 +- scripts/tracetool/format/tcg_helper_h.py | 2 +- scripts/tracetool/format/tcg_helper_wrapper_h.py | 2 +- scripts/tracetool/format/ust_events_c.py | 4 +- scripts/tracetool/format/ust_events_h.py | 15 +- stubs/trace-control.c | 8 +- target-i386/Makefile.objs | 2 + target-i386/kvm.c | 2 +- target-i386/trace-events | 2 + target-ppc/Makefile.objs | 2 + target-ppc/kvm.c | 2 +- target-ppc/trace-events | 2 + target-s390x/Makefile.objs | 2 + target-s390x/cpu.c | 2 +- target-s390x/ioinst.c | 2 +- target-s390x/kvm.c | 2 +- target-s390x/mmu_helper.c | 2 +- target-s390x/trace-events | 2 + target-sparc/Makefile.objs | 2 + target-sparc/int32_helper.c | 2 +- target-sparc/int64_helper.c | 2 +- target-sparc/mmu_helper.c | 2 +- target-sparc/trace-events | 2 + target-sparc/win_helper.c | 2 +- trace-events | 19 +-- trace/Makefile.objs | 98 ++---------- trace/control-internal.h | 39 ++--- trace/control-target.c | 20 +-- trace/control.c | 191 +++++++++++++++++------ trace/control.h | 108 +++---------- trace/event-internal.h | 7 +- trace/qmp.c | 29 ++-- trace/simple.c | 6 +- trace/simple.h | 6 +- ui/Makefile.objs | 3 + ui/console.c | 2 +- ui/gtk-egl.c | 2 +- ui/gtk-gl-area.c | 2 +- ui/gtk.c | 2 +- ui/input.c | 2 +- ui/spice-display.c | 2 +- ui/trace-events | 2 + ui/vnc.c | 2 +- util/Makefile.objs | 2 + util/buffer.c | 2 +- util/hbitmap.c | 2 +- util/oslib-posix.c | 2 +- util/oslib-win32.c | 2 +- util/qemu-coroutine-lock.c | 2 +- util/qemu-coroutine.c | 2 +- util/trace-events | 21 +++ vl.c | 2 + 321 files changed, 1058 insertions(+), 768 deletions(-) create mode 100644 hw/mem/trace-events delete mode 100644 include/trace.h delete mode 100644 scripts/tracetool/format/events_c.py delete mode 100644 scripts/tracetool/format/events_h.py -- 2.7.4