* [PULL 01/21] qmp: Support for querying stats
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 02/21] kvm: Support for querying fd-based stats Paolo Bonzini
` (20 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Mark Kanda, Markus Armbruster
From: Mark Kanda <mark.kanda@oracle.com>
Gathering statistics is important for development, for monitoring and
for performance measurement. There are tools such as kvm_stat that do
this and they rely on the _user_ knowing the interesting data points
rather than the tool (which can treat them as opaque).
The commands introduced in this commit introduce QMP support for
querying stats; the goal is to take the capabilities of these tools
and making them available throughout the whole virtualization stack,
so that one can observe, monitor and measure virtual machines without
having shell access + root on the host that runs them.
query-stats returns a list of all stats per target type (only VM
and vCPU to start); future commits add extra options for specifying
stat names, vCPU qom paths, and providers. All these are used by the
HMP command "info stats". Because of the development usecases around
statistics, a good HMP interface is important.
query-stats-schemas returns a list of stats included in each target
type, with an option for specifying the provider. The concepts in the
schema are based on the KVM binary stats' own introspection data, just
translated to QAPI.
There are two reasons to have a separate schema that is not tied to
the QAPI schema. The first is the contents of the schemas: the new
introspection data provides different information than the QAPI data,
namely unit of measurement, how the numbers are gathered and change
(peak/instant/cumulative/histogram), and histogram bucket sizes.
There's really no reason to have this kind of metadata in the QAPI
introspection schema (except possibly for the unit of measure, but
there's a very weak justification).
Another reason is the dynamicity of the schema. The QAPI introspection
data is very much static; and while QOM is somewhat more dynamic,
generally we consider that to be a bug rather than a feature these days.
On the other hand, the statistics that are exposed by QEMU might be
passed through from another source, such as KVM, and the disadvantages of
manually updating the QAPI schema for outweight the benefits from vetting
the statistics and filtering out anything that seems "too unstable".
Running old QEMU with new kernel is a supported usecase; if old QEMU
cannot expose statistics from a new kernel, or if a kernel developer
needs to change QEMU before gathering new info from the new kernel,
then that is a poor user interface.
The framework provides a method to register callbacks for these QMP
commands. Most of the work in fact is done by the callbacks, and a
large majority of this patch is new QAPI structs and commands.
Examples (with KVM stats):
- Query all VM stats:
{ "execute": "query-stats", "arguments" : { "target": "vm" } }
{ "return": [
{ "provider": "kvm",
"stats": [
{ "name": "max_mmu_page_hash_collisions", "value": 0 },
{ "name": "max_mmu_rmap_size", "value": 0 },
{ "name": "nx_lpage_splits", "value": 148 },
... ] },
{ "provider": "xyz",
"stats": [ ... ] }
] }
- Query all vCPU stats:
{ "execute": "query-stats", "arguments" : { "target": "vcpu" } }
{ "return": [
{ "provider": "kvm",
"qom_path": "/machine/unattached/device[0]"
"stats": [
{ "name": "guest_mode", "value": 0 },
{ "name": "directed_yield_successful", "value": 0 },
{ "name": "directed_yield_attempted", "value": 106 },
... ] },
{ "provider": "kvm",
"qom_path": "/machine/unattached/device[1]"
"stats": [
{ "name": "guest_mode", "value": 0 },
{ "name": "directed_yield_successful", "value": 0 },
{ "name": "directed_yield_attempted", "value": 106 },
... ] },
] }
- Retrieve the schemas:
{ "execute": "query-stats-schemas" }
{ "return": [
{ "provider": "kvm",
"target": "vcpu",
"stats": [
{ "name": "guest_mode",
"unit": "none",
"base": 10,
"exponent": 0,
"type": "instant" },
{ "name": "directed_yield_successful",
"unit": "none",
"base": 10,
"exponent": 0,
"type": "cumulative" },
... ]
},
{ "provider": "kvm",
"target": "vm",
"stats": [
{ "name": "max_mmu_page_hash_collisions",
"unit": "none",
"base": 10,
"exponent": 0,
"type": "peak" },
... ]
},
{ "provider": "xyz",
"target": "vm",
"stats": [ ... ]
}
] }
Signed-off-by: Mark Kanda <mark.kanda@oracle.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
include/monitor/stats.h | 34 +++++++
monitor/qmp-cmds.c | 95 ++++++++++++++++++
qapi/meson.build | 1 +
qapi/qapi-schema.json | 1 +
qapi/stats.json | 216 ++++++++++++++++++++++++++++++++++++++++
5 files changed, 347 insertions(+)
create mode 100644 include/monitor/stats.h
create mode 100644 qapi/stats.json
diff --git a/include/monitor/stats.h b/include/monitor/stats.h
new file mode 100644
index 0000000000..912eeadb2f
--- /dev/null
+++ b/include/monitor/stats.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (c) 2022 Oracle and/or its affiliates.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef STATS_H
+#define STATS_H
+
+#include "qapi/qapi-types-stats.h"
+
+typedef void StatRetrieveFunc(StatsResultList **result, StatsTarget target,
+ Error **errp);
+typedef void SchemaRetrieveFunc(StatsSchemaList **result, Error **errp);
+
+/*
+ * Register callbacks for the QMP query-stats command.
+ *
+ * @stats_fn: routine to query stats:
+ * @schema_fn: routine to query stat schemas:
+ */
+void add_stats_callbacks(StatRetrieveFunc *stats_fn,
+ SchemaRetrieveFunc *schemas_fn);
+
+/*
+ * Helper routines for adding stats entries to the results lists.
+ */
+void add_stats_entry(StatsResultList **, StatsProvider, const char *id,
+ StatsList *stats_list);
+void add_stats_schema(StatsSchemaList **, StatsProvider, StatsTarget,
+ StatsSchemaValueList *);
+
+#endif /* STATS_H */
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 1ebb89f46c..a6ac8d7473 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -35,6 +35,7 @@
#include "qapi/qapi-commands-control.h"
#include "qapi/qapi-commands-machine.h"
#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-commands-stats.h"
#include "qapi/qapi-commands-ui.h"
#include "qapi/type-helpers.h"
#include "qapi/qmp/qerror.h"
@@ -43,6 +44,7 @@
#include "hw/acpi/acpi_dev_interface.h"
#include "hw/intc/intc.h"
#include "hw/rdma/rdma.h"
+#include "monitor/stats.h"
NameInfo *qmp_query_name(Error **errp)
{
@@ -441,3 +443,96 @@ HumanReadableText *qmp_x_query_irq(Error **errp)
return human_readable_text_from_str(buf);
}
+
+typedef struct StatsCallbacks {
+ StatRetrieveFunc *stats_cb;
+ SchemaRetrieveFunc *schemas_cb;
+ QTAILQ_ENTRY(StatsCallbacks) next;
+} StatsCallbacks;
+
+static QTAILQ_HEAD(, StatsCallbacks) stats_callbacks =
+ QTAILQ_HEAD_INITIALIZER(stats_callbacks);
+
+void add_stats_callbacks(StatRetrieveFunc *stats_fn,
+ SchemaRetrieveFunc *schemas_fn)
+{
+ StatsCallbacks *entry = g_new(StatsCallbacks, 1);
+ entry->stats_cb = stats_fn;
+ entry->schemas_cb = schemas_fn;
+
+ QTAILQ_INSERT_TAIL(&stats_callbacks, entry, next);
+}
+
+static bool invoke_stats_cb(StatsCallbacks *entry,
+ StatsResultList **stats_results,
+ StatsFilter *filter,
+ Error **errp)
+{
+ ERRP_GUARD();
+
+ entry->stats_cb(stats_results, filter->target, errp);
+ if (*errp) {
+ qapi_free_StatsResultList(*stats_results);
+ *stats_results = NULL;
+ return false;
+ }
+ return true;
+}
+
+StatsResultList *qmp_query_stats(StatsFilter *filter, Error **errp)
+{
+ StatsResultList *stats_results = NULL;
+ StatsCallbacks *entry;
+
+ QTAILQ_FOREACH(entry, &stats_callbacks, next) {
+ if (!invoke_stats_cb(entry, &stats_results, filter, errp)) {
+ break;
+ }
+ }
+
+ return stats_results;
+}
+
+StatsSchemaList *qmp_query_stats_schemas(Error **errp)
+{
+ StatsSchemaList *stats_results = NULL;
+ StatsCallbacks *entry;
+ ERRP_GUARD();
+
+ QTAILQ_FOREACH(entry, &stats_callbacks, next) {
+ entry->schemas_cb(&stats_results, errp);
+ if (*errp) {
+ qapi_free_StatsSchemaList(stats_results);
+ return NULL;
+ }
+ }
+
+ return stats_results;
+}
+
+void add_stats_entry(StatsResultList **stats_results, StatsProvider provider,
+ const char *qom_path, StatsList *stats_list)
+{
+ StatsResult *entry = g_new0(StatsResult, 1);
+
+ entry->provider = provider;
+ if (qom_path) {
+ entry->has_qom_path = true;
+ entry->qom_path = g_strdup(qom_path);
+ }
+ entry->stats = stats_list;
+
+ QAPI_LIST_PREPEND(*stats_results, entry);
+}
+
+void add_stats_schema(StatsSchemaList **schema_results,
+ StatsProvider provider, StatsTarget target,
+ StatsSchemaValueList *stats_list)
+{
+ StatsSchema *entry = g_new0(StatsSchema, 1);
+
+ entry->provider = provider;
+ entry->target = target;
+ entry->stats = stats_list;
+ QAPI_LIST_PREPEND(*schema_results, entry);
+}
diff --git a/qapi/meson.build b/qapi/meson.build
index 656ef0e039..fd5c93d643 100644
--- a/qapi/meson.build
+++ b/qapi/meson.build
@@ -46,6 +46,7 @@ qapi_all_modules = [
'replay',
'run-state',
'sockets',
+ 'stats',
'trace',
'transaction',
'yank',
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 4912b9744e..92d7ecc52c 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -93,3 +93,4 @@
{ 'include': 'audio.json' }
{ 'include': 'acpi.json' }
{ 'include': 'pci.json' }
+{ 'include': 'stats.json' }
diff --git a/qapi/stats.json b/qapi/stats.json
new file mode 100644
index 0000000000..ada0fbf26f
--- /dev/null
+++ b/qapi/stats.json
@@ -0,0 +1,216 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# Copyright (c) 2022 Oracle and/or its affiliates.
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+##
+# = Statistics
+##
+
+##
+# @StatsType:
+#
+# Enumeration of statistics types
+#
+# @cumulative: stat is cumulative; value can only increase.
+# @instant: stat is instantaneous; value can increase or decrease.
+# @peak: stat is the peak value; value can only increase.
+# @linear-histogram: stat is a linear histogram.
+# @log2-histogram: stat is a logarithmic histogram, with one bucket
+# for each power of two.
+#
+# Since: 7.1
+##
+{ 'enum' : 'StatsType',
+ 'data' : [ 'cumulative', 'instant', 'peak', 'linear-histogram',
+ 'log2-histogram' ] }
+
+##
+# @StatsUnit:
+#
+# Enumeration of unit of measurement for statistics
+#
+# @bytes: stat reported in bytes.
+# @seconds: stat reported in seconds.
+# @cycles: stat reported in clock cycles.
+#
+# Since: 7.1
+##
+{ 'enum' : 'StatsUnit',
+ 'data' : [ 'bytes', 'seconds', 'cycles' ] }
+
+##
+# @StatsProvider:
+#
+# Enumeration of statistics providers.
+#
+# Since: 7.1
+##
+{ 'enum': 'StatsProvider',
+ 'data': [ ] }
+
+##
+# @StatsTarget:
+#
+# The kinds of objects on which one can request statistics.
+#
+# @vm: statistics that apply to the entire virtual machine or
+# the entire QEMU process.
+#
+# @vcpu: statistics that apply to a single virtual CPU.
+#
+# Since: 7.1
+##
+{ 'enum': 'StatsTarget',
+ 'data': [ 'vm', 'vcpu' ] }
+
+##
+# @StatsFilter:
+#
+# The arguments to the query-stats command; specifies a target for which to
+# request statistics.
+#
+# Since: 7.1
+##
+{ 'struct': 'StatsFilter',
+ 'data': { 'target': 'StatsTarget' } }
+
+##
+# @StatsValue:
+#
+# @scalar: single unsigned 64-bit integers.
+# @list: list of unsigned 64-bit integers (used for histograms).
+#
+# Since: 7.1
+##
+{ 'alternate': 'StatsValue',
+ 'data': { 'scalar': 'uint64',
+ 'list': [ 'uint64' ] } }
+
+##
+# @Stats:
+#
+# @name: name of stat.
+# @value: stat value.
+#
+# Since: 7.1
+##
+{ 'struct': 'Stats',
+ 'data': { 'name': 'str',
+ 'value' : 'StatsValue' } }
+
+##
+# @StatsResult:
+#
+# @provider: provider for this set of statistics.
+#
+# @qom-path: Path to the object for which the statistics are returned,
+# if the object is exposed in the QOM tree
+#
+# @stats: list of statistics.
+#
+# Since: 7.1
+##
+{ 'struct': 'StatsResult',
+ 'data': { 'provider': 'StatsProvider',
+ '*qom-path': 'str',
+ 'stats': [ 'Stats' ] } }
+
+##
+# @query-stats:
+#
+# Return runtime-collected statistics for objects such as the
+# VM or its vCPUs.
+#
+# The arguments are a StatsFilter and specify the provider and objects
+# to return statistics about.
+#
+# Returns: a list of StatsResult, one for each provider and object
+# (e.g., for each vCPU).
+#
+# Since: 7.1
+##
+{ 'command': 'query-stats',
+ 'data': 'StatsFilter',
+ 'boxed': true,
+ 'returns': [ 'StatsResult' ] }
+
+##
+# @StatsSchemaValue:
+#
+# Schema for a single statistic.
+#
+# @name: name of the statistic; each element of the schema is uniquely
+# identified by a target, a provider (both available in @StatsSchema)
+# and the name.
+#
+# @type: kind of statistic.
+#
+# @unit: basic unit of measure for the statistic; if missing, the statistic
+# is a simple number or counter.
+#
+# @base: base for the multiple of @unit in which the statistic is measured.
+# Only present if @exponent is non-zero; @base and @exponent together
+# form a SI prefix (e.g., _nano-_ for ``base=10`` and ``exponent=-9``)
+# or IEC binary prefix (e.g. _kibi-_ for ``base=2`` and ``exponent=10``)
+#
+# @exponent: exponent for the multiple of @unit in which the statistic is
+# expressed, or 0 for the basic unit
+#
+# @bucket-size: Present when @type is "linear-histogram", contains the width
+# of each bucket of the histogram.
+#
+# Since: 7.1
+##
+{ 'struct': 'StatsSchemaValue',
+ 'data': { 'name': 'str',
+ 'type': 'StatsType',
+ '*unit': 'StatsUnit',
+ '*base': 'int8',
+ 'exponent': 'int16',
+ '*bucket-size': 'uint32' } }
+
+##
+# @StatsSchema:
+#
+# Schema for all available statistics for a provider and target.
+#
+# @provider: provider for this set of statistics.
+#
+# @target: the kind of object that can be queried through the provider.
+#
+# @stats: list of statistics.
+#
+# Since: 7.1
+##
+{ 'struct': 'StatsSchema',
+ 'data': { 'provider': 'StatsProvider',
+ 'target': 'StatsTarget',
+ 'stats': [ 'StatsSchemaValue' ] } }
+
+##
+# @query-stats-schemas:
+#
+# Return the schema for all available runtime-collected statistics.
+#
+# Note: runtime-collected statistics and their names fall outside QEMU's usual
+# deprecation policies. QEMU will try to keep the set of available deport 0. Such
+# changes, however, are expected to be rare.
+#
+# Since: 7.1
+##
+{ 'command': 'query-stats-schemas',
+ 'data': { },
+ 'returns': [ 'StatsSchema' ] }
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 02/21] kvm: Support for querying fd-based stats
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
2022-06-16 8:31 ` [PULL 01/21] qmp: Support for querying stats Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 03/21] qmp: add filtering of statistics by target vCPU Paolo Bonzini
` (19 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Mark Kanda
From: Mark Kanda <mark.kanda@oracle.com>
Add support for querying fd-based KVM stats - as introduced by Linux kernel
commit:
cb082bfab59a ("KVM: stats: Add fd-based API to read binary stats data")
This allows the user to analyze the behavior of the VM without access
to debugfs.
Signed-off-by: Mark Kanda <mark.kanda@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 392 ++++++++++++++++++++++++++++++++++++++++++++
qapi/stats.json | 2 +-
2 files changed, 393 insertions(+), 1 deletion(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index a4c4863f53..7cc9e33bab 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -47,6 +47,7 @@
#include "kvm-cpus.h"
#include "hw/boards.h"
+#include "monitor/stats.h"
/* This check must be after config-host.h is included */
#ifdef CONFIG_EVENTFD
@@ -2310,6 +2311,9 @@ bool kvm_dirty_ring_enabled(void)
return kvm_state->kvm_dirty_ring_size ? true : false;
}
+static void query_stats_cb(StatsResultList **result, StatsTarget target, Error **errp);
+static void query_stats_schemas_cb(StatsSchemaList **result, Error **errp);
+
static int kvm_init(MachineState *ms)
{
MachineClass *mc = MACHINE_GET_CLASS(ms);
@@ -2638,6 +2642,10 @@ static int kvm_init(MachineState *ms)
}
}
+ if (kvm_check_extension(kvm_state, KVM_CAP_BINARY_STATS_FD)) {
+ add_stats_callbacks(query_stats_cb, query_stats_schemas_cb);
+ }
+
return 0;
err:
@@ -3697,3 +3705,387 @@ static void kvm_type_init(void)
}
type_init(kvm_type_init);
+
+typedef struct StatsArgs {
+ union StatsResultsType {
+ StatsResultList **stats;
+ StatsSchemaList **schema;
+ } result;
+ Error **errp;
+} StatsArgs;
+
+static StatsList *add_kvmstat_entry(struct kvm_stats_desc *pdesc,
+ uint64_t *stats_data,
+ StatsList *stats_list,
+ Error **errp)
+{
+
+ Stats *stats;
+ uint64List *val_list = NULL;
+
+ /* Only add stats that we understand. */
+ switch (pdesc->flags & KVM_STATS_TYPE_MASK) {
+ case KVM_STATS_TYPE_CUMULATIVE:
+ case KVM_STATS_TYPE_INSTANT:
+ case KVM_STATS_TYPE_PEAK:
+ case KVM_STATS_TYPE_LINEAR_HIST:
+ case KVM_STATS_TYPE_LOG_HIST:
+ break;
+ default:
+ return stats_list;
+ }
+
+ switch (pdesc->flags & KVM_STATS_UNIT_MASK) {
+ case KVM_STATS_UNIT_NONE:
+ case KVM_STATS_UNIT_BYTES:
+ case KVM_STATS_UNIT_CYCLES:
+ case KVM_STATS_UNIT_SECONDS:
+ break;
+ default:
+ return stats_list;
+ }
+
+ switch (pdesc->flags & KVM_STATS_BASE_MASK) {
+ case KVM_STATS_BASE_POW10:
+ case KVM_STATS_BASE_POW2:
+ break;
+ default:
+ return stats_list;
+ }
+
+ /* Alloc and populate data list */
+ stats = g_new0(Stats, 1);
+ stats->name = g_strdup(pdesc->name);
+ stats->value = g_new0(StatsValue, 1);;
+
+ if (pdesc->size == 1) {
+ stats->value->u.scalar = *stats_data;
+ stats->value->type = QTYPE_QNUM;
+ } else {
+ int i;
+ for (i = 0; i < pdesc->size; i++) {
+ QAPI_LIST_PREPEND(val_list, stats_data[i]);
+ }
+ stats->value->u.list = val_list;
+ stats->value->type = QTYPE_QLIST;
+ }
+
+ QAPI_LIST_PREPEND(stats_list, stats);
+ return stats_list;
+}
+
+static StatsSchemaValueList *add_kvmschema_entry(struct kvm_stats_desc *pdesc,
+ StatsSchemaValueList *list,
+ Error **errp)
+{
+ StatsSchemaValueList *schema_entry = g_new0(StatsSchemaValueList, 1);
+ schema_entry->value = g_new0(StatsSchemaValue, 1);
+
+ switch (pdesc->flags & KVM_STATS_TYPE_MASK) {
+ case KVM_STATS_TYPE_CUMULATIVE:
+ schema_entry->value->type = STATS_TYPE_CUMULATIVE;
+ break;
+ case KVM_STATS_TYPE_INSTANT:
+ schema_entry->value->type = STATS_TYPE_INSTANT;
+ break;
+ case KVM_STATS_TYPE_PEAK:
+ schema_entry->value->type = STATS_TYPE_PEAK;
+ break;
+ case KVM_STATS_TYPE_LINEAR_HIST:
+ schema_entry->value->type = STATS_TYPE_LINEAR_HISTOGRAM;
+ schema_entry->value->bucket_size = pdesc->bucket_size;
+ schema_entry->value->has_bucket_size = true;
+ break;
+ case KVM_STATS_TYPE_LOG_HIST:
+ schema_entry->value->type = STATS_TYPE_LOG2_HISTOGRAM;
+ break;
+ default:
+ goto exit;
+ }
+
+ switch (pdesc->flags & KVM_STATS_UNIT_MASK) {
+ case KVM_STATS_UNIT_NONE:
+ break;
+ case KVM_STATS_UNIT_BYTES:
+ schema_entry->value->has_unit = true;
+ schema_entry->value->unit = STATS_UNIT_BYTES;
+ break;
+ case KVM_STATS_UNIT_CYCLES:
+ schema_entry->value->has_unit = true;
+ schema_entry->value->unit = STATS_UNIT_CYCLES;
+ break;
+ case KVM_STATS_UNIT_SECONDS:
+ schema_entry->value->has_unit = true;
+ schema_entry->value->unit = STATS_UNIT_SECONDS;
+ break;
+ default:
+ goto exit;
+ }
+
+ schema_entry->value->exponent = pdesc->exponent;
+ if (pdesc->exponent) {
+ switch (pdesc->flags & KVM_STATS_BASE_MASK) {
+ case KVM_STATS_BASE_POW10:
+ schema_entry->value->has_base = true;
+ schema_entry->value->base = 10;
+ break;
+ case KVM_STATS_BASE_POW2:
+ schema_entry->value->has_base = true;
+ schema_entry->value->base = 2;
+ break;
+ default:
+ goto exit;
+ }
+ }
+
+ schema_entry->value->name = g_strdup(pdesc->name);
+ schema_entry->next = list;
+ return schema_entry;
+exit:
+ g_free(schema_entry->value);
+ g_free(schema_entry);
+ return list;
+}
+
+/* Cached stats descriptors */
+typedef struct StatsDescriptors {
+ const char *ident; /* cache key, currently the StatsTarget */
+ struct kvm_stats_desc *kvm_stats_desc;
+ struct kvm_stats_header *kvm_stats_header;
+ QTAILQ_ENTRY(StatsDescriptors) next;
+} StatsDescriptors;
+
+static QTAILQ_HEAD(, StatsDescriptors) stats_descriptors =
+ QTAILQ_HEAD_INITIALIZER(stats_descriptors);
+
+/*
+ * Return the descriptors for 'target', that either have already been read
+ * or are retrieved from 'stats_fd'.
+ */
+static StatsDescriptors *find_stats_descriptors(StatsTarget target, int stats_fd,
+ Error **errp)
+{
+ StatsDescriptors *descriptors;
+ const char *ident;
+ struct kvm_stats_desc *kvm_stats_desc;
+ struct kvm_stats_header *kvm_stats_header;
+ size_t size_desc;
+ ssize_t ret;
+
+ ident = StatsTarget_str(target);
+ QTAILQ_FOREACH(descriptors, &stats_descriptors, next) {
+ if (g_str_equal(descriptors->ident, ident)) {
+ return descriptors;
+ }
+ }
+
+ descriptors = g_new0(StatsDescriptors, 1);
+
+ /* Read stats header */
+ kvm_stats_header = g_malloc(sizeof(*kvm_stats_header));
+ ret = read(stats_fd, kvm_stats_header, sizeof(*kvm_stats_header));
+ if (ret != sizeof(*kvm_stats_header)) {
+ error_setg(errp, "KVM stats: failed to read stats header: "
+ "expected %zu actual %zu",
+ sizeof(*kvm_stats_header), ret);
+ return NULL;
+ }
+ size_desc = sizeof(*kvm_stats_desc) + kvm_stats_header->name_size;
+
+ /* Read stats descriptors */
+ kvm_stats_desc = g_malloc0_n(kvm_stats_header->num_desc, size_desc);
+ ret = pread(stats_fd, kvm_stats_desc,
+ size_desc * kvm_stats_header->num_desc,
+ kvm_stats_header->desc_offset);
+
+ if (ret != size_desc * kvm_stats_header->num_desc) {
+ error_setg(errp, "KVM stats: failed to read stats descriptors: "
+ "expected %zu actual %zu",
+ size_desc * kvm_stats_header->num_desc, ret);
+ g_free(descriptors);
+ g_free(kvm_stats_desc);
+ return NULL;
+ }
+ descriptors->kvm_stats_header = kvm_stats_header;
+ descriptors->kvm_stats_desc = kvm_stats_desc;
+ descriptors->ident = ident;
+ QTAILQ_INSERT_TAIL(&stats_descriptors, descriptors, next);
+ return descriptors;
+}
+
+static void query_stats(StatsResultList **result, StatsTarget target,
+ int stats_fd, Error **errp)
+{
+ struct kvm_stats_desc *kvm_stats_desc;
+ struct kvm_stats_header *kvm_stats_header;
+ StatsDescriptors *descriptors;
+ g_autofree uint64_t *stats_data = NULL;
+ struct kvm_stats_desc *pdesc;
+ StatsList *stats_list = NULL;
+ size_t size_desc, size_data = 0;
+ ssize_t ret;
+ int i;
+
+ descriptors = find_stats_descriptors(target, stats_fd, errp);
+ if (!descriptors) {
+ return;
+ }
+
+ kvm_stats_header = descriptors->kvm_stats_header;
+ kvm_stats_desc = descriptors->kvm_stats_desc;
+ size_desc = sizeof(*kvm_stats_desc) + kvm_stats_header->name_size;
+
+ /* Tally the total data size; read schema data */
+ for (i = 0; i < kvm_stats_header->num_desc; ++i) {
+ pdesc = (void *)kvm_stats_desc + i * size_desc;
+ size_data += pdesc->size * sizeof(*stats_data);
+ }
+
+ stats_data = g_malloc0(size_data);
+ ret = pread(stats_fd, stats_data, size_data, kvm_stats_header->data_offset);
+
+ if (ret != size_data) {
+ error_setg(errp, "KVM stats: failed to read data: "
+ "expected %zu actual %zu", size_data, ret);
+ return;
+ }
+
+ for (i = 0; i < kvm_stats_header->num_desc; ++i) {
+ uint64_t *stats;
+ pdesc = (void *)kvm_stats_desc + i * size_desc;
+
+ /* Add entry to the list */
+ stats = (void *)stats_data + pdesc->offset;
+ stats_list = add_kvmstat_entry(pdesc, stats, stats_list, errp);
+ }
+
+ if (!stats_list) {
+ return;
+ }
+
+ switch (target) {
+ case STATS_TARGET_VM:
+ add_stats_entry(result, STATS_PROVIDER_KVM, NULL, stats_list);
+ break;
+ case STATS_TARGET_VCPU:
+ add_stats_entry(result, STATS_PROVIDER_KVM,
+ current_cpu->parent_obj.canonical_path,
+ stats_list);
+ break;
+ default:
+ break;
+ }
+}
+
+static void query_stats_schema(StatsSchemaList **result, StatsTarget target,
+ int stats_fd, Error **errp)
+{
+ struct kvm_stats_desc *kvm_stats_desc;
+ struct kvm_stats_header *kvm_stats_header;
+ StatsDescriptors *descriptors;
+ struct kvm_stats_desc *pdesc;
+ StatsSchemaValueList *stats_list = NULL;
+ size_t size_desc;
+ int i;
+
+ descriptors = find_stats_descriptors(target, stats_fd, errp);
+ if (!descriptors) {
+ return;
+ }
+
+ kvm_stats_header = descriptors->kvm_stats_header;
+ kvm_stats_desc = descriptors->kvm_stats_desc;
+ size_desc = sizeof(*kvm_stats_desc) + kvm_stats_header->name_size;
+
+ /* Tally the total data size; read schema data */
+ for (i = 0; i < kvm_stats_header->num_desc; ++i) {
+ pdesc = (void *)kvm_stats_desc + i * size_desc;
+ stats_list = add_kvmschema_entry(pdesc, stats_list, errp);
+ }
+
+ add_stats_schema(result, STATS_PROVIDER_KVM, target, stats_list);
+}
+
+static void query_stats_vcpu(CPUState *cpu, run_on_cpu_data data)
+{
+ StatsArgs *kvm_stats_args = (StatsArgs *) data.host_ptr;
+ int stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
+ Error *local_err = NULL;
+
+ if (stats_fd == -1) {
+ error_setg_errno(&local_err, errno, "KVM stats: ioctl failed");
+ error_propagate(kvm_stats_args->errp, local_err);
+ return;
+ }
+ query_stats(kvm_stats_args->result.stats, STATS_TARGET_VCPU, stats_fd,
+ kvm_stats_args->errp);
+ close(stats_fd);
+}
+
+static void query_stats_schema_vcpu(CPUState *cpu, run_on_cpu_data data)
+{
+ StatsArgs *kvm_stats_args = (StatsArgs *) data.host_ptr;
+ int stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
+ Error *local_err = NULL;
+
+ if (stats_fd == -1) {
+ error_setg_errno(&local_err, errno, "KVM stats: ioctl failed");
+ error_propagate(kvm_stats_args->errp, local_err);
+ return;
+ }
+ query_stats_schema(kvm_stats_args->result.schema, STATS_TARGET_VCPU, stats_fd,
+ kvm_stats_args->errp);
+ close(stats_fd);
+}
+
+static void query_stats_cb(StatsResultList **result, StatsTarget target, Error **errp)
+{
+ KVMState *s = kvm_state;
+ CPUState *cpu;
+ int stats_fd;
+
+ switch (target) {
+ case STATS_TARGET_VM:
+ {
+ stats_fd = kvm_vm_ioctl(s, KVM_GET_STATS_FD, NULL);
+ if (stats_fd == -1) {
+ error_setg_errno(errp, errno, "KVM stats: ioctl failed");
+ return;
+ }
+ query_stats(result, target, stats_fd, errp);
+ close(stats_fd);
+ break;
+ }
+ case STATS_TARGET_VCPU:
+ {
+ StatsArgs stats_args;
+ stats_args.result.stats = result;
+ stats_args.errp = errp;
+ CPU_FOREACH(cpu) {
+ run_on_cpu(cpu, query_stats_vcpu, RUN_ON_CPU_HOST_PTR(&stats_args));
+ }
+ break;
+ }
+ default:
+ break;
+ }
+}
+
+void query_stats_schemas_cb(StatsSchemaList **result, Error **errp)
+{
+ StatsArgs stats_args;
+ KVMState *s = kvm_state;
+ int stats_fd;
+
+ stats_fd = kvm_vm_ioctl(s, KVM_GET_STATS_FD, NULL);
+ if (stats_fd == -1) {
+ error_setg_errno(errp, errno, "KVM stats: ioctl failed");
+ return;
+ }
+ query_stats_schema(result, STATS_TARGET_VM, stats_fd, errp);
+ close(stats_fd);
+
+ stats_args.result.schema = result;
+ stats_args.errp = errp;
+ run_on_cpu(first_cpu, query_stats_schema_vcpu, RUN_ON_CPU_HOST_PTR(&stats_args));
+}
diff --git a/qapi/stats.json b/qapi/stats.json
index ada0fbf26f..df7c4d886c 100644
--- a/qapi/stats.json
+++ b/qapi/stats.json
@@ -52,7 +52,7 @@
# Since: 7.1
##
{ 'enum': 'StatsProvider',
- 'data': [ ] }
+ 'data': [ 'kvm' ] }
##
# @StatsTarget:
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 03/21] qmp: add filtering of statistics by target vCPU
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
2022-06-16 8:31 ` [PULL 01/21] qmp: Support for querying stats Paolo Bonzini
2022-06-16 8:31 ` [PULL 02/21] kvm: Support for querying fd-based stats Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 04/21] cutils: add functions for IEC and SI prefixes Paolo Bonzini
` (18 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Markus Armbruster
Introduce a simple filtering of statistics, that allows to retrieve
statistics for a subset of the guest vCPUs. This will be used for
example by the HMP monitor, in order to retrieve the statistics
for the currently selected CPU.
Example:
{ "execute": "query-stats",
"arguments": {
"target": "vcpu",
"vcpus": [ "/machine/unattached/device[2]",
"/machine/unattached/device[4]" ] } }
Extracted from a patch by Mark Kanda.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 9 +++++++--
include/monitor/stats.h | 11 ++++++++++-
monitor/qmp-cmds.c | 34 +++++++++++++++++++++++++++++++++-
qapi/stats.json | 24 +++++++++++++++++++-----
4 files changed, 69 insertions(+), 9 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 7cc9e33bab..547de842fd 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2311,7 +2311,8 @@ bool kvm_dirty_ring_enabled(void)
return kvm_state->kvm_dirty_ring_size ? true : false;
}
-static void query_stats_cb(StatsResultList **result, StatsTarget target, Error **errp);
+static void query_stats_cb(StatsResultList **result, StatsTarget target,
+ strList *targets, Error **errp);
static void query_stats_schemas_cb(StatsSchemaList **result, Error **errp);
static int kvm_init(MachineState *ms)
@@ -4038,7 +4039,8 @@ static void query_stats_schema_vcpu(CPUState *cpu, run_on_cpu_data data)
close(stats_fd);
}
-static void query_stats_cb(StatsResultList **result, StatsTarget target, Error **errp)
+static void query_stats_cb(StatsResultList **result, StatsTarget target,
+ strList *targets, Error **errp)
{
KVMState *s = kvm_state;
CPUState *cpu;
@@ -4062,6 +4064,9 @@ static void query_stats_cb(StatsResultList **result, StatsTarget target, Error *
stats_args.result.stats = result;
stats_args.errp = errp;
CPU_FOREACH(cpu) {
+ if (!apply_str_list_filter(cpu->parent_obj.canonical_path, targets)) {
+ continue;
+ }
run_on_cpu(cpu, query_stats_vcpu, RUN_ON_CPU_HOST_PTR(&stats_args));
}
break;
diff --git a/include/monitor/stats.h b/include/monitor/stats.h
index 912eeadb2f..8c50feeaa9 100644
--- a/include/monitor/stats.h
+++ b/include/monitor/stats.h
@@ -11,7 +11,7 @@
#include "qapi/qapi-types-stats.h"
typedef void StatRetrieveFunc(StatsResultList **result, StatsTarget target,
- Error **errp);
+ strList *targets, Error **errp);
typedef void SchemaRetrieveFunc(StatsSchemaList **result, Error **errp);
/*
@@ -31,4 +31,13 @@ void add_stats_entry(StatsResultList **, StatsProvider, const char *id,
void add_stats_schema(StatsSchemaList **, StatsProvider, StatsTarget,
StatsSchemaValueList *);
+/*
+ * True if a string matches the filter passed to the stats_fn callabck,
+ * false otherwise.
+ *
+ * Note that an empty list means no filtering, i.e. all strings will
+ * return true.
+ */
+bool apply_str_list_filter(const char *string, strList *list);
+
#endif /* STATS_H */
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index a6ac8d7473..5f8f1e620b 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -468,9 +468,26 @@ static bool invoke_stats_cb(StatsCallbacks *entry,
StatsFilter *filter,
Error **errp)
{
+ strList *targets = NULL;
ERRP_GUARD();
- entry->stats_cb(stats_results, filter->target, errp);
+ switch (filter->target) {
+ case STATS_TARGET_VM:
+ break;
+ case STATS_TARGET_VCPU:
+ if (filter->u.vcpu.has_vcpus) {
+ if (!filter->u.vcpu.vcpus) {
+ /* No targets allowed? Return no statistics. */
+ return true;
+ }
+ targets = filter->u.vcpu.vcpus;
+ }
+ break;
+ default:
+ abort();
+ }
+
+ entry->stats_cb(stats_results, filter->target, targets, errp);
if (*errp) {
qapi_free_StatsResultList(*stats_results);
*stats_results = NULL;
@@ -536,3 +553,18 @@ void add_stats_schema(StatsSchemaList **schema_results,
entry->stats = stats_list;
QAPI_LIST_PREPEND(*schema_results, entry);
}
+
+bool apply_str_list_filter(const char *string, strList *list)
+{
+ strList *str_list = NULL;
+
+ if (!list) {
+ return true;
+ }
+ for (str_list = list; str_list; str_list = str_list->next) {
+ if (g_str_equal(string, str_list->value)) {
+ return true;
+ }
+ }
+ return false;
+}
diff --git a/qapi/stats.json b/qapi/stats.json
index df7c4d886c..8c9abb57f1 100644
--- a/qapi/stats.json
+++ b/qapi/stats.json
@@ -70,15 +70,29 @@
'data': [ 'vm', 'vcpu' ] }
##
-# @StatsFilter:
+# @StatsVCPUFilter:
#
-# The arguments to the query-stats command; specifies a target for which to
-# request statistics.
+# @vcpus: list of QOM paths for the desired vCPU objects.
#
# Since: 7.1
##
-{ 'struct': 'StatsFilter',
- 'data': { 'target': 'StatsTarget' } }
+{ 'struct': 'StatsVCPUFilter',
+ 'data': { '*vcpus': [ 'str' ] } }
+
+##
+# @StatsFilter:
+#
+# The arguments to the query-stats command; specifies a target for which to
+# request statistics and optionally the required subset of information for
+# that target:
+# - which vCPUs to request statistics for
+#
+# Since: 7.1
+##
+{ 'union': 'StatsFilter',
+ 'base': { 'target': 'StatsTarget' },
+ 'discriminator': 'target',
+ 'data': { 'vcpu': 'StatsVCPUFilter' } }
##
# @StatsValue:
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 04/21] cutils: add functions for IEC and SI prefixes
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (2 preceding siblings ...)
2022-06-16 8:31 ` [PULL 03/21] qmp: add filtering of statistics by target vCPU Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 05/21] hmp: add basic "info stats" implementation Paolo Bonzini
` (17 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel
Extract the knowledge of IEC and SI prefixes out of size_to_str and
freq_to_str, so that it can be reused when printing statistics.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
include/qemu/cutils.h | 18 ++++++++++++++
tests/unit/test-cutils.c | 52 ++++++++++++++++++++++++++++++++++++++++
util/cutils.c | 34 +++++++++++++++++++-------
3 files changed, 95 insertions(+), 9 deletions(-)
diff --git a/include/qemu/cutils.h b/include/qemu/cutils.h
index 40e10e19a7..d3e532b64c 100644
--- a/include/qemu/cutils.h
+++ b/include/qemu/cutils.h
@@ -1,6 +1,24 @@
#ifndef QEMU_CUTILS_H
#define QEMU_CUTILS_H
+/*
+ * si_prefix:
+ * @exp10: exponent of 10, a multiple of 3 between -18 and 18 inclusive.
+ *
+ * Return a SI prefix (n, u, m, K, M, etc.) corresponding
+ * to the given exponent of 10.
+ */
+const char *si_prefix(unsigned int exp10);
+
+/*
+ * iec_binary_prefix:
+ * @exp2: exponent of 2, a multiple of 10 between 0 and 60 inclusive.
+ *
+ * Return an IEC binary prefix (Ki, Mi, etc.) corresponding
+ * to the given exponent of 2.
+ */
+const char *iec_binary_prefix(unsigned int exp2);
+
/**
* pstrcpy:
* @buf: buffer to copy string into
diff --git a/tests/unit/test-cutils.c b/tests/unit/test-cutils.c
index 98671f1ac3..f5b780f012 100644
--- a/tests/unit/test-cutils.c
+++ b/tests/unit/test-cutils.c
@@ -2450,6 +2450,50 @@ static void test_qemu_strtosz_metric(void)
g_assert(endptr == str + 7);
}
+static void test_freq_to_str(void)
+{
+ g_assert_cmpstr(freq_to_str(999), ==, "999 Hz");
+ g_assert_cmpstr(freq_to_str(1000), ==, "1 KHz");
+ g_assert_cmpstr(freq_to_str(1010), ==, "1.01 KHz");
+}
+
+static void test_size_to_str(void)
+{
+ g_assert_cmpstr(size_to_str(0), ==, "0 B");
+ g_assert_cmpstr(size_to_str(1), ==, "1 B");
+ g_assert_cmpstr(size_to_str(1016), ==, "0.992 KiB");
+ g_assert_cmpstr(size_to_str(1024), ==, "1 KiB");
+ g_assert_cmpstr(size_to_str(512ull << 20), ==, "512 MiB");
+}
+
+static void test_iec_binary_prefix(void)
+{
+ g_assert_cmpstr(iec_binary_prefix(0), ==, "");
+ g_assert_cmpstr(iec_binary_prefix(10), ==, "Ki");
+ g_assert_cmpstr(iec_binary_prefix(20), ==, "Mi");
+ g_assert_cmpstr(iec_binary_prefix(30), ==, "Gi");
+ g_assert_cmpstr(iec_binary_prefix(40), ==, "Ti");
+ g_assert_cmpstr(iec_binary_prefix(50), ==, "Pi");
+ g_assert_cmpstr(iec_binary_prefix(60), ==, "Ei");
+}
+
+static void test_si_prefix(void)
+{
+ g_assert_cmpstr(si_prefix(-18), ==, "a");
+ g_assert_cmpstr(si_prefix(-15), ==, "f");
+ g_assert_cmpstr(si_prefix(-12), ==, "p");
+ g_assert_cmpstr(si_prefix(-9), ==, "n");
+ g_assert_cmpstr(si_prefix(-6), ==, "u");
+ g_assert_cmpstr(si_prefix(-3), ==, "m");
+ g_assert_cmpstr(si_prefix(0), ==, "");
+ g_assert_cmpstr(si_prefix(3), ==, "K");
+ g_assert_cmpstr(si_prefix(6), ==, "M");
+ g_assert_cmpstr(si_prefix(9), ==, "G");
+ g_assert_cmpstr(si_prefix(12), ==, "T");
+ g_assert_cmpstr(si_prefix(15), ==, "P");
+ g_assert_cmpstr(si_prefix(18), ==, "E");
+}
+
int main(int argc, char **argv)
{
g_test_init(&argc, &argv, NULL);
@@ -2729,5 +2773,13 @@ int main(int argc, char **argv)
g_test_add_func("/cutils/strtosz/metric",
test_qemu_strtosz_metric);
+ g_test_add_func("/cutils/size_to_str",
+ test_size_to_str);
+ g_test_add_func("/cutils/freq_to_str",
+ test_freq_to_str);
+ g_test_add_func("/cutils/iec_binary_prefix",
+ test_iec_binary_prefix);
+ g_test_add_func("/cutils/si_prefix",
+ test_si_prefix);
return g_test_run();
}
diff --git a/util/cutils.c b/util/cutils.c
index a58bcfd80e..6d04e52907 100644
--- a/util/cutils.c
+++ b/util/cutils.c
@@ -872,6 +872,25 @@ int parse_debug_env(const char *name, int max, int initial)
return debug;
}
+const char *si_prefix(unsigned int exp10)
+{
+ static const char *prefixes[] = {
+ "a", "f", "p", "n", "u", "m", "", "K", "M", "G", "T", "P", "E"
+ };
+
+ exp10 += 18;
+ assert(exp10 % 3 == 0 && exp10 / 3 < ARRAY_SIZE(prefixes));
+ return prefixes[exp10 / 3];
+}
+
+const char *iec_binary_prefix(unsigned int exp2)
+{
+ static const char *prefixes[] = { "", "Ki", "Mi", "Gi", "Ti", "Pi", "Ei" };
+
+ assert(exp2 % 10 == 0 && exp2 / 10 < ARRAY_SIZE(prefixes));
+ return prefixes[exp2 / 10];
+}
+
/*
* Return human readable string for size @val.
* @val can be anything that uint64_t allows (no more than "16 EiB").
@@ -880,7 +899,6 @@ int parse_debug_env(const char *name, int max, int initial)
*/
char *size_to_str(uint64_t val)
{
- static const char *suffixes[] = { "", "Ki", "Mi", "Gi", "Ti", "Pi", "Ei" };
uint64_t div;
int i;
@@ -891,25 +909,23 @@ char *size_to_str(uint64_t val)
* (see e41b509d68afb1f for more info)
*/
frexp(val / (1000.0 / 1024.0), &i);
- i = (i - 1) / 10;
- div = 1ULL << (i * 10);
+ i = (i - 1) / 10 * 10;
+ div = 1ULL << i;
- return g_strdup_printf("%0.3g %sB", (double)val / div, suffixes[i]);
+ return g_strdup_printf("%0.3g %sB", (double)val / div, iec_binary_prefix(i));
}
char *freq_to_str(uint64_t freq_hz)
{
- static const char *const suffixes[] = { "", "K", "M", "G", "T", "P", "E" };
double freq = freq_hz;
- size_t idx = 0;
+ size_t exp10 = 0;
while (freq >= 1000.0) {
freq /= 1000.0;
- idx++;
+ exp10 += 3;
}
- assert(idx < ARRAY_SIZE(suffixes));
- return g_strdup_printf("%0.3g %sHz", freq, suffixes[idx]);
+ return g_strdup_printf("%0.3g %sHz", freq, si_prefix(exp10));
}
int qemu_pstrcmp0(const char **str1, const char **str2)
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 05/21] hmp: add basic "info stats" implementation
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (3 preceding siblings ...)
2022-06-16 8:31 ` [PULL 04/21] cutils: add functions for IEC and SI prefixes Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 06/21] qmp: add filtering of statistics by provider Paolo Bonzini
` (16 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Mark Kanda, Dr . David Alan Gilbert
From: Mark Kanda <mark.kanda@oracle.com>
Add an HMP command to retrieve statistics collected at run-time.
The command will retrieve and print either all VM-level statistics,
or all vCPU-level statistics for the currently selected CPU.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hmp-commands-info.hx | 13 +++
include/monitor/hmp.h | 1 +
monitor/hmp-cmds.c | 190 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 204 insertions(+)
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 834bed089e..28757768f7 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -894,3 +894,16 @@ SRST
``info via``
Show guest mos6522 VIA devices.
ERST
+
+ {
+ .name = "stats",
+ .args_type = "target:s",
+ .params = "target",
+ .help = "show statistics; target is either vm or vcpu",
+ .cmd = hmp_info_stats,
+ },
+
+SRST
+ ``stats``
+ Show runtime-collected statistics
+ERST
diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
index 96d014826a..2e89a97bd6 100644
--- a/include/monitor/hmp.h
+++ b/include/monitor/hmp.h
@@ -133,5 +133,6 @@ void hmp_info_dirty_rate(Monitor *mon, const QDict *qdict);
void hmp_calc_dirty_rate(Monitor *mon, const QDict *qdict);
void hmp_human_readable_text_helper(Monitor *mon,
HumanReadableText *(*qmp_handler)(Error **));
+void hmp_info_stats(Monitor *mon, const QDict *qdict);
#endif
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 622c783c32..04d5ee8fb7 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -40,6 +40,7 @@
#include "qapi/qapi-commands-pci.h"
#include "qapi/qapi-commands-rocker.h"
#include "qapi/qapi-commands-run-state.h"
+#include "qapi/qapi-commands-stats.h"
#include "qapi/qapi-commands-tpm.h"
#include "qapi/qapi-commands-ui.h"
#include "qapi/qapi-visit-net.h"
@@ -52,6 +53,7 @@
#include "ui/console.h"
#include "qemu/cutils.h"
#include "qemu/error-report.h"
+#include "hw/core/cpu.h"
#include "hw/intc/intc.h"
#include "migration/snapshot.h"
#include "migration/misc.h"
@@ -2239,3 +2241,191 @@ void hmp_info_memory_size_summary(Monitor *mon, const QDict *qdict)
}
hmp_handle_error(mon, err);
}
+
+static void print_stats_schema_value(Monitor *mon, StatsSchemaValue *value)
+{
+ const char *unit = NULL;
+ monitor_printf(mon, " %s (%s%s", value->name, StatsType_str(value->type),
+ value->has_unit || value->exponent ? ", " : "");
+
+ if (value->has_unit) {
+ if (value->unit == STATS_UNIT_SECONDS) {
+ unit = "s";
+ } else if (value->unit == STATS_UNIT_BYTES) {
+ unit = "B";
+ }
+ }
+
+ if (unit && value->base == 10 &&
+ value->exponent >= -18 && value->exponent <= 18 &&
+ value->exponent % 3 == 0) {
+ monitor_printf(mon, "%s", si_prefix(value->exponent));
+ } else if (unit && value->base == 2 &&
+ value->exponent >= 0 && value->exponent <= 60 &&
+ value->exponent % 10 == 0) {
+
+ monitor_printf(mon, "%s", iec_binary_prefix(value->exponent));
+ } else if (value->exponent) {
+ /* Use exponential notation and write the unit's English name */
+ monitor_printf(mon, "* %d^%d%s",
+ value->base, value->exponent,
+ value->has_unit ? " " : "");
+ unit = NULL;
+ }
+
+ if (value->has_unit) {
+ monitor_printf(mon, "%s", unit ? unit : StatsUnit_str(value->unit));
+ }
+
+ /* Print bucket size for linear histograms */
+ if (value->type == STATS_TYPE_LINEAR_HISTOGRAM && value->has_bucket_size) {
+ monitor_printf(mon, ", bucket size=%d", value->bucket_size);
+ }
+ monitor_printf(mon, ")");
+}
+
+static StatsSchemaValueList *find_schema_value_list(
+ StatsSchemaList *list, StatsProvider provider,
+ StatsTarget target)
+{
+ StatsSchemaList *node;
+
+ for (node = list; node; node = node->next) {
+ if (node->value->provider == provider &&
+ node->value->target == target) {
+ return node->value->stats;
+ }
+ }
+ return NULL;
+}
+
+static void print_stats_results(Monitor *mon, StatsTarget target,
+ StatsResult *result,
+ StatsSchemaList *schema)
+{
+ /* Find provider schema */
+ StatsSchemaValueList *schema_value_list =
+ find_schema_value_list(schema, result->provider, target);
+ StatsList *stats_list;
+
+ if (!schema_value_list) {
+ monitor_printf(mon, "failed to find schema list for %s\n",
+ StatsProvider_str(result->provider));
+ return;
+ }
+
+ monitor_printf(mon, "provider: %s\n",
+ StatsProvider_str(result->provider));
+
+ for (stats_list = result->stats; stats_list;
+ stats_list = stats_list->next,
+ schema_value_list = schema_value_list->next) {
+
+ Stats *stats = stats_list->value;
+ StatsValue *stats_value = stats->value;
+ StatsSchemaValue *schema_value = schema_value_list->value;
+
+ /* Find schema entry */
+ while (!g_str_equal(stats->name, schema_value->name)) {
+ if (!schema_value_list->next) {
+ monitor_printf(mon, "failed to find schema entry for %s\n",
+ stats->name);
+ return;
+ }
+ schema_value_list = schema_value_list->next;
+ schema_value = schema_value_list->value;
+ }
+
+ print_stats_schema_value(mon, schema_value);
+
+ if (stats_value->type == QTYPE_QNUM) {
+ monitor_printf(mon, ": %" PRId64 "\n", stats_value->u.scalar);
+ } else if (stats_value->type == QTYPE_QLIST) {
+ uint64List *list;
+ int i;
+
+ monitor_printf(mon, ": ");
+ for (list = stats_value->u.list, i = 1;
+ list;
+ list = list->next, i++) {
+ monitor_printf(mon, "[%d]=%" PRId64 " ", i, list->value);
+ }
+ monitor_printf(mon, "\n");
+ }
+ }
+}
+
+/* Create the StatsFilter that is needed for an "info stats" invocation. */
+static StatsFilter *stats_filter(StatsTarget target, int cpu_index)
+{
+ StatsFilter *filter = g_malloc0(sizeof(*filter));
+
+ filter->target = target;
+ switch (target) {
+ case STATS_TARGET_VM:
+ break;
+ case STATS_TARGET_VCPU:
+ {
+ strList *vcpu_list = NULL;
+ CPUState *cpu = qemu_get_cpu(cpu_index);
+ char *canonical_path = object_get_canonical_path(OBJECT(cpu));
+
+ QAPI_LIST_PREPEND(vcpu_list, canonical_path);
+ filter->u.vcpu.has_vcpus = true;
+ filter->u.vcpu.vcpus = vcpu_list;
+ break;
+ }
+ default:
+ break;
+ }
+ return filter;
+}
+
+void hmp_info_stats(Monitor *mon, const QDict *qdict)
+{
+ const char *target_str = qdict_get_str(qdict, "target");
+ StatsTarget target;
+ Error *err = NULL;
+ g_autoptr(StatsSchemaList) schema = NULL;
+ g_autoptr(StatsResultList) stats = NULL;
+ g_autoptr(StatsFilter) filter = NULL;
+ StatsResultList *entry;
+
+ target = qapi_enum_parse(&StatsTarget_lookup, target_str, -1, &err);
+ if (err) {
+ monitor_printf(mon, "invalid stats target %s\n", target_str);
+ goto exit_no_print;
+ }
+
+ schema = qmp_query_stats_schemas(&err);
+ if (err) {
+ goto exit;
+ }
+
+ switch (target) {
+ case STATS_TARGET_VM:
+ filter = stats_filter(target, -1);
+ break;
+ case STATS_TARGET_VCPU: {}
+ int cpu_index = monitor_get_cpu_index(mon);
+ filter = stats_filter(target, cpu_index);
+ break;
+ default:
+ abort();
+ }
+
+ stats = qmp_query_stats(filter, &err);
+ if (err) {
+ goto exit;
+ }
+ for (entry = stats; entry; entry = entry->next) {
+ print_stats_results(mon, target, entry->value, schema);
+ }
+
+exit:
+ if (err) {
+ monitor_printf(mon, "%s\n", error_get_pretty(err));
+ }
+exit_no_print:
+ error_free(err);
+}
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 06/21] qmp: add filtering of statistics by provider
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (4 preceding siblings ...)
2022-06-16 8:31 ` [PULL 05/21] hmp: add basic "info stats" implementation Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 07/21] hmp: " Paolo Bonzini
` (15 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Markus Armbruster, Dr . David Alan Gilbert
Allow retrieving the statistics from a specific provider only.
This can be used in the future by HMP commands such as "info
sync-profile" or "info profile". The next patch also adds
filter-by-provider capabilities to the HMP equivalent of
query-stats, "info stats".
Example:
{ "execute": "query-stats",
"arguments": {
"target": "vm",
"providers": [
{ "provider": "kvm" } ] } }
The QAPI is a bit more verbose than just a list of StatsProvider,
so that it can be subsequently extended with filtering of statistics
by name.
If a provider is specified more than once in the filter, each request
will be included separately in the output.
Extracted from a patch by Mark Kanda.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 3 ++-
include/monitor/stats.h | 4 +++-
monitor/hmp-cmds.c | 2 +-
monitor/qmp-cmds.c | 41 ++++++++++++++++++++++++++++++++---------
qapi/stats.json | 19 +++++++++++++++++--
5 files changed, 55 insertions(+), 14 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 547de842fd..2e819beaeb 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2644,7 +2644,8 @@ static int kvm_init(MachineState *ms)
}
if (kvm_check_extension(kvm_state, KVM_CAP_BINARY_STATS_FD)) {
- add_stats_callbacks(query_stats_cb, query_stats_schemas_cb);
+ add_stats_callbacks(STATS_PROVIDER_KVM, query_stats_cb,
+ query_stats_schemas_cb);
}
return 0;
diff --git a/include/monitor/stats.h b/include/monitor/stats.h
index 8c50feeaa9..80a523dd29 100644
--- a/include/monitor/stats.h
+++ b/include/monitor/stats.h
@@ -17,10 +17,12 @@ typedef void SchemaRetrieveFunc(StatsSchemaList **result, Error **errp);
/*
* Register callbacks for the QMP query-stats command.
*
+ * @provider: stats provider checked against QMP command arguments
* @stats_fn: routine to query stats:
* @schema_fn: routine to query stat schemas:
*/
-void add_stats_callbacks(StatRetrieveFunc *stats_fn,
+void add_stats_callbacks(StatsProvider provider,
+ StatRetrieveFunc *stats_fn,
SchemaRetrieveFunc *schemas_fn);
/*
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 04d5ee8fb7..9180cf1841 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -2397,7 +2397,7 @@ void hmp_info_stats(Monitor *mon, const QDict *qdict)
goto exit_no_print;
}
- schema = qmp_query_stats_schemas(&err);
+ schema = qmp_query_stats_schemas(false, STATS_PROVIDER__MAX, &err);
if (err) {
goto exit;
}
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 5f8f1e620b..e49ab345d7 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -445,6 +445,7 @@ HumanReadableText *qmp_x_query_irq(Error **errp)
}
typedef struct StatsCallbacks {
+ StatsProvider provider;
StatRetrieveFunc *stats_cb;
SchemaRetrieveFunc *schemas_cb;
QTAILQ_ENTRY(StatsCallbacks) next;
@@ -453,10 +454,12 @@ typedef struct StatsCallbacks {
static QTAILQ_HEAD(, StatsCallbacks) stats_callbacks =
QTAILQ_HEAD_INITIALIZER(stats_callbacks);
-void add_stats_callbacks(StatRetrieveFunc *stats_fn,
+void add_stats_callbacks(StatsProvider provider,
+ StatRetrieveFunc *stats_fn,
SchemaRetrieveFunc *schemas_fn)
{
StatsCallbacks *entry = g_new(StatsCallbacks, 1);
+ entry->provider = provider;
entry->stats_cb = stats_fn;
entry->schemas_cb = schemas_fn;
@@ -465,12 +468,18 @@ void add_stats_callbacks(StatRetrieveFunc *stats_fn,
static bool invoke_stats_cb(StatsCallbacks *entry,
StatsResultList **stats_results,
- StatsFilter *filter,
+ StatsFilter *filter, StatsRequest *request,
Error **errp)
{
strList *targets = NULL;
ERRP_GUARD();
+ if (request) {
+ if (request->provider != entry->provider) {
+ return true;
+ }
+ }
+
switch (filter->target) {
case STATS_TARGET_VM:
break;
@@ -500,27 +509,41 @@ StatsResultList *qmp_query_stats(StatsFilter *filter, Error **errp)
{
StatsResultList *stats_results = NULL;
StatsCallbacks *entry;
+ StatsRequestList *request;
QTAILQ_FOREACH(entry, &stats_callbacks, next) {
- if (!invoke_stats_cb(entry, &stats_results, filter, errp)) {
- break;
+ if (filter->has_providers) {
+ for (request = filter->providers; request; request = request->next) {
+ if (!invoke_stats_cb(entry, &stats_results, filter,
+ request->value, errp)) {
+ break;
+ }
+ }
+ } else {
+ if (!invoke_stats_cb(entry, &stats_results, filter, NULL, errp)) {
+ break;
+ }
}
}
return stats_results;
}
-StatsSchemaList *qmp_query_stats_schemas(Error **errp)
+StatsSchemaList *qmp_query_stats_schemas(bool has_provider,
+ StatsProvider provider,
+ Error **errp)
{
StatsSchemaList *stats_results = NULL;
StatsCallbacks *entry;
ERRP_GUARD();
QTAILQ_FOREACH(entry, &stats_callbacks, next) {
- entry->schemas_cb(&stats_results, errp);
- if (*errp) {
- qapi_free_StatsSchemaList(stats_results);
- return NULL;
+ if (!has_provider || provider == entry->provider) {
+ entry->schemas_cb(&stats_results, errp);
+ if (*errp) {
+ qapi_free_StatsSchemaList(stats_results);
+ return NULL;
+ }
}
}
diff --git a/qapi/stats.json b/qapi/stats.json
index 8c9abb57f1..503918ea4c 100644
--- a/qapi/stats.json
+++ b/qapi/stats.json
@@ -69,6 +69,18 @@
{ 'enum': 'StatsTarget',
'data': [ 'vm', 'vcpu' ] }
+##
+# @StatsRequest:
+#
+# Indicates a set of statistics that should be returned by query-stats.
+#
+# @provider: provider for which to return statistics.
+#
+# Since: 7.1
+##
+{ 'struct': 'StatsRequest',
+ 'data': { 'provider': 'StatsProvider' } }
+
##
# @StatsVCPUFilter:
#
@@ -86,11 +98,14 @@
# request statistics and optionally the required subset of information for
# that target:
# - which vCPUs to request statistics for
+# - which providers to request statistics from
#
# Since: 7.1
##
{ 'union': 'StatsFilter',
- 'base': { 'target': 'StatsTarget' },
+ 'base': {
+ 'target': 'StatsTarget',
+ '*providers': [ 'StatsRequest' ] },
'discriminator': 'target',
'data': { 'vcpu': 'StatsVCPUFilter' } }
@@ -226,5 +241,5 @@
# Since: 7.1
##
{ 'command': 'query-stats-schemas',
- 'data': { },
+ 'data': { '*provider': 'StatsProvider' },
'returns': [ 'StatsSchema' ] }
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 07/21] hmp: add filtering of statistics by provider
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (5 preceding siblings ...)
2022-06-16 8:31 ` [PULL 06/21] qmp: add filtering of statistics by provider Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 08/21] qmp: add filtering of statistics by name Paolo Bonzini
` (14 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert
Allow the user to request statistics for a single provider of interest.
Extracted from a patch by Mark Kanda.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hmp-commands-info.hx | 7 ++++---
monitor/hmp-cmds.c | 39 ++++++++++++++++++++++++++++++++-------
2 files changed, 36 insertions(+), 10 deletions(-)
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 28757768f7..a67040443b 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -897,9 +897,10 @@ ERST
{
.name = "stats",
- .args_type = "target:s",
- .params = "target",
- .help = "show statistics; target is either vm or vcpu",
+ .args_type = "target:s,provider:s?",
+ .params = "target [provider]",
+ .help = "show statistics for the given target (vm or vcpu); optionally filter by "
+ "provider",
.cmd = hmp_info_stats,
},
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 9180cf1841..9278439533 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -2300,6 +2300,7 @@ static StatsSchemaValueList *find_schema_value_list(
}
static void print_stats_results(Monitor *mon, StatsTarget target,
+ bool show_provider,
StatsResult *result,
StatsSchemaList *schema)
{
@@ -2314,8 +2315,10 @@ static void print_stats_results(Monitor *mon, StatsTarget target,
return;
}
- monitor_printf(mon, "provider: %s\n",
- StatsProvider_str(result->provider));
+ if (show_provider) {
+ monitor_printf(mon, "provider: %s\n",
+ StatsProvider_str(result->provider));
+ }
for (stats_list = result->stats; stats_list;
stats_list = stats_list->next,
@@ -2356,7 +2359,8 @@ static void print_stats_results(Monitor *mon, StatsTarget target,
}
/* Create the StatsFilter that is needed for an "info stats" invocation. */
-static StatsFilter *stats_filter(StatsTarget target, int cpu_index)
+static StatsFilter *stats_filter(StatsTarget target, int cpu_index,
+ StatsProvider provider)
{
StatsFilter *filter = g_malloc0(sizeof(*filter));
@@ -2378,12 +2382,25 @@ static StatsFilter *stats_filter(StatsTarget target, int cpu_index)
default:
break;
}
+
+ if (provider == STATS_PROVIDER__MAX) {
+ return filter;
+ }
+
+ /* "info stats" can only query either one or all the providers. */
+ filter->has_providers = true;
+ filter->providers = g_new0(StatsRequestList, 1);
+ filter->providers->value = g_new0(StatsRequest, 1);
+ filter->providers->value->provider = provider;
return filter;
}
void hmp_info_stats(Monitor *mon, const QDict *qdict)
{
const char *target_str = qdict_get_str(qdict, "target");
+ const char *provider_str = qdict_get_try_str(qdict, "provider");
+
+ StatsProvider provider = STATS_PROVIDER__MAX;
StatsTarget target;
Error *err = NULL;
g_autoptr(StatsSchemaList) schema = NULL;
@@ -2396,19 +2413,27 @@ void hmp_info_stats(Monitor *mon, const QDict *qdict)
monitor_printf(mon, "invalid stats target %s\n", target_str);
goto exit_no_print;
}
+ if (provider_str) {
+ provider = qapi_enum_parse(&StatsProvider_lookup, provider_str, -1, &err);
+ if (err) {
+ monitor_printf(mon, "invalid stats provider %s\n", provider_str);
+ goto exit_no_print;
+ }
+ }
- schema = qmp_query_stats_schemas(false, STATS_PROVIDER__MAX, &err);
+ schema = qmp_query_stats_schemas(provider_str ? true : false,
+ provider, &err);
if (err) {
goto exit;
}
switch (target) {
case STATS_TARGET_VM:
- filter = stats_filter(target, -1);
+ filter = stats_filter(target, -1, provider);
break;
case STATS_TARGET_VCPU: {}
int cpu_index = monitor_get_cpu_index(mon);
- filter = stats_filter(target, cpu_index);
+ filter = stats_filter(target, cpu_index, provider);
break;
default:
abort();
@@ -2419,7 +2444,7 @@ void hmp_info_stats(Monitor *mon, const QDict *qdict)
goto exit;
}
for (entry = stats; entry; entry = entry->next) {
- print_stats_results(mon, target, entry->value, schema);
+ print_stats_results(mon, target, provider_str == NULL, entry->value, schema);
}
exit:
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 08/21] qmp: add filtering of statistics by name
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (6 preceding siblings ...)
2022-06-16 8:31 ` [PULL 07/21] hmp: " Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 09/21] hmp: " Paolo Bonzini
` (13 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert
Allow retrieving only a subset of statistics. This can be useful
for example in order to plot a subset of the statistics many times
a second: KVM publishes ~40 statistics for each vCPU on x86; retrieving
and serializing all of them would be useless.
Another use will be in HMP in the following patch; implementing the
filter in the backend is easy enough that it was deemed okay to make
this a public interface.
Example:
{ "execute": "query-stats",
"arguments": {
"target": "vcpu",
"vcpus": [ "/machine/unattached/device[2]",
"/machine/unattached/device[4]" ],
"providers": [
{ "provider": "kvm",
"names": [ "l1d_flush", "exits" ] } } }
{ "return": {
"vcpus": [
{ "path": "/machine/unattached/device[2]"
"providers": [
{ "provider": "kvm",
"stats": [ { "name": "l1d_flush", "value": 41213 },
{ "name": "exits", "value": 74291 } ] } ] },
{ "path": "/machine/unattached/device[4]"
"providers": [
{ "provider": "kvm",
"stats": [ { "name": "l1d_flush", "value": 16132 },
{ "name": "exits", "value": 57922 } ] } ] } ] } }
Extracted from a patch by Mark Kanda.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 17 +++++++++++------
include/monitor/stats.h | 2 +-
monitor/qmp-cmds.c | 7 ++++++-
qapi/stats.json | 6 +++++-
4 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 2e819beaeb..ba3210b1c1 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2312,7 +2312,7 @@ bool kvm_dirty_ring_enabled(void)
}
static void query_stats_cb(StatsResultList **result, StatsTarget target,
- strList *targets, Error **errp);
+ strList *names, strList *targets, Error **errp);
static void query_stats_schemas_cb(StatsSchemaList **result, Error **errp);
static int kvm_init(MachineState *ms)
@@ -3713,6 +3713,7 @@ typedef struct StatsArgs {
StatsResultList **stats;
StatsSchemaList **schema;
} result;
+ strList *names;
Error **errp;
} StatsArgs;
@@ -3916,7 +3917,7 @@ static StatsDescriptors *find_stats_descriptors(StatsTarget target, int stats_fd
}
static void query_stats(StatsResultList **result, StatsTarget target,
- int stats_fd, Error **errp)
+ strList *names, int stats_fd, Error **errp)
{
struct kvm_stats_desc *kvm_stats_desc;
struct kvm_stats_header *kvm_stats_header;
@@ -3958,6 +3959,9 @@ static void query_stats(StatsResultList **result, StatsTarget target,
/* Add entry to the list */
stats = (void *)stats_data + pdesc->offset;
+ if (!apply_str_list_filter(pdesc->name, names)) {
+ continue;
+ }
stats_list = add_kvmstat_entry(pdesc, stats, stats_list, errp);
}
@@ -4019,8 +4023,8 @@ static void query_stats_vcpu(CPUState *cpu, run_on_cpu_data data)
error_propagate(kvm_stats_args->errp, local_err);
return;
}
- query_stats(kvm_stats_args->result.stats, STATS_TARGET_VCPU, stats_fd,
- kvm_stats_args->errp);
+ query_stats(kvm_stats_args->result.stats, STATS_TARGET_VCPU,
+ kvm_stats_args->names, stats_fd, kvm_stats_args->errp);
close(stats_fd);
}
@@ -4041,7 +4045,7 @@ static void query_stats_schema_vcpu(CPUState *cpu, run_on_cpu_data data)
}
static void query_stats_cb(StatsResultList **result, StatsTarget target,
- strList *targets, Error **errp)
+ strList *names, strList *targets, Error **errp)
{
KVMState *s = kvm_state;
CPUState *cpu;
@@ -4055,7 +4059,7 @@ static void query_stats_cb(StatsResultList **result, StatsTarget target,
error_setg_errno(errp, errno, "KVM stats: ioctl failed");
return;
}
- query_stats(result, target, stats_fd, errp);
+ query_stats(result, target, names, stats_fd, errp);
close(stats_fd);
break;
}
@@ -4063,6 +4067,7 @@ static void query_stats_cb(StatsResultList **result, StatsTarget target,
{
StatsArgs stats_args;
stats_args.result.stats = result;
+ stats_args.names = names;
stats_args.errp = errp;
CPU_FOREACH(cpu) {
if (!apply_str_list_filter(cpu->parent_obj.canonical_path, targets)) {
diff --git a/include/monitor/stats.h b/include/monitor/stats.h
index 80a523dd29..fcf0983154 100644
--- a/include/monitor/stats.h
+++ b/include/monitor/stats.h
@@ -11,7 +11,7 @@
#include "qapi/qapi-types-stats.h"
typedef void StatRetrieveFunc(StatsResultList **result, StatsTarget target,
- strList *targets, Error **errp);
+ strList *names, strList *targets, Error **errp);
typedef void SchemaRetrieveFunc(StatsSchemaList **result, Error **errp);
/*
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index e49ab345d7..7314cd813d 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -472,12 +472,17 @@ static bool invoke_stats_cb(StatsCallbacks *entry,
Error **errp)
{
strList *targets = NULL;
+ strList *names = NULL;
ERRP_GUARD();
if (request) {
if (request->provider != entry->provider) {
return true;
}
+ if (request->has_names && !request->names) {
+ return true;
+ }
+ names = request->has_names ? request->names : NULL;
}
switch (filter->target) {
@@ -496,7 +501,7 @@ static bool invoke_stats_cb(StatsCallbacks *entry,
abort();
}
- entry->stats_cb(stats_results, filter->target, targets, errp);
+ entry->stats_cb(stats_results, filter->target, names, targets, errp);
if (*errp) {
qapi_free_StatsResultList(*stats_results);
*stats_results = NULL;
diff --git a/qapi/stats.json b/qapi/stats.json
index 503918ea4c..2f8bfe8fdb 100644
--- a/qapi/stats.json
+++ b/qapi/stats.json
@@ -75,11 +75,14 @@
# Indicates a set of statistics that should be returned by query-stats.
#
# @provider: provider for which to return statistics.
+
+# @names: statistics to be returned (all if omitted).
#
# Since: 7.1
##
{ 'struct': 'StatsRequest',
- 'data': { 'provider': 'StatsProvider' } }
+ 'data': { 'provider': 'StatsProvider',
+ '*names': [ 'str' ] } }
##
# @StatsVCPUFilter:
@@ -99,6 +102,7 @@
# that target:
# - which vCPUs to request statistics for
# - which providers to request statistics from
+# - which named values to return within each provider
#
# Since: 7.1
##
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 09/21] hmp: add filtering of statistics by name
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (7 preceding siblings ...)
2022-06-16 8:31 ` [PULL 08/21] qmp: add filtering of statistics by name Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 10/21] block: add more commands to preconfig mode Paolo Bonzini
` (12 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert
Allow the user to request only a specific subset of statistics.
This can be useful when working on a feature or optimization that is
known to affect that statistic.
Example:
(qemu) info stats vcpu halt_poll_fail_ns
provider: kvm
halt_poll_fail_ns (cumulative, ns): 0
In case multiple providers have the same statistic, the provider can be
specified too:
(qemu) info stats vcpu halt_poll_fail_ns kvm
provider: kvm
halt_poll_fail_ns (cumulative, ns): 0
Extracted from a patch by Mark Kanda.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hmp-commands-info.hx | 8 ++++----
monitor/hmp-cmds.c | 35 ++++++++++++++++++++++++++---------
2 files changed, 30 insertions(+), 13 deletions(-)
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index a67040443b..3ffa24bd67 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -897,10 +897,10 @@ ERST
{
.name = "stats",
- .args_type = "target:s,provider:s?",
- .params = "target [provider]",
- .help = "show statistics for the given target (vm or vcpu); optionally filter by "
- "provider",
+ .args_type = "target:s,names:s?,provider:s?",
+ .params = "target [names] [provider]",
+ .help = "show statistics for the given target (vm or vcpu); optionally filter by"
+ "name (comma-separated list, or * for all) and provider",
.cmd = hmp_info_stats,
},
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 9278439533..47a27326ee 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -2359,10 +2359,12 @@ static void print_stats_results(Monitor *mon, StatsTarget target,
}
/* Create the StatsFilter that is needed for an "info stats" invocation. */
-static StatsFilter *stats_filter(StatsTarget target, int cpu_index,
- StatsProvider provider)
+static StatsFilter *stats_filter(StatsTarget target, const char *names,
+ int cpu_index, StatsProvider provider)
{
StatsFilter *filter = g_malloc0(sizeof(*filter));
+ StatsProvider provider_idx;
+ StatsRequestList *request_list = NULL;
filter->target = target;
switch (target) {
@@ -2383,15 +2385,29 @@ static StatsFilter *stats_filter(StatsTarget target, int cpu_index,
break;
}
- if (provider == STATS_PROVIDER__MAX) {
+ if (!names && provider == STATS_PROVIDER__MAX) {
return filter;
}
- /* "info stats" can only query either one or all the providers. */
+ /*
+ * "info stats" can only query either one or all the providers. Querying
+ * by name, but not by provider, requires the creation of one filter per
+ * provider.
+ */
+ for (provider_idx = 0; provider_idx < STATS_PROVIDER__MAX; provider_idx++) {
+ if (provider == STATS_PROVIDER__MAX || provider == provider_idx) {
+ StatsRequest *request = g_new0(StatsRequest, 1);
+ request->provider = provider_idx;
+ if (names && !g_str_equal(names, "*")) {
+ request->has_names = true;
+ request->names = strList_from_comma_list(names);
+ }
+ QAPI_LIST_PREPEND(request_list, request);
+ }
+ }
+
filter->has_providers = true;
- filter->providers = g_new0(StatsRequestList, 1);
- filter->providers->value = g_new0(StatsRequest, 1);
- filter->providers->value->provider = provider;
+ filter->providers = request_list;
return filter;
}
@@ -2399,6 +2415,7 @@ void hmp_info_stats(Monitor *mon, const QDict *qdict)
{
const char *target_str = qdict_get_str(qdict, "target");
const char *provider_str = qdict_get_try_str(qdict, "provider");
+ const char *names = qdict_get_try_str(qdict, "names");
StatsProvider provider = STATS_PROVIDER__MAX;
StatsTarget target;
@@ -2429,11 +2446,11 @@ void hmp_info_stats(Monitor *mon, const QDict *qdict)
switch (target) {
case STATS_TARGET_VM:
- filter = stats_filter(target, -1, provider);
+ filter = stats_filter(target, names, -1, provider);
break;
case STATS_TARGET_VCPU: {}
int cpu_index = monitor_get_cpu_index(mon);
- filter = stats_filter(target, cpu_index, provider);
+ filter = stats_filter(target, names, cpu_index, provider);
break;
default:
abort();
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 10/21] block: add more commands to preconfig mode
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (8 preceding siblings ...)
2022-06-16 8:31 ` [PULL 09/21] hmp: " Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:31 ` [PULL 11/21] s390x: simplify virtio_ccw_reset_virtio Paolo Bonzini
` (11 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Daniel P . Berrangé
Of the block device commands, those that are available outside system
emulators do not require a fully constructed machine by definition.
Allow running them before machine initialization has concluded.
Of the ones that are available inside system emulation, allow querying
the PR managers, and setting up accounting and throttling.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hmp-commands.hx | 14 +++++
qapi/block-core.json | 117 +++++++++++++++++++++++++++--------------
qapi/block-export.json | 21 +++++---
qapi/block.json | 6 ++-
4 files changed, 110 insertions(+), 48 deletions(-)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 564f1de364..c9d465735a 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -78,6 +78,7 @@ ERST
.help = "resize a block image",
.cmd = hmp_block_resize,
.coroutine = true,
+ .flags = "p",
},
SRST
@@ -94,6 +95,7 @@ ERST
.params = "device [speed [base]]",
.help = "copy data from a backing file into a block device",
.cmd = hmp_block_stream,
+ .flags = "p",
},
SRST
@@ -107,6 +109,7 @@ ERST
.params = "device speed",
.help = "set maximum speed for a background block operation",
.cmd = hmp_block_job_set_speed,
+ .flags = "p",
},
SRST
@@ -122,6 +125,7 @@ ERST
"\n\t\t\t if you want to abort the operation immediately"
"\n\t\t\t instead of keep running until data is in sync)",
.cmd = hmp_block_job_cancel,
+ .flags = "p",
},
SRST
@@ -135,6 +139,7 @@ ERST
.params = "device",
.help = "stop an active background block operation",
.cmd = hmp_block_job_complete,
+ .flags = "p",
},
SRST
@@ -149,6 +154,7 @@ ERST
.params = "device",
.help = "pause an active background block operation",
.cmd = hmp_block_job_pause,
+ .flags = "p",
},
SRST
@@ -162,6 +168,7 @@ ERST
.params = "device",
.help = "resume a paused background block operation",
.cmd = hmp_block_job_resume,
+ .flags = "p",
},
SRST
@@ -1406,6 +1413,7 @@ ERST
.params = "nbd_server_start [-a] [-w] host:port",
.help = "serve block devices on the given host and port",
.cmd = hmp_nbd_server_start,
+ .flags = "p",
},
SRST
``nbd_server_start`` *host*:*port*
@@ -1421,6 +1429,7 @@ ERST
.params = "nbd_server_add [-w] device [name]",
.help = "export a block device via NBD",
.cmd = hmp_nbd_server_add,
+ .flags = "p",
},
SRST
``nbd_server_add`` *device* [ *name* ]
@@ -1436,6 +1445,7 @@ ERST
.params = "nbd_server_remove [-f] name",
.help = "remove an export previously exposed via NBD",
.cmd = hmp_nbd_server_remove,
+ .flags = "p",
},
SRST
``nbd_server_remove [-f]`` *name*
@@ -1452,6 +1462,7 @@ ERST
.params = "nbd_server_stop",
.help = "stop serving block devices using the NBD protocol",
.cmd = hmp_nbd_server_stop,
+ .flags = "p",
},
SRST
``nbd_server_stop``
@@ -1481,6 +1492,7 @@ ERST
.params = "getfd name",
.help = "receive a file descriptor via SCM rights and assign it a name",
.cmd = hmp_getfd,
+ .flags = "p",
},
SRST
@@ -1496,6 +1508,7 @@ ERST
.params = "closefd name",
.help = "close a file descriptor previously passed via SCM rights",
.cmd = hmp_closefd,
+ .flags = "p",
},
SRST
@@ -1511,6 +1524,7 @@ ERST
.params = "device bps bps_rd bps_wr iops iops_rd iops_wr",
.help = "change I/O throttle limits for a block drive",
.cmd = hmp_block_set_io_throttle,
+ .flags = "p",
},
SRST
diff --git a/qapi/block-core.json b/qapi/block-core.json
index f0383c7925..457df16638 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -737,7 +737,8 @@
# }
#
##
-{ 'command': 'query-block', 'returns': ['BlockInfo'] }
+{ 'command': 'query-block', 'returns': ['BlockInfo'],
+ 'allow-preconfig': true }
##
# @BlockDeviceTimedStats:
@@ -1113,7 +1114,8 @@
##
{ 'command': 'query-blockstats',
'data': { '*query-nodes': 'bool' },
- 'returns': ['BlockStats'] }
+ 'returns': ['BlockStats'],
+ 'allow-preconfig': true }
##
# @BlockdevOnError:
@@ -1262,7 +1264,8 @@
#
# Since: 1.1
##
-{ 'command': 'query-block-jobs', 'returns': ['BlockJobInfo'] }
+{ 'command': 'query-block-jobs', 'returns': ['BlockJobInfo'],
+ 'allow-preconfig': true }
##
# @block_resize:
@@ -1293,7 +1296,8 @@
'data': { '*device': 'str',
'*node-name': 'str',
'size': 'int' },
- 'coroutine': true }
+ 'coroutine': true,
+ 'allow-preconfig': true }
##
# @NewImageMode:
@@ -1509,7 +1513,8 @@
#
##
{ 'command': 'blockdev-snapshot-sync',
- 'data': 'BlockdevSnapshotSync' }
+ 'data': 'BlockdevSnapshotSync',
+ 'allow-preconfig': true }
##
# @blockdev-snapshot:
@@ -1550,7 +1555,8 @@
##
{ 'command': 'blockdev-snapshot',
'data': 'BlockdevSnapshot',
- 'features': [ 'allow-write-only-overlay' ] }
+ 'features': [ 'allow-write-only-overlay' ],
+ 'allow-preconfig': true }
##
# @change-backing-file:
@@ -1582,7 +1588,8 @@
##
{ 'command': 'change-backing-file',
'data': { 'device': 'str', 'image-node-name': 'str',
- 'backing-file': 'str' } }
+ 'backing-file': 'str' },
+ 'allow-preconfig': true }
##
# @block-commit:
@@ -1692,7 +1699,8 @@
'*backing-file': 'str', '*speed': 'int',
'*on-error': 'BlockdevOnError',
'*filter-node-name': 'str',
- '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
+ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' },
+ 'allow-preconfig': true }
##
# @drive-backup:
@@ -1721,7 +1729,8 @@
#
##
{ 'command': 'drive-backup', 'boxed': true,
- 'data': 'DriveBackup', 'features': ['deprecated'] }
+ 'data': 'DriveBackup', 'features': ['deprecated'],
+ 'allow-preconfig': true }
##
# @blockdev-backup:
@@ -1747,7 +1756,8 @@
#
##
{ 'command': 'blockdev-backup', 'boxed': true,
- 'data': 'BlockdevBackup' }
+ 'data': 'BlockdevBackup',
+ 'allow-preconfig': true }
##
# @query-named-block-nodes:
@@ -1813,7 +1823,8 @@
##
{ 'command': 'query-named-block-nodes',
'returns': [ 'BlockDeviceInfo' ],
- 'data': { '*flat': 'bool' } }
+ 'data': { '*flat': 'bool' },
+ 'allow-preconfig': true }
##
# @XDbgBlockGraphNodeType:
@@ -1922,7 +1933,8 @@
# Since: 4.0
##
{ 'command': 'x-debug-query-block-graph', 'returns': 'XDbgBlockGraph',
- 'features': [ 'unstable' ] }
+ 'features': [ 'unstable' ],
+ 'allow-preconfig': true }
##
# @drive-mirror:
@@ -1950,7 +1962,8 @@
#
##
{ 'command': 'drive-mirror', 'boxed': true,
- 'data': 'DriveMirror' }
+ 'data': 'DriveMirror',
+ 'allow-preconfig': true }
##
# @DriveMirror:
@@ -2123,7 +2136,8 @@
#
##
{ 'command': 'block-dirty-bitmap-add',
- 'data': 'BlockDirtyBitmapAdd' }
+ 'data': 'BlockDirtyBitmapAdd',
+ 'allow-preconfig': true }
##
# @block-dirty-bitmap-remove:
@@ -2147,7 +2161,8 @@
#
##
{ 'command': 'block-dirty-bitmap-remove',
- 'data': 'BlockDirtyBitmap' }
+ 'data': 'BlockDirtyBitmap',
+ 'allow-preconfig': true }
##
# @block-dirty-bitmap-clear:
@@ -2170,7 +2185,8 @@
#
##
{ 'command': 'block-dirty-bitmap-clear',
- 'data': 'BlockDirtyBitmap' }
+ 'data': 'BlockDirtyBitmap',
+ 'allow-preconfig': true }
##
# @block-dirty-bitmap-enable:
@@ -2191,7 +2207,8 @@
#
##
{ 'command': 'block-dirty-bitmap-enable',
- 'data': 'BlockDirtyBitmap' }
+ 'data': 'BlockDirtyBitmap',
+ 'allow-preconfig': true }
##
# @block-dirty-bitmap-disable:
@@ -2212,7 +2229,8 @@
#
##
{ 'command': 'block-dirty-bitmap-disable',
- 'data': 'BlockDirtyBitmap' }
+ 'data': 'BlockDirtyBitmap',
+ 'allow-preconfig': true }
##
# @block-dirty-bitmap-merge:
@@ -2244,7 +2262,8 @@
#
##
{ 'command': 'block-dirty-bitmap-merge',
- 'data': 'BlockDirtyBitmapMerge' }
+ 'data': 'BlockDirtyBitmapMerge',
+ 'allow-preconfig': true }
##
# @BlockDirtyBitmapSha256:
@@ -2275,7 +2294,8 @@
##
{ 'command': 'x-debug-block-dirty-bitmap-sha256',
'data': 'BlockDirtyBitmap', 'returns': 'BlockDirtyBitmapSha256',
- 'features': [ 'unstable' ] }
+ 'features': [ 'unstable' ],
+ 'allow-preconfig': true }
##
# @blockdev-mirror:
@@ -2361,7 +2381,8 @@
'*on-target-error': 'BlockdevOnError',
'*filter-node-name': 'str',
'*copy-mode': 'MirrorCopyMode',
- '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
+ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' },
+ 'allow-preconfig': true }
##
# @BlockIOThrottle:
@@ -2663,7 +2684,8 @@
'*base-node': 'str', '*backing-file': 'str', '*bottom': 'str',
'*speed': 'int', '*on-error': 'BlockdevOnError',
'*filter-node-name': 'str',
- '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
+ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' },
+ 'allow-preconfig': true }
##
# @block-job-set-speed:
@@ -2687,7 +2709,8 @@
# Since: 1.1
##
{ 'command': 'block-job-set-speed',
- 'data': { 'device': 'str', 'speed': 'int' } }
+ 'data': { 'device': 'str', 'speed': 'int' },
+ 'allow-preconfig': true }
##
# @block-job-cancel:
@@ -2726,7 +2749,8 @@
#
# Since: 1.1
##
-{ 'command': 'block-job-cancel', 'data': { 'device': 'str', '*force': 'bool' } }
+{ 'command': 'block-job-cancel', 'data': { 'device': 'str', '*force': 'bool' },
+ 'allow-preconfig': true }
##
# @block-job-pause:
@@ -2750,7 +2774,8 @@
#
# Since: 1.3
##
-{ 'command': 'block-job-pause', 'data': { 'device': 'str' } }
+{ 'command': 'block-job-pause', 'data': { 'device': 'str' },
+ 'allow-preconfig': true }
##
# @block-job-resume:
@@ -2772,7 +2797,8 @@
#
# Since: 1.3
##
-{ 'command': 'block-job-resume', 'data': { 'device': 'str' } }
+{ 'command': 'block-job-resume', 'data': { 'device': 'str' },
+ 'allow-preconfig': true }
##
# @block-job-complete:
@@ -2800,7 +2826,8 @@
#
# Since: 1.3
##
-{ 'command': 'block-job-complete', 'data': { 'device': 'str' } }
+{ 'command': 'block-job-complete', 'data': { 'device': 'str' },
+ 'allow-preconfig': true }
##
# @block-job-dismiss:
@@ -2820,7 +2847,8 @@
#
# Since: 2.12
##
-{ 'command': 'block-job-dismiss', 'data': { 'id': 'str' } }
+{ 'command': 'block-job-dismiss', 'data': { 'id': 'str' },
+ 'allow-preconfig': true }
##
# @block-job-finalize:
@@ -2838,7 +2866,8 @@
#
# Since: 2.12
##
-{ 'command': 'block-job-finalize', 'data': { 'id': 'str' } }
+{ 'command': 'block-job-finalize', 'data': { 'id': 'str' },
+ 'allow-preconfig': true }
##
# @BlockdevDiscardOptions:
@@ -4354,7 +4383,8 @@
# <- { "return": {} }
#
##
-{ 'command': 'blockdev-add', 'data': 'BlockdevOptions', 'boxed': true }
+{ 'command': 'blockdev-add', 'data': 'BlockdevOptions', 'boxed': true,
+ 'allow-preconfig': true }
##
# @blockdev-reopen:
@@ -4398,7 +4428,8 @@
# Since: 6.1
##
{ 'command': 'blockdev-reopen',
- 'data': { 'options': ['BlockdevOptions'] } }
+ 'data': { 'options': ['BlockdevOptions'] },
+ 'allow-preconfig': true }
##
# @blockdev-del:
@@ -4431,7 +4462,8 @@
# <- { "return": {} }
#
##
-{ 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
+{ 'command': 'blockdev-del', 'data': { 'node-name': 'str' },
+ 'allow-preconfig': true }
##
# @BlockdevCreateOptionsFile:
@@ -4872,7 +4904,8 @@
##
{ 'command': 'blockdev-create',
'data': { 'job-id': 'str',
- 'options': 'BlockdevCreateOptions' } }
+ 'options': 'BlockdevCreateOptions' },
+ 'allow-preconfig': true }
##
# @BlockdevAmendOptionsLUKS:
@@ -4944,7 +4977,8 @@
'node-name': 'str',
'options': 'BlockdevAmendOptions',
'*force': 'bool' },
- 'features': [ 'unstable' ] }
+ 'features': [ 'unstable' ],
+ 'allow-preconfig': true }
##
# @BlockErrorAction:
@@ -5294,7 +5328,8 @@
#
##
{ 'command': 'block-set-write-threshold',
- 'data': { 'node-name': 'str', 'write-threshold': 'uint64' } }
+ 'data': { 'node-name': 'str', 'write-threshold': 'uint64' },
+ 'allow-preconfig': true }
##
# @x-blockdev-change:
@@ -5355,7 +5390,8 @@
'data' : { 'parent': 'str',
'*child': 'str',
'*node': 'str' },
- 'features': [ 'unstable' ] }
+ 'features': [ 'unstable' ],
+ 'allow-preconfig': true }
##
# @x-blockdev-set-iothread:
@@ -5397,7 +5433,8 @@
'data' : { 'node-name': 'str',
'iothread': 'StrOrNull',
'*force': 'bool' },
- 'features': [ 'unstable' ] }
+ 'features': [ 'unstable' ],
+ 'allow-preconfig': true }
##
# @QuorumOpType:
@@ -5529,7 +5566,8 @@
#
##
{ 'command': 'blockdev-snapshot-internal-sync',
- 'data': 'BlockdevSnapshotInternal' }
+ 'data': 'BlockdevSnapshotInternal',
+ 'allow-preconfig': true }
##
# @blockdev-snapshot-delete-internal-sync:
@@ -5576,4 +5614,5 @@
##
{ 'command': 'blockdev-snapshot-delete-internal-sync',
'data': { 'device': 'str', '*id': 'str', '*name': 'str'},
- 'returns': 'SnapshotInfo' }
+ 'returns': 'SnapshotInfo',
+ 'allow-preconfig': true }
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 0685cb8b9a..8afb1b65b3 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -65,7 +65,8 @@
'data': { 'addr': 'SocketAddressLegacy',
'*tls-creds': 'str',
'*tls-authz': 'str',
- '*max-connections': 'uint32' } }
+ '*max-connections': 'uint32' },
+ 'allow-preconfig': true }
##
# @BlockExportOptionsNbdBase:
@@ -215,7 +216,8 @@
# Since: 1.3
##
{ 'command': 'nbd-server-add',
- 'data': 'NbdServerAddOptions', 'boxed': true, 'features': ['deprecated'] }
+ 'data': 'NbdServerAddOptions', 'boxed': true, 'features': ['deprecated'],
+ 'allow-preconfig': true }
##
# @BlockExportRemoveMode:
@@ -260,7 +262,8 @@
##
{ 'command': 'nbd-server-remove',
'data': {'name': 'str', '*mode': 'BlockExportRemoveMode'},
- 'features': ['deprecated'] }
+ 'features': ['deprecated'],
+ 'allow-preconfig': true }
##
# @nbd-server-stop:
@@ -270,7 +273,8 @@
#
# Since: 1.3
##
-{ 'command': 'nbd-server-stop' }
+{ 'command': 'nbd-server-stop',
+ 'allow-preconfig': true }
##
# @BlockExportType:
@@ -342,7 +346,8 @@
# Since: 5.2
##
{ 'command': 'block-export-add',
- 'data': 'BlockExportOptions', 'boxed': true }
+ 'data': 'BlockExportOptions', 'boxed': true,
+ 'allow-preconfig': true }
##
# @block-export-del:
@@ -362,7 +367,8 @@
# Since: 5.2
##
{ 'command': 'block-export-del',
- 'data': { 'id': 'str', '*mode': 'BlockExportRemoveMode' } }
+ 'data': { 'id': 'str', '*mode': 'BlockExportRemoveMode' },
+ 'allow-preconfig': true }
##
# @BLOCK_EXPOR '*boundaries': ['uint64'],
'*boundaries-read': ['uint64'],
'*boundaries-write': ['uint64'],
- '*boundaries-flush': ['uint64'] } }
+ '*boundaries-flush': ['uint64'] },
+ 'allow-preconfig': true }
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 11/21] s390x: simplify virtio_ccw_reset_virtio
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (9 preceding siblings ...)
2022-06-16 8:31 ` [PULL 10/21] block: add more commands to preconfig mode Paolo Bonzini
@ 2022-06-16 8:31 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 12/21] virtio-mmio: stop ioeventfd on legacy reset Paolo Bonzini
` (10 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:31 UTC (permalink / raw)
To: qemu-devel; +Cc: Cornelia Huck
Call virtio_bus_reset instead of virtio_reset, so that the function
need not receive the VirtIODevice.
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/s390x/virtio-ccw.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index 15b458527e..066a387802 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -249,12 +249,12 @@ static int virtio_ccw_set_vqs(SubchDev *sch, VqInfoBlock *info,
return 0;
}
-static void virtio_ccw_reset_virtio(VirtioCcwDevice *dev, VirtIODevice *vdev)
+static void virtio_ccw_reset_virtio(VirtioCcwDevice *dev)
{
CcwDevice *ccw_dev = CCW_DEVICE(dev);
virtio_ccw_stop_ioeventfd(dev);
- virtio_reset(vdev);
+ virtio_bus_reset(&dev->bus);
if (dev->indicators) {
release_indicator(&dev->routes.adapter, dev->indicators);
dev->indicators = NULL;
@@ -359,7 +359,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
ret = virtio_ccw_handle_set_vq(sch, ccw, check_len, dev->revision < 1);
break;
case CCW_CMD_VDEV_RESET:
- virtio_ccw_reset_virtio(dev, vdev);
+ virtio_ccw_reset_virtio(dev);
ret = 0;
break;
case CCW_CMD_READ_FEAT:
@@ -536,7 +536,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
}
if (virtio_set_status(vdev, status) == 0) {
if (vdev->status == 0) {
- virtio_ccw_reset_virtio(dev, vdev);
+ virtio_ccw_reset_virtio(dev);
}
if (status & VIRTIO_CONFIG_S_DRIVER_OK) {
virtio_ccw_start_ioeventfd(dev);
@@ -921,10 +921,9 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
static void virtio_ccw_reset(DeviceState *d)
{
VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
- VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
VirtIOCCWDeviceClass *vdc = VIRTIO_CCW_DEVICE_GET_CLASS(dev);
- virtio_ccw_reset_virtio(dev, vdev);
+ virtio_ccw_reset_virtio(dev);
if (vdc->parent_reset) {
vdc->parent_reset(d);
}
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 12/21] virtio-mmio: stop ioeventfd on legacy reset
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (10 preceding siblings ...)
2022-06-16 8:31 ` [PULL 11/21] s390x: simplify virtio_ccw_reset_virtio Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 13/21] virtio: stop ioeventfd on reset Paolo Bonzini
` (9 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel; +Cc: Cornelia Huck
If the queue PFN is set to zero on a virtio-mmio device, the device is reset.
In that case however the virtio_bus_stop_ioeventfd function was not
called; add it so that the behavior is similar to when status is set to 0.
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/virtio/virtio-mmio.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index 688eccda94..41a35d31c8 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -376,6 +376,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
return;
}
if (value == 0) {
+ virtio_mmio_stop_ioeventfd(proxy);
virtio_reset(vdev);
} else {
virtio_queue_set_addr(vdev, vdev->queue_sel,
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 13/21] virtio: stop ioeventfd on reset
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (11 preceding siblings ...)
2022-06-16 8:32 ` [PULL 12/21] virtio-mmio: stop ioeventfd on legacy reset Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 14/21] virtio-mmio: cleanup reset Paolo Bonzini
` (8 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel; +Cc: Cornelia Huck
All calls to virtio_bus_reset are preceded by virtio_bus_stop_ioeventfd,
move the call in virtio_bus_reset: that makes sense and clarifies
that the vdc->reset function is called with ioeventfd already stopped.
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/s390x/virtio-ccw.c | 1 -
hw/virtio/virtio-bus.c | 1 +
hw/virtio/virtio-mmio.c | 4 +---
hw/virtio/virtio-pci.c | 1 -
4 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index 066a387802..e33e5207ab 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -253,7 +253,6 @@ static void virtio_ccw_reset_virtio(VirtioCcwDevice *dev)
{
CcwDevice *ccw_dev = CCW_DEVICE(dev);
- virtio_ccw_stop_ioeventfd(dev);
virtio_bus_reset(&dev->bus);
if (dev->indicators) {
release_indicator(&dev->routes.adapter, dev->indicators);
diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index d7ec023adf..896feb37a1 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -104,6 +104,7 @@ void virtio_bus_reset(VirtioBusState *bus)
VirtIODevice *vdev = virtio_bus_get_device(bus);
DPRINTF("%s: reset device.\n", BUS(bus)->name);
+ virtio_bus_stop_ioeventfd(bus);
if (vdev != NULL) {
virtio_reset(vdev);
}
diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index 41a35d31c8..6d81a26473 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -376,8 +376,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
return;
}
if (value == 0) {
- virtio_mmio_stop_ioeventfd(proxy);
- virtio_reset(vdev);
+ virtio_bus_reset(&vdev->bus);
} else {
virtio_queue_set_addr(vdev, vdev->queue_sel,
value << proxy->guest_page_shift);
@@ -628,7 +627,6 @@ static void virtio_mmio_reset(DeviceState *d)
VirtIOMMIOProxy *proxy = VIRTIO_MMIO(d);
int i;
- virtio_mmio_stop_ioeventfd(proxy);
virtio_bus_reset(&proxy->bus);
proxy->host_features_sel = 0;
proxy->guest_features_sel = 0;
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 0566ad7d00..45327f0b31 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1945,7 +1945,6 @@ static void virtio_pci_reset(DeviceState *qdev)
PCIDevice *dev = PCI_DEVICE(qdev);
int i;
- virtio_pci_stop_ioeventfd(proxy);
virtio_bus_reset(bus);
msix_unuse_all_vectors(&proxy->pci_dev);
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 14/21] virtio-mmio: cleanup reset
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (12 preceding siblings ...)
2022-06-16 8:32 ` [PULL 13/21] virtio: stop ioeventfd on reset Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 15/21] configure: update list of preserved environment variables Paolo Bonzini
` (7 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel
Make virtio_mmio_soft_reset reset the virtio device, which is performed by
both the "soft" and the "hard" reset; and then call virtio_mmio_soft_reset
from virtio_mmio_reset to emphasize that the latter is a superset of the
former.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/virtio/virtio-mmio.c | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/hw/virtio/virtio-mmio.c b/hw/virtio/virtio-mmio.c
index 6d81a26473..d240efef97 100644
--- a/hw/virtio/virtio-mmio.c
+++ b/hw/virtio/virtio-mmio.c
@@ -72,12 +72,12 @@ static void virtio_mmio_soft_reset(VirtIOMMIOProxy *proxy)
{
int i;
- if (proxy->legacy) {
- return;
- }
+ virtio_bus_reset(&proxy->bus);
- for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
- proxy->vqs[i].enabled = 0;
+ if (!proxy->legacy) {
+ for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
+ proxy->vqs[i].enabled = 0;
+ }
}
}
@@ -376,7 +376,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
return;
}
if (value == 0) {
- virtio_bus_reset(&vdev->bus);
+ virtio_mmio_soft_reset(proxy);
} else {
virtio_queue_set_addr(vdev, vdev->queue_sel,
value << proxy->guest_page_shift);
@@ -432,7 +432,6 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
}
if (vdev->status == 0) {
- virtio_reset(vdev);
virtio_mmio_soft_reset(proxy);
}
break;
@@ -627,7 +626,8 @@ static void virtio_mmio_reset(DeviceState *d)
VirtIOMMIOProxy *proxy = VIRTIO_MMIO(d);
int i;
- virtio_bus_reset(&proxy->bus);
+ virtio_mmio_soft_reset(proxy);
+
proxy->host_features_sel = 0;
proxy->guest_features_sel = 0;
proxy->guest_page_shift = 0;
@@ -636,7 +636,6 @@ static void virtio_mmio_reset(DeviceState *d)
proxy->guest_features[0] = proxy->guest_features[1] = 0;
for (i = 0; i < VIRTIO_QUEUE_MAX; i++) {
- proxy->vqs[i].enabled = 0;
proxy->vqs[i].num = 0;
proxy->vqs[i].desc[0] = proxy->vqs[i].desc[1] = 0;
proxy->vqs[i].avail[0] = proxy->vqs[i].avail[1] = 0;
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 15/21] configure: update list of preserved environment variables
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (13 preceding siblings ...)
2022-06-16 8:32 ` [PULL 14/21] virtio-mmio: cleanup reset Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 16/21] configure: cleanup -fno-pie detection Paolo Bonzini
` (6 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel
INSTALL and LIBTOOL are not used anymore, but OBJCFLAGS is new and
was not listed.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
configure | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/configure b/configure
index 4b12a8094c..d41c7eddff 100755
--- a/configure
+++ b/configure
@@ -2722,13 +2722,12 @@ preserve_env CC
preserve_env CFLAGS
preserve_env CXX
preserve_env CXXFLAGS
-preserve_env INSTALL
preserve_env LD
preserve_env LDFLAGS
preserve_env LD_LIBRARY_PATH
-preserve_env LIBTOOL
preserve_env MAKE
preserve_env NM
+preserve_env OBJCFLAGS
preserve_env OBJCOPY
preserve_env PATH
preserve_env PKG_CONFIG
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 16/21] configure: cleanup -fno-pie detection
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (14 preceding siblings ...)
2022-06-16 8:32 ` [PULL 15/21] configure: update list of preserved environment variables Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 17/21] tests/vm: allow running tests in an unconfigured source tree Paolo Bonzini
` (5 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel; +Cc: Richard Henderson
Place it only inside the 'if test "$pie" = "no"' conditional. Since
commit 43924d1e53 ("pc-bios/optionrom: detect -fno-pie", 2022-05-12),
the PIE options are detected independently by pc-bios/optionrom/Makefile,
and the CFLAGS_NOPIE/LDFLAGS_NOPIE variables are not used anymore.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
configure | 13 ++++---------
1 file changed, 4 insertions(+), 9 deletions(-)
diff --git a/configure b/configure
index d41c7eddff..9fba134746 100755
--- a/configure
+++ b/configure
@@ -1346,13 +1346,6 @@ static THREAD int tls_var;
int main(void) { return tls_var; }
EOF
-# Check we support -fno-pie and -no-pie first; we will need the former for
-# building ROMs, and both for everything if --disable-pie is passed.
-if compile_prog "-Werror -fno-pie" "-no-pie"; then
- CFLAGS_NOPIE="-fno-pie"
- LDFLAGS_NOPIE="-no-pie"
-fi
-
if test "$static" = "yes"; then
if test "$pie" != "no" && compile_prog "-Werror -fPIE -DPIE" "-static-pie"; then
CONFIGURE_CFLAGS="-fPIE -DPIE $CONFIGURE_CFLAGS"
@@ -1365,8 +1358,10 @@ if test "$static" = "yes"; then
pie="no"
fi
elif test "$pie" = "no"; then
- CONFIGURE_CFLAGS="$CFLAGS_NOPIE $CONFIGURE_CFLAGS"
- CONFIGURE_LDFLAGS="$LDFLAGS_NOPIE $CONFIGURE_LDFLAGS"
+ if compile_prog "-Werror -fno-pie" "-no-pie"; then
+ CONFIGURE_CFLAGS="-fno-pie $CONFIGURE_CFLAGS"
+ CONFIGURE_LDFLAGS="-no-pie $CONFIGURE_LDFLAGS"
+ fi
elif compile_prog "-Werror -fPIE -DPIE" "-pie"; then
CONFIGURE_CFLAGS="-fPIE -DPIE $CONFIGURE_CFLAGS"
CONFIGURE_LDFLAGS="-pie $CONFIGURE_LDFLAGS"
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 17/21] tests/vm: allow running tests in an unconfigured source tree
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (15 preceding siblings ...)
2022-06-16 8:32 ` [PULL 16/21] configure: cleanup -fno-pie detection Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 18/21] build: fix check for -fsanitize-coverage-allowlist Paolo Bonzini
` (4 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow
tests/vm/Makefile.include used to assume that it could run in an unconfigured
source tree, and Cirrus CI relies on that. It was however broken by commit
f4c66f1705 ("tests: use tests/venv to run basevm.py-based scripts", 2022-06-06),
which co-opted the virtual environment being used by avocado tests
to also run the basevm.py tests.
For now, reintroduce the usage of qemu.qmp from the source directory, but
without the sys.path() hacks. The CI configuration can be changed to
install the package via pip when qemu.qmp is removed from the source tree.
Cc: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
tests/vm/Makefile.include | 26 +++++++++++++++++---------
1 file changed, 17 insertions(+), 9 deletions(-)
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 588bc999cc..5f5b1fbfe6 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -1,8 +1,17 @@
# Makefile for VM tests
-.PHONY: vm-build-all vm-clean-all
+# Hack to allow running in an unconfigured build tree
+ifeq ($(wildcard $(SRC_PATH)/config-host.mak),)
+VM_PYTHON = PYTHONPATH=$(SRC_PATH)/python /usr/bin/env python3
+VM_VENV =
+HOST_ARCH := $(shell uname -m)
+else
+VM_PYTHON = $(TESTS_PYTHON)
+VM_VENV = check-venv
+HOST_ARCH = $(ARCH)
+endif
-HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
+.PHONY: vm-build-all vm-clean-all
EFI_AARCH64 = $(wildcard $(BUILD_DIR)/pc-bios/edk2-aarch64-code.fd)
@@ -85,10 +94,10 @@ vm-clean-all:
$(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \
$(SRC_PATH)/tests/vm/basevm.py \
$(SRC_PATH)/tests/vm/Makefile.include \
- check-venv
+ $(VM_VENV)
@mkdir -p $(IMAGES_DIR)
$(call quiet-command, \
- $(TESTS_PYTHON) $< \
+ $(VM_PYTHON) $< \
$(if $(V)$(DEBUG), --debug) \
$(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \
$(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
@@ -100,11 +109,10 @@ $(IMAGES_DIR)/%.img: $(SRC_PATH)/tests/vm/% \
--build-image $@, \
" VM-IMAGE $*")
-
# Build in VM $(IMAGE)
-vm-build-%: $(IMAGES_DIR)/%.img check-venv
+vm-build-%: $(IMAGES_DIR)/%.img $(VM_VENV)
$(call quiet-command, \
- $(TESTS_PYTHON) $(SRC_PATH)/tests/vm/$* \
+ $(VM_PYTHON) $(SRC_PATH)/tests/vm/$* \
$(if $(V)$(DEBUG), --debug) \
$(if $(DEBUG), --interactive) \
$(if $(J),--jobs $(J)) \
@@ -128,9 +136,9 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img
-device virtio-net-pci,netdev=vnet \
|| true
-vm-boot-ssh-%: $(IMAGES_DIR)/%.img check-venv
+vm-boot-ssh-%: $(IMAGES_DIR)/%.img $(VM_VENV)
$(call quiet-command, \
- $(TESTS_PYTHON) $(SRC_PATH)/tests/vm/$* \
+ $(VM_PYTHON) $(SRC_PATH)/tests/vm/$* \
$(if $(J),--jobs $(J)) \
$(if $(V)$(DEBUG), --debug) \
$(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 18/21] build: fix check for -fsanitize-coverage-allowlist
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (16 preceding siblings ...)
2022-06-16 8:32 ` [PULL 17/21] tests/vm: allow running tests in an unconfigured source tree Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 19/21] q35：Enable TSEG only when G_SMRAME and TSEG_EN both enabled Paolo Bonzini
` (3 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel; +Cc: Alexander Bulekov
From: Alexander Bulekov <alxndr@bu.edu>
The existing check has two problems:
1. Meson uses a private directory for the get_supported_arguments check.
./instrumentation-filter does not exist in that private directory (it is
copied into the root of the build-directory).
2. fsanitize-coverage-allowlist is unused when coverage instrumentation
is not configured. No instrumentation are passed for the
get_supported_arguments check
Thus the check always fails. To work around this, change the check to an
"if cc.compiles" check and provide /dev/null, instead of the real
filter.
Meson log:
Working directory: build/meson-private/tmpl6wld2d9
Command line: clang-13 -m64 -mcx16
build/meson-private/tmpl6wld2d9/output.obj -c -O3 -D_FILE_OFFSET_BITS=64
-O0 -Werror=implicit-function-declaration -Werror=unknown-warning-option
-Werror=unused-command-line-argument
-Werror=ignored-optimization-argument
-fsanitize-coverage-allowlist=instrumentation-filter
Error:
error: argument unused during compilation:
'-fsanitize-coverage-allowlist=instrumentation-filter'
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20220614155415.4023833-1-alxndr@bu.edu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
meson.build | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/meson.build b/meson.build
index 21cd949082..fe5d6632fb 100644
--- a/meson.build
+++ b/meson.build
@@ -209,9 +209,13 @@ if get_option('fuzzing')
configure_file(output: 'instrumentation-filter',
input: 'scripts/oss-fuzz/instrumentation-filter-template',
copy: true)
- add_global_arguments(
- cc.get_supported_arguments('-fsanitize-coverage-allowlist=instrumentation-filter'),
- native: false, language: ['c', 'cpp', 'objc'])
+
+ if cc.compiles('int main () { return 0; }',
+ name: '-fsanitize-coverage-allowlist=/dev/null',
+ args: ['-fsanitize-coverage-allowlist=/dev/null'] )
+ add_global_arguments('-fsanitize-coverage-allowlist=instrumentation-filter',
+ native: false, language: ['c', 'cpp', 'objc'])
+ endif
if get_option('fuzzing_engine') == ''
# Add CFLAGS to tell clang to add fuzzer-related instrumentation to all the
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 19/21] q35：Enable TSEG only when G_SMRAME and TSEG_EN both enabled
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (17 preceding siblings ...)
2022-06-16 8:32 ` [PULL 18/21] build: fix check for -fsanitize-coverage-allowlist Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 20/21] meson: put cross compiler info in a separate section Paolo Bonzini
` (2 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel; +Cc: Zhenzhong Duan
From: Zhenzhong Duan <zhenzhong.duan@intel.com>
According to spec:
"TSEG Enable (T_EN): Enabling of SMRAM memory for Extended SMRAM space
only. When G_SMRAME = 1 and TSEG_EN = 1, the TSEG is enabled to appear
in the appropriate physical address space. Note that once D_LCK is set,
this bit becomes read only."
Changed to match the spec description.
Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
Message-Id: <20220615034501.2733802-1-zhenzhong.duan@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hw/pci-host/q35.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index ab5a47aff5..20da121374 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -379,7 +379,8 @@ static void mch_update_smram(MCHPCIState *mch)
memory_region_set_enabled(&mch->high_smram, false);
}
- if (pd->config[MCH_HOST_BRIDGE_ESMRAMC] & MCH_HOST_BRIDGE_ESMRAMC_T_EN) {
+ if ((pd->config[MCH_HOST_BRIDGE_ESMRAMC] & MCH_HOST_BRIDGE_ESMRAMC_T_EN) &&
+ (pd->config[MCH_HOST_BRIDGE_SMRAM] & SMRAM_G_SMRAME)) {
switch (pd->config[MCH_HOST_BRIDGE_ESMRAMC] &
MCH_HOST_BRIDGE_ESMRAMC_TSEG_SZ_MASK) {
case MCH_HOST_BRIDGE_ESMRAMC_TSEG_SZ_1MB:
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 20/21] meson: put cross compiler info in a separate section
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (18 preceding siblings ...)
2022-06-16 8:32 ` [PULL 19/21] q35：Enable TSEG only when G_SMRAME and TSEG_EN both enabled Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 8:32 ` [PULL 21/21] build: include pc-bios/ part in the ROMS variable Paolo Bonzini
2022-06-16 16:14 ` [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Richard Henderson
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel
While at it, remove a dead assignment and simply inline the value of the
"target" variable, which is used just once.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
meson.build | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/meson.build b/meson.build
index fe5d6632fb..0458b69cdf 100644
--- a/meson.build
+++ b/meson.build
@@ -3744,21 +3744,24 @@ endif
summary_info += {'strip binaries': get_option('strip')}
summary_info += {'sparse': sparse}
summary_info += {'mingw32 support': targetos == 'windows'}
+summary(summary_info, bool_yn: true, section: 'Compilation')
# snarf the cross-compilation information for tests
+summary_info = {}
+have_cross = false
foreach target: target_dirs
tcg_mak = meson.current_build_dir() / 'tests/tcg' / 'config-' + target + '.mak'
if fs.exists(tcg_mak)
config_cross_tcg = keyval.load(tcg_mak)
- target = config_cross_tcg['TARGET_NAME']
- compiler = ''
if 'CC' in config_cross_tcg
- summary_info += {target + ' tests': config_cross_tcg['CC']}
+ summary_info += {config_cross_tcg['TARGET_NAME']: config_cross_tcg['CC']}
+ have_cross = true
endif
- endif
+ endif
endforeach
-
-summary(summary_info, bool_yn: true, section: 'Compilation')
+if have_cross
+ summary(summary_info, bool_yn: true, section: 'Cross compilers')
+endif
# Targets and accelerators
summary_info = {}
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 21/21] build: include pc-bios/ part in the ROMS variable
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (19 preceding siblings ...)
2022-06-16 8:32 ` [PULL 20/21] meson: put cross compiler info in a separate section Paolo Bonzini
@ 2022-06-16 8:32 ` Paolo Bonzini
2022-06-16 16:14 ` [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Richard Henderson
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2022-06-16 8:32 UTC (permalink / raw)
To: qemu-devel
Include the full path in TARGET_DIR, so that messages from sub-Makefiles
are clearer. Also, prepare for possibly building firmware outside
pc-bios/ from the Makefile,
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
Makefile | 12 +++++-------
configure | 6 +++---
2 files changed, 8 insertions(+), 10 deletions(-)
diff --git a/Makefile b/Makefile
index 3c0d89057e..ec4445db9a 100644
--- a/Makefile
+++ b/Makefile
@@ -186,16 +186,14 @@ include $(SRC_PATH)/tests/Makefile.include
all: recurse-all
-ROM_DIRS = $(addprefix pc-bios/, $(ROMS))
-ROM_DIRS_RULES=$(foreach t, all clean, $(addsuffix /$(t), $(ROM_DIRS)))
-# Only keep -O and -g cflags
-.PHONY: $(ROM_DIRS_RULES)
-$(ROM_DIRS_RULES):
+ROMS_RULES=$(foreach t, all clean, $(addsuffix /$(t), $(ROMS)))
+.PHONY: $(ROMS_RULES)
+$(ROMS_RULES):
$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $(dir $@) V="$(V)" TARGET_DIR="$(dir $@)" $(notdir $@),)
.PHONY: recurse-all recurse-clean
-recurse-all: $(addsuffix /all, $(ROM_DIRS))
-recurse-clean: $(addsuffix /clean, $(ROM_DIRS))
+recurse-all: $(addsuffix /all, $(ROMS))
+recurse-clean: $(addsuffix /clean, $(ROMS))
######################################################################
diff --git a/configure b/configure
index 9fba134746..a4d61fe504 100755
--- a/configure
+++ b/configure
@@ -2236,7 +2236,7 @@ if test -n "$target_cc" &&
fi
done
if test -n "$ld_i386_emulation"; then
- roms="optionrom"
+ roms="pc-bios/optionrom"
config_mak=pc-bios/optionrom/config.mak
echo "# Automatically generated by configure - do not modify" > $config_mak
echo "TOPSRC_DIR=$source_path" >> $config_mak
@@ -2247,7 +2247,7 @@ fi
probe_target_compilers ppc ppc64
if test -n "$target_cc" && test "$softmmu" = yes; then
- roms="$roms vof"
+ roms="$roms pc-bios/vof"
config_mak=pc-bios/vof/config.mak
echo "# Automatically generated by configure - do not modify" > $config_mak
echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
@@ -2266,7 +2266,7 @@ if test -n "$target_cc" && test "$softmmu" = yes; then
echo "WARNING: Your compiler does not support the z900!"
echo " The s390-ccw bios will only work with guest CPUs >= z10."
fi
- roms="$roms s390-ccw"
+ roms="$roms pc-bios/s390-ccw"
config_mak=pc-bios/s390-ccw/config-host.mak
echo "# Automatically generated by configure - do not modify" > $config_mak
echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
--
2.36.1
^ permalink raw reply related [flat|nested] 23+ messages in thread* Re: [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16
2022-06-16 8:31 [PULL 00/21] Statistics, preconfig and cleanup patches for 2022-06-16 Paolo Bonzini
` (20 preceding siblings ...)
2022-06-16 8:32 ` [PULL 21/21] build: include pc-bios/ part in the ROMS variable Paolo Bonzini
@ 2022-06-16 16:14 ` Richard Henderson
21 siblings, 0 replies; 23+ messages in thread
From: Richard Henderson @ 2022-06-16 16:14 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel
On 6/16/22 01:31, Paolo Bonzini wrote:
> The following changes since commit debd0753663bc89c86f5462a53268f2e3f680f60:
>
> Merge tag 'pull-testing-next-140622-1' of https://github.com/stsquad/qemu into staging (2022-06-13 21:10:57 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 76ca98b0f85222601bd449252ac71df19e0dab29:
>
> build: include pc-bios/ part in the ROMS variable (2022-06-15 11:12:32 +0200)
>
> ----------------------------------------------------------------
> * statistics subsystem
> * virtio reset cleanups
> * build system cleanups
> * fix Cirrus CI
Applied, thanks. Please update https://wiki.qemu.org/ChangeLog/7.1 as appropriate.
r~
>
> ----------------------------------------------------------------
> Alexander Bulekov (1):
> build: fix check for -fsanitize-coverage-allowlist
>
> Mark Kanda (3):
> qmp: Support for querying stats
> kvm: Support for querying fd-based stats
> hmp: add basic "info stats" implementation
>
> Paolo Bonzini (16):
> qmp: add filtering of statistics by target vCPU
> cutils: add functions for IEC and SI prefixes
> qmp: add filtering of statistics by provider
> hmp: add filtering of statistics by provider
> qmp: add filtering of statistics by name
> hmp: add filtering of statistics by name
> block: add more commands to preconfig mode
> s390x: simplify virtio_ccw_reset_virtio
> virtio-mmio: stop ioeventfd on legacy reset
> virtio: stop ioeventfd on reset
> virtio-mmio: cleanup reset
> configure: update list of preserved environment variables
> configure: cleanup -fno-pie detection
> tests/vm: allow running tests in an unconfigured source tree
> meson: put cross compiler info in a separate section
> build: include pc-bios/ part in the ROMS variable
>
> Zhenzhong Duan (1):
> q35：Enable TSEG only when G_SMRAME and TSEG_EN both enabled
>
> Makefile | 12 +-
> accel/kvm/kvm-all.c | 403 ++++++++++++++++++++++++++++++++++++++++++++++
> configure | 22 +--
> hmp-commands-info.hx | 14 ++
> hmp-commands.hx | 14 ++
> hw/pci-host/q35.c | 3 +-
> hw/s390x/virtio-ccw.c | 12 +-
> hw/virtio/virtio-bus.c | 1 +
> hw/virtio/virtio-mmio.c | 18 +--
> hw/virtio/virtio-pci.c | 1 -
> include/monitor/hmp.h | 1 +
> include/monitor/stats.h | 45 ++++++
> include/qemu/cutils.h | 18 +++
> meson.build | 25 +--
> monitor/hmp-cmds.c | 232 ++++++++++++++++++++++++++
> monitor/qmp-cmds.c | 155 ++++++++++++++++++
> qapi/block-core.json | 117 +++++++++-----
> qapi/block-export.json | 21 ++-
> qapi/block.json | 6 +-
> qapi/meson.build | 1 +
> qapi/qapi-schema.json | 1 +
> qapi/stats.json | 249 ++++++++++++++++++++++++++++
> tests/unit/test-cutils.c | 52 ++++++
> tests/vm/Makefile.include | 26 +--
> util/cutils.c | 34 ++--
> 25 files changed, 1368 insertions(+), 115 deletions(-)
> create mode 100644 include/monitor/stats.h
> create mode 100644 qapi/stats.json
^ permalink raw reply [flat|nested] 23+ messages in thread