* [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature
@ 2014-09-30 10:49 Chao Peng
2014-09-30 10:49 ` [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall Chao Peng
` (9 more replies)
0 siblings, 10 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Changes from v17:
- Address comments from Andrew and Jan, including:
* Add per-entry return value for resource_op;
* Refine code for resource_op;
Changes from v16:
- Address comments from Konrad, Andrew and Jan, including:
* Correct copyin/copyout for resource_op.
* Improve documentation/coding style
* Other minor fix.
Changes from v15:
- Keywords change: Intel changed the names for PQOS/CQM in latest SDM.
Adjust the code accordingly:
PQOS(Platform QOS) => PSR(Platform Shared Resource)
CQM(Cache QoS Monitoring) => CMT(Cache Monitoring Technology)
- Make resource operation more clean:
* do_platform_op is the minimum unit for non-preemptible operation, it
accepts Small, non-preempt operations as well as single operation.
* Other preemptible batch operations are performed with multicall mechanism.
- Add padding field in xenpf_resource_data structure and check for that.
Changes from v14:
- Address comments from Jan and Andrew, including:
* Add non-preemption ability to multicall;
* Build the resource batch operation on top of multicall;
* Simplify pqos option.
Changes from v13:
- Address comments from Jan and Andrew, including:
* Support mixed resource types in one invocation;
* Remove some unused fields(rmid_min/rmid_inuse);
* Other minor changes and code clean up;
Changes from v12:
- Address comments from Jan, like QoS feature setting when booting,
avoid unbound memory allocation in Xen, put resource access
hypercall in platform_hypercall.c to avoid creating new files,
specifically enumerate L3 cache size for CQM (instead of
x86_cache_size), get random socket CPU in user space tools, and
also some coding styles. However the continue_hypercall_on_cpu()
suggestion is not adopted in this version due to the potential
issue in our usage case.
- Add a white list to limit the capability for resource access from
tool side.
- Address comments from Ian on the xl/libxl/libxc side.
Changes from v11:
- Turn off pqos and pqos_monitor in Xen command line by default.
- Modify the original specific MSR access hypercall into a generic
resource access hypercall. This hypercall could be used to access
MSR, Port I/O, etc. Use platform_op to replace sysctl so that both
dom0 kernel and userspace could use this hypercall.
- Address various comments from Jan, Ian, Konrad, and Daniel.
Changes from v10:
- Re-design and re-implement the whole logic. In this version,
hypervisor provides basic mechanisms (like access MSRs) while all
policies are put in user space.
patch 1-3 provide a generic MSR hypercall for toolstack to access
patch 4-9 implement the cache QoS monitoring feature
Changes from v9:
- Revise the readonly mapping mechanism to share data between Xen and
userspace. We create L3C buffer for each socket, share both buffer
address MFNs and buffer MFNs to userspace.
- Split the pqos.c into pqos.c and cqm.c for better code structure.
- Show the total L3 cache size when issueing xl pqos-list cqm command.
- Abstract a libxl_getcqminfo() function to fetch cqm data from Xen.
- Several coding style fixes.
Changes from v8:
- Address comments from Ian Campbell, including:
* Modify the return handling for xc_sysctl();
* Add man page items for platform QoS related commands.
* Fix typo in commit message.
Changes from v7:
- Address comments from Andrew Cooper, including:
* Check CQM capability before allocating cpumask memory.
* Move one function declaration into the correct patch.
Changes from v6:
- Address comments from Jan Beulich, including:
* Remove the unnecessary CPUID feature check.
* Remove the unnecessary socket_cpu_map.
* Spin_lock related changes, avoid spin_lock_irqsave().
* Use readonly mapping to pass cqm data between Xen/Userspace,
to avoid data copying.
* Optimize RDMSR/WRMSR logic to avoid unnecessary calls.
* Misc fixes including __read_mostly prefix, return value, etc.
Changes from v5:
- Address comments from Dario Faggioli, including:
* Define a new libxl_cqminfo structure to avoid reference of xc
structure in libxl functions.
* Use LOGE() instead of the LIBXL__LOG() functions.
Changes from v4:
- When comparing xl cqm parameter, use strcmp instead of strncmp,
otherwise, "xl pqos-attach cqmabcd domid" will be considered as
a valid command line.
- Address comments from Andrew Cooper, including:
* Adjust the pqos parameter parsing function.
* Modify the pqos related documentation.
* Add a check for opt_cqm_max_rmid in initialization code.
* Do not IPI CPU that is in same socket with current CPU.
- Address comments from Dario Faggioli, including:
* Fix an typo in export symbols.
* Return correct libxl error code for qos related functions.
* Abstract the error printing logic into a function.
- Address comment from Daniel De Graaf, including:
* Add return value in for pqos related check.
- Address comments from Konrad Rzeszutek Wilk, including:
* Modify the GPLv2 related file header, remove the address.
Changes from v3:
- Use structure to better organize CQM related global variables.
- Address comments from Andrew Cooper, including:
* Remove the domain creation flag for CQM RMID allocation.
* Adjust the boot parameter format, use custom_param().
* Add documentation for the new added boot parameter.
* Change QoS type flag to be uint64_t.
* Initialize the per socket cpu bitmap in system boot time.
* Remove get_cqm_avail() function.
* Misc of format changes.
- Address comment from Daniel De Graaf, including:
* Use avc_current_has_perm() for XEN2__PQOS_OP that belongs to SECCLASS_XEN2.
Changes from v2:
- Address comments from Andrew Cooper, including:
* Merging tools stack changes into one patch.
* Reduce the IPI number to one per socket.
* Change structures for CQM data exchange between tools and Xen.
* Misc of format/variable/function name changes.
- Address comments from Konrad Rzeszutek Wilk, including:
* Simplify the error printing logic.
* Add xsm check for the new added hypercalls.
Changes from v1:
- Address comments from Andrew Cooper, including:
* Change function names, e.g., alloc_cqm_rmid(), system_supports_cqm(), etc.
* Change some structure element order to save packing cost.
* Correct some function's return value.
* Some programming styles change.
* ...
The Intel Xeon processor E5 v3 family introduced resource monitoring capability
in each logical processor to measure specific platform shared resource metrics,
for example, L3 cache occupancy. Detailed information please refer to Intel SDM
chapter 17.14.
Cache Monitoring Technology provides a layer of abstraction between applications
and logical processors through the use of Resource Monitoring IDs (RMIDs).
In Xen design, each guest in the system can be assigned an RMID independently,
while RMID=0 is reserved for monitoring domains that doesn't enable CMT service.
When any of the domain's vcpu is scheduled on a logical processor, the domain's
RMID will be activated by programming the value into one specific MSR, and when
the vcpu is scheduled out, a RMID=0 will be programmed into that MSR.
The CMT Hardware tracks cache utilization of memory accesses according to the
RMIDs and reports monitored data via a counter register. With this solution,
we can get the knowledge how much L3 cache is used by a certain guest.
To attach CMT service to a certain guest:
xl psr-cmt-attach domid
To detached CMT service from a guest:
xl psr-cmt-detach domid
To get the L3 cache usage:
$ xl psr-cmt-show cache_occupancy <domid>
The below data is just an example showing how the CMT related data is exposed to
end user.
[root@localhost]# xl psr-cmt-show cache_occupancy
Total RMID: 55
Per-Socket L3 Cache Size: 35840 KB
Name ID Socket 0 Socket 1
Domain-0 0 20720 KB 15960 KB
ExampleHVMDomain 1 4200 KB 2352 KB
Chao Peng (10):
x86: add generic resource (e.g. MSR) access hypercall
xsm: add resource operation related xsm policy
tools: provide interface for generic resource access
x86: detect and initialize Cache Monitoring Technology feature
x86: dynamically attach/detach CMT service for a guest
x86: collect global CMT information
x86: enable CMT for each domain RMID
x86: add CMT related MSRs in allowed list
xsm: add CMT related xsm policies
tools: CMDs and APIs for Cache Monitoring Technology
docs/man/xl.pod.1 | 25 +++
docs/misc/xen-command-line.markdown | 21 +++
tools/flask/policy/policy/modules/xen/xen.if | 2 +-
tools/flask/policy/policy/modules/xen/xen.te | 6 +-
tools/libxc/Makefile | 2 +
tools/libxc/xc_msr_x86.h | 36 +++++
tools/libxc/xc_private.h | 51 ++++++
tools/libxc/xc_psr.c | 214 ++++++++++++++++++++++++++
tools/libxc/xc_resource.c | 150 ++++++++++++++++++
tools/libxc/xenctrl.h | 34 ++++
tools/libxl/Makefile | 2 +-
tools/libxl/libxl.h | 19 +++
tools/libxl/libxl_psr.c | 184 ++++++++++++++++++++++
tools/libxl/libxl_types.idl | 4 +
tools/libxl/libxl_utils.c | 28 ++++
tools/libxl/xl.h | 3 +
tools/libxl/xl_cmdimpl.c | 132 ++++++++++++++++
tools/libxl/xl_cmdtable.c | 17 ++
xen/arch/x86/Makefile | 1 +
xen/arch/x86/cpu/intel_cacheinfo.c | 49 +-----
xen/arch/x86/domain.c | 8 +
xen/arch/x86/domctl.c | 29 ++++
xen/arch/x86/platform_hypercall.c | 165 ++++++++++++++++++++
xen/arch/x86/psr.c | 187 ++++++++++++++++++++++
xen/arch/x86/setup.c | 1 +
xen/arch/x86/sysctl.c | 43 ++++++
xen/arch/x86/x86_64/platform_hypercall.c | 4 +
xen/include/asm-x86/cpufeature.h | 46 ++++++
xen/include/asm-x86/domain.h | 2 +
xen/include/asm-x86/msr-index.h | 5 +
xen/include/asm-x86/psr.h | 62 ++++++++
xen/include/public/domctl.h | 12 ++
xen/include/public/platform.h | 35 +++++
xen/include/public/sysctl.h | 14 ++
xen/include/xlat.lst | 1 +
xen/xsm/flask/hooks.c | 10 ++
xen/xsm/flask/policy/access_vectors | 18 ++-
xen/xsm/flask/policy/security_classes | 1 +
38 files changed, 1570 insertions(+), 53 deletions(-)
create mode 100644 tools/libxc/xc_msr_x86.h
create mode 100644 tools/libxc/xc_psr.c
create mode 100644 tools/libxc/xc_resource.c
create mode 100644 tools/libxl/libxl_psr.c
create mode 100644 xen/arch/x86/psr.c
create mode 100644 xen/include/asm-x86/psr.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 11:00 ` Andrew Cooper
2014-09-30 12:12 ` Jan Beulich
2014-09-30 10:49 ` [PATCH v18 02/10] xsm: add resource operation related xsm policy Chao Peng
` (8 subsequent siblings)
9 siblings, 2 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Add a generic resource access hypercall for tool stack or other
components, e.g., accessing MSR, port I/O, etc.
The resource is abstracted as a resource address/value pair.
The resource access can be any type of XEN_RESOURCE_OP_*(current
only support MSR and it's white-listed). The resource operations
are always runs on cpu that caller specified. If caller does not
care this, it should use current cpu to eliminate the IPI overhead.
Batch resource operations in one call are also supported but the
max number currently is limited to 2. The operations in a batch are
non-preemptible and execute in their original order. If preemptible
batch is desirable, then multicall mechanism can be used.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
---
xen/arch/x86/platform_hypercall.c | 157 ++++++++++++++++++++++++++++++
xen/arch/x86/x86_64/platform_hypercall.c | 4 +
xen/include/public/platform.h | 35 +++++++
xen/include/xlat.lst | 1 +
4 files changed, 197 insertions(+)
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index 2162811..3d873b5 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -61,6 +61,94 @@ long cpu_down_helper(void *data);
long core_parking_helper(void *data);
uint32_t get_cur_idle_nums(void);
+#define RESOURCE_ACCESS_MAX_ENTRIES 2
+struct xen_resource_access {
+ unsigned int ret;
+ unsigned int nr_entries;
+ xenpf_resource_entry_t *entries;
+};
+
+static bool_t allow_access_msr(unsigned int msr)
+{
+ return 0;
+}
+
+static unsigned int check_resource_access(struct xen_resource_access *ra)
+{
+ xenpf_resource_entry_t *entry;
+ int ret = 0;
+ unsigned int i;
+
+ for ( i = 0; i < ra->nr_entries; i++ )
+ {
+ entry = ra->entries + i;
+
+ if ( entry->rsvd )
+ {
+ entry->u.ret = -EINVAL;
+ break;
+ }
+
+ switch ( entry->u.cmd )
+ {
+ case XEN_RESOURCE_OP_MSR_READ:
+ case XEN_RESOURCE_OP_MSR_WRITE:
+ if ( entry->idx >> 32 )
+ ret = -EINVAL;
+ else if ( !allow_access_msr(entry->idx) )
+ ret = -EACCES;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ if ( ret )
+ {
+ entry->u.ret = ret;
+ break;
+ }
+ }
+
+ /* Return the number of successes. */
+ return i;
+}
+
+static void resource_access(void *info)
+{
+ struct xen_resource_access *ra = info;
+ xenpf_resource_entry_t *entry;
+ int ret;
+ unsigned int i;
+
+ for ( i = 0; i < ra->nr_entries; i++ )
+ {
+ entry = ra->entries + i;
+
+ switch ( entry->u.cmd )
+ {
+ case XEN_RESOURCE_OP_MSR_READ:
+ ret = rdmsr_safe(entry->idx, entry->val);
+ break;
+ case XEN_RESOURCE_OP_MSR_WRITE:
+ ret = wrmsr_safe(entry->idx, entry->val);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ if ( ret )
+ {
+ entry->u.ret = ret;
+ break;
+ }
+ }
+
+ /* Return the number of successes. */
+ ra->ret = i;
+}
+
ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
{
ret_t ret = 0;
@@ -601,6 +689,75 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
}
break;
+ case XENPF_resource_op:
+ {
+ struct xen_resource_access ra;
+ uint32_t cpu;
+ XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
+
+ ra.nr_entries = op->u.resource_op.nr_entries;
+ if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
+ {
+ ret = -EINVAL;
+ break;
+ }
+
+ ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
+ if ( !ra.entries )
+ {
+ ret = -ENOMEM;
+ break;
+ }
+
+ guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
+
+ if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
+ {
+ xfree(ra.entries);
+ ret = -EFAULT;
+ break;
+ }
+
+ /* Do sanity check earlier to omit the potential IPI overhead. */
+ if ( check_resource_access(&ra) < ra.nr_entries )
+ {
+ /* Copy the return value for failed entry. */
+ if ( __copy_to_guest_offset(guest_entries, ret,
+ ra.entries + ret, 1) )
+ ret = -EFAULT;
+ else
+ ret = 0;
+
+ xfree(ra.entries);
+ break;
+ }
+
+ cpu = op->u.resource_op.cpu;
+ if ( cpu == smp_processor_id() )
+ resource_access(&ra);
+ else if ( cpu_online(cpu) )
+ on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
+ else
+ {
+ xfree(ra.entries);
+ ret = -ENODEV;
+ break;
+ }
+
+ /* Copy all if succeeded or up to the failed entry. */
+ if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
+ min(ra.nr_entries, ra.ret + 1)) )
+ {
+ xfree(ra.entries);
+ ret = -EFAULT;
+ break;
+ }
+
+ ret = ra.ret;
+ xfree(ra.entries);
+ }
+ break;
+
default:
ret = -ENOSYS;
break;
diff --git a/xen/arch/x86/x86_64/platform_hypercall.c b/xen/arch/x86/x86_64/platform_hypercall.c
index b6f380e..ccfd30d 100644
--- a/xen/arch/x86/x86_64/platform_hypercall.c
+++ b/xen/arch/x86/x86_64/platform_hypercall.c
@@ -32,6 +32,10 @@ CHECK_pf_pcpu_version;
CHECK_pf_enter_acpi_sleep;
#undef xen_pf_enter_acpi_sleep
+#define xen_pf_resource_entry xenpf_resource_entry
+CHECK_pf_resource_entry;
+#undef xen_pf_resource_entry
+
#define COMPAT
#define _XEN_GUEST_HANDLE(t) XEN_GUEST_HANDLE(t)
#define _XEN_GUEST_HANDLE_PARAM(t) XEN_GUEST_HANDLE_PARAM(t)
diff --git a/xen/include/public/platform.h b/xen/include/public/platform.h
index 053b9fa..abf916c 100644
--- a/xen/include/public/platform.h
+++ b/xen/include/public/platform.h
@@ -528,6 +528,40 @@ typedef struct xenpf_core_parking xenpf_core_parking_t;
DEFINE_XEN_GUEST_HANDLE(xenpf_core_parking_t);
/*
+ * Access generic platform resources(e.g., accessing MSR, port I/O, etc)
+ * in unified way. Batch resource operations in one call are supported and
+ * thay are always non-preemptible and executed in their original order.
+ * The batch itself returns a negative integer for general errors, or a
+ * non-negative integer for the number of successful operations. For latter
+ * case, the @ret in the failed entry(if have) indicates the exact error and
+ * it's meaningful only when it has a negative value.
+ */
+#define XENPF_resource_op 61
+
+#define XEN_RESOURCE_OP_MSR_READ 0
+#define XEN_RESOURCE_OP_MSR_WRITE 1
+
+struct xenpf_resource_entry {
+ union {
+ uint32_t cmd; /* IN: XEN_RESOURCE_OP_* */
+ int32_t ret; /* OUT: return value for this entry */
+ } u;
+ uint32_t rsvd; /* IN: padding and must be zero */
+ uint64_t idx; /* IN: resource address to access */
+ uint64_t val; /* IN/OUT: resource value to set/get */
+};
+typedef struct xenpf_resource_entry xenpf_resource_entry_t;
+DEFINE_XEN_GUEST_HANDLE(xenpf_resource_entry_t);
+
+struct xenpf_resource_op {
+ uint32_t nr_entries; /* number of resource entry */
+ uint32_t cpu; /* which cpu to run */
+ XEN_GUEST_HANDLE(xenpf_resource_entry_t) entries;
+};
+typedef struct xenpf_resource_op xenpf_resource_op_t;
+DEFINE_XEN_GUEST_HANDLE(xenpf_resource_op_t);
+
+/*
* ` enum neg_errnoval
* ` HYPERVISOR_platform_op(const struct xen_platform_op*);
*/
@@ -553,6 +587,7 @@ struct xen_platform_op {
struct xenpf_cpu_hotadd cpu_add;
struct xenpf_mem_hotadd mem_add;
struct xenpf_core_parking core_parking;
+ struct xenpf_resource_op resource_op;
uint8_t pad[128];
} u;
};
diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst
index 9a35dd7..234b668 100644
--- a/xen/include/xlat.lst
+++ b/xen/include/xlat.lst
@@ -88,6 +88,7 @@
? xenpf_enter_acpi_sleep platform.h
? xenpf_pcpuinfo platform.h
? xenpf_pcpu_version platform.h
+? xenpf_resource_entry platform.h
! sched_poll sched.h
? sched_remote_shutdown sched.h
? sched_shutdown sched.h
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 02/10] xsm: add resource operation related xsm policy
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
2014-09-30 10:49 ` [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 03/10] tools: provide interface for generic resource access Chao Peng
` (7 subsequent siblings)
9 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Add xsm policies for resource access related hypercall, such as MSR
access, port I/O read/write, and other related resource operations.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
tools/flask/policy/policy/modules/xen/xen.te | 3 +++
xen/xsm/flask/hooks.c | 4 ++++
xen/xsm/flask/policy/access_vectors | 14 +++++++++++---
xen/xsm/flask/policy/security_classes | 1 +
4 files changed, 19 insertions(+), 3 deletions(-)
diff --git a/tools/flask/policy/policy/modules/xen/xen.te b/tools/flask/policy/policy/modules/xen/xen.te
index 1937883..6cecf97 100644
--- a/tools/flask/policy/policy/modules/xen/xen.te
+++ b/tools/flask/policy/policy/modules/xen/xen.te
@@ -64,6 +64,9 @@ allow dom0_t xen_t:xen {
getidle debug getcpuinfo heap pm_op mca_op lockprof cpupool_op tmem_op
tmem_control getscheduler setscheduler
};
+allow dom0_t xen_t:xen2 {
+ resource_op
+};
allow dom0_t xen_t:mmu memorymap;
# Allow dom0 to use these domctls on itself. For domctls acting on other
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index df05566..9f36503 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1391,6 +1391,10 @@ static int flask_platform_op(uint32_t op)
case XENPF_get_cpuinfo:
return domain_has_xen(current->domain, XEN__GETCPUINFO);
+ case XENPF_resource_op:
+ return avc_current_has_perm(SECINITSID_XEN, SECCLASS_XEN2,
+ XEN2__RESOURCE_OP, NULL);
+
default:
printk("flask_platform_op: Unknown op %d\n", op);
return -EPERM;
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index d279841..daf0de5 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -3,9 +3,9 @@
#
# class class_name { permission_name ... }
-# Class xen consists of dom0-only operations dealing with the hypervisor itself.
-# Unless otherwise specified, the source is the domain executing the hypercall,
-# and the target is the xen initial sid (type xen_t).
+# Class xen and xen2 consists of dom0-only operations dealing with the
+# hypervisor itself. Unless otherwise specified, the source is the domain
+# executing the hypercall, and the target is the xen initial sid (type xen_t).
class xen
{
# XENPF_settime
@@ -75,6 +75,14 @@ class xen
setscheduler
}
+# This is a continuation of class xen, since only 32 permissions can be
+# defined per class
+class xen2
+{
+# XENPF_resource_op
+ resource_op
+}
+
# Classes domain and domain2 consist of operations that a domain performs on
# another domain or on itself. Unless otherwise specified, the source is the
# domain executing the hypercall, and the target is the domain being operated on
diff --git a/xen/xsm/flask/policy/security_classes b/xen/xsm/flask/policy/security_classes
index ef134a7..ca191db 100644
--- a/xen/xsm/flask/policy/security_classes
+++ b/xen/xsm/flask/policy/security_classes
@@ -8,6 +8,7 @@
# for userspace object managers
class xen
+class xen2
class domain
class domain2
class hvm
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 03/10] tools: provide interface for generic resource access
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
2014-09-30 10:49 ` [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall Chao Peng
2014-09-30 10:49 ` [PATCH v18 02/10] xsm: add resource operation related xsm policy Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature Chao Peng
` (6 subsequent siblings)
9 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Xen added a new platform_op hypercall for generic MSR access, and this
is the the tool side change to wrapper the hypercall into xc APIs.
For non-preemptible batch resource operations, group them in entries of
xc_resource_op structure. For preemptible ones, use multiple
xc_resource_op structure instead.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxc/Makefile | 1 +
tools/libxc/xc_private.h | 51 +++++++++++++++
tools/libxc/xc_resource.c | 150 +++++++++++++++++++++++++++++++++++++++++++++
tools/libxc/xenctrl.h | 17 +++++
4 files changed, 219 insertions(+)
create mode 100644 tools/libxc/xc_resource.c
diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index 3b04027..dde6109 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -34,6 +34,7 @@ CTRL_SRCS-y += xc_foreign_memory.c
CTRL_SRCS-y += xc_kexec.c
CTRL_SRCS-y += xtl_core.c
CTRL_SRCS-y += xtl_logger_stdio.c
+CTRL_SRCS-y += xc_resource.c
CTRL_SRCS-$(CONFIG_X86) += xc_pagetab.c
CTRL_SRCS-$(CONFIG_Linux) += xc_linux.c xc_linux_osdep.c
CTRL_SRCS-$(CONFIG_FreeBSD) += xc_freebsd.c xc_freebsd_osdep.c
diff --git a/tools/libxc/xc_private.h b/tools/libxc/xc_private.h
index 94df688..3fd5324 100644
--- a/tools/libxc/xc_private.h
+++ b/tools/libxc/xc_private.h
@@ -46,6 +46,7 @@
#define DECLARE_SYSCTL struct xen_sysctl sysctl
#define DECLARE_PHYSDEV_OP struct physdev_op physdev_op
#define DECLARE_FLASK_OP struct xen_flask_op op
+#define DECLARE_PLATFORM_OP struct xen_platform_op platform_op
#undef PAGE_SHIFT
#undef PAGE_SIZE
@@ -310,6 +311,56 @@ static inline int do_sysctl(xc_interface *xch, struct xen_sysctl *sysctl)
return ret;
}
+static inline int do_platform_op(xc_interface *xch,
+ struct xen_platform_op *platform_op)
+{
+ int ret = -1;
+ DECLARE_HYPERCALL;
+ DECLARE_HYPERCALL_BOUNCE(platform_op, sizeof(*platform_op),
+ XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
+
+ platform_op->interface_version = XENPF_INTERFACE_VERSION;
+
+ if ( xc_hypercall_bounce_pre(xch, platform_op) )
+ {
+ PERROR("Could not bounce buffer for platform_op hypercall");
+ return -1;
+ }
+
+ hypercall.op = __HYPERVISOR_platform_op;
+ hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(platform_op);
+ if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
+ {
+ if ( errno == EACCES )
+ DPRINTF("platform operation failed -- need to"
+ " rebuild the user-space tool set?\n");
+ }
+
+ xc_hypercall_bounce_post(xch, platform_op);
+ return ret;
+}
+
+static inline int do_multicall_op(xc_interface *xch,
+ xc_hypercall_buffer_t *call_list,
+ uint32_t nr_calls)
+{
+ int ret = -1;
+ DECLARE_HYPERCALL;
+ DECLARE_HYPERCALL_BUFFER_ARGUMENT(call_list);
+
+ hypercall.op = __HYPERVISOR_multicall;
+ hypercall.arg[0] = HYPERCALL_BUFFER_AS_ARG(call_list);
+ hypercall.arg[1] = nr_calls;
+ if ( (ret = do_xen_hypercall(xch, &hypercall)) < 0 )
+ {
+ if ( errno == EACCES )
+ DPRINTF("multicall operation failed -- need to"
+ " rebuild the user-space tool set?\n");
+ }
+
+ return ret;
+}
+
int do_memory_op(xc_interface *xch, int cmd, void *arg, size_t len);
void *xc_map_foreign_ranges(xc_interface *xch, uint32_t dom,
diff --git a/tools/libxc/xc_resource.c b/tools/libxc/xc_resource.c
new file mode 100644
index 0000000..3abadbd
--- /dev/null
+++ b/tools/libxc/xc_resource.c
@@ -0,0 +1,150 @@
+/*
+ * xc_resource.c
+ *
+ * Generic resource access API
+ *
+ * Copyright (C) 2014 Intel Corporation
+ * Author Dongxiao Xu <dongxiao.xu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "xc_private.h"
+
+static int xc_resource_op_one(xc_interface *xch, xc_resource_op_t *op)
+{
+ int rc;
+ DECLARE_PLATFORM_OP;
+ DECLARE_NAMED_HYPERCALL_BOUNCE(entries, op->entries,
+ op->nr_entries * sizeof(*op->entries),
+ XC_HYPERCALL_BUFFER_BOUNCE_BOTH);
+
+ if ( xc_hypercall_bounce_pre(xch, entries) )
+ return -1;
+
+ platform_op.cmd = XENPF_resource_op;
+ platform_op.u.resource_op.nr_entries = op->nr_entries;
+ platform_op.u.resource_op.cpu = op->cpu;
+ set_xen_guest_handle(platform_op.u.resource_op.entries, entries);
+
+ rc = do_platform_op(xch, &platform_op);
+ op->result = rc;
+
+ xc_hypercall_bounce_post(xch, entries);
+
+ return rc;
+}
+
+static int xc_resource_op_multi(xc_interface *xch, uint32_t nr_ops, xc_resource_op_t *ops)
+{
+ int rc, i, entries_size;
+ xc_resource_op_t *op;
+ multicall_entry_t *call;
+ DECLARE_HYPERCALL_BUFFER(multicall_entry_t, call_list);
+ xc_hypercall_buffer_array_t *platform_ops, *entries_list = NULL;
+
+ call_list = xc_hypercall_buffer_alloc(xch, call_list,
+ sizeof(*call_list) * nr_ops);
+ if ( !call_list )
+ return -1;
+
+ platform_ops = xc_hypercall_buffer_array_create(xch, nr_ops);
+ if ( !platform_ops )
+ {
+ rc = -1;
+ goto out;
+ }
+
+ entries_list = xc_hypercall_buffer_array_create(xch, nr_ops);
+ if ( !entries_list )
+ {
+ rc = -1;
+ goto out;
+ }
+
+ for ( i = 0; i < nr_ops; i++ )
+ {
+ DECLARE_HYPERCALL_BUFFER(xen_platform_op_t, platform_op);
+ DECLARE_HYPERCALL_BUFFER(xc_resource_entry_t, entries);
+
+ op = ops + i;
+
+ platform_op = xc_hypercall_buffer_array_alloc(xch, platform_ops, i,
+ platform_op, sizeof(xen_platform_op_t));
+ if ( !platform_op )
+ {
+ rc = -1;
+ goto out;
+ }
+
+ entries_size = sizeof(xc_resource_entry_t) * op->nr_entries;
+ entries = xc_hypercall_buffer_array_alloc(xch, entries_list, i,
+ entries, entries_size);
+ if ( !entries)
+ {
+ rc = -1;
+ goto out;
+ }
+ memcpy(entries, op->entries, entries_size);
+
+ call = call_list + i;
+ call->op = __HYPERVISOR_platform_op;
+ call->args[0] = HYPERCALL_BUFFER_AS_ARG(platform_op);
+
+ platform_op->interface_version = XENPF_INTERFACE_VERSION;
+ platform_op->cmd = XENPF_resource_op;
+ platform_op->u.resource_op.cpu = op->cpu;
+ platform_op->u.resource_op.nr_entries = op->nr_entries;
+ set_xen_guest_handle(platform_op->u.resource_op.entries, entries);
+ }
+
+ rc = do_multicall_op(xch, HYPERCALL_BUFFER(call_list), nr_ops);
+
+ for ( i = 0; i < nr_ops; i++ )
+ {
+ DECLARE_HYPERCALL_BUFFER(xc_resource_entry_t, entries);
+ op = ops + i;
+
+ call = call_list + i;
+ op->result = call->result;
+
+ entries_size = sizeof(xc_resource_entry_t) * op->nr_entries;
+ entries = xc_hypercall_buffer_array_get(xch, entries_list, i,
+ entries, entries_size);
+ memcpy(op->entries, entries, entries_size);
+ }
+
+out:
+ xc_hypercall_buffer_array_destroy(xch, entries_list);
+ xc_hypercall_buffer_array_destroy(xch, platform_ops);
+ xc_hypercall_buffer_free(xch, call_list);
+ return rc;
+}
+
+int xc_resource_op(xc_interface *xch, uint32_t nr_ops, xc_resource_op_t *ops)
+{
+ if ( nr_ops == 1 )
+ return xc_resource_op_one(xch, ops);
+ else if ( nr_ops > 1 )
+ return xc_resource_op_multi(xch, nr_ops, ops);
+ else
+ return -1;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index 514b241..6edb738 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -47,6 +47,7 @@
#include <xen/xsm/flask_op.h>
#include <xen/tmem.h>
#include <xen/kexec.h>
+#include <xen/platform.h>
#include "xentoollog.h"
@@ -2655,6 +2656,22 @@ int xc_kexec_load(xc_interface *xch, uint8_t type, uint16_t arch,
*/
int xc_kexec_unload(xc_interface *xch, int type);
+typedef xenpf_resource_entry_t xc_resource_entry_t;
+
+/*
+ * Generic resource operation which contains multiple non-preemptible
+ * resource access entries that passed to xc_resource_op().
+ */
+struct xc_resource_op {
+ uint64_t result; /* on return, check this field first */
+ uint32_t cpu; /* which cpu to run */
+ uint32_t nr_entries; /* number of resource entries */
+ xc_resource_entry_t *entries;
+};
+
+typedef struct xc_resource_op xc_resource_op_t;
+int xc_resource_op(xc_interface *xch, uint32_t nr_ops, xc_resource_op_t *ops);
+
#endif /* XENCTRL_H */
/*
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (2 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 03/10] tools: provide interface for generic resource access Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 13:13 ` Jan Beulich
2014-09-30 10:49 ` [PATCH v18 05/10] x86: dynamically attach/detach CMT service for a guest Chao Peng
` (5 subsequent siblings)
9 siblings, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Detect Cache Monitoring Technology(CMT) feature and enumerate the
resource types, one of which is to monitor the L3 cache occupancy.
Also introduce a Xen command line parameter to control the Platform
Shared Resource such as CMT.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
docs/misc/xen-command-line.markdown | 21 +++++++
xen/arch/x86/Makefile | 1 +
xen/arch/x86/psr.c | 114 +++++++++++++++++++++++++++++++++++
xen/arch/x86/setup.c | 1 +
xen/include/asm-x86/cpufeature.h | 1 +
xen/include/asm-x86/psr.h | 51 ++++++++++++++++
6 files changed, 189 insertions(+)
create mode 100644 xen/arch/x86/psr.c
create mode 100644 xen/include/asm-x86/psr.h
diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index af93e17..5514431 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1005,6 +1005,27 @@ This option can be specified more than once (up to 8 times at present).
### ple\_window
> `= <integer>`
+### psr (Intel)
+> `= List of ( cmt:<boolean> | rmid_max:<integer> )`
+
+> Default: `psr=cmt:0,rmid_max:255`
+
+Platform Shared Resource(PSR) Services. Intel Haswell and later server
+platforms offer information about the sharing of resources.
+
+To use the PSR monitoring service for a certain domain, a Resource
+Monitoring ID(RMID) is used to bind the domain to corresponding shared
+resource. RMID is a hardware-provided layer of abstraction between software
+and logical processors.
+
+The following resources are available:
+* Cache Monitoring Technology (Haswell and later). Information
+regarding the L3 cache occupancy.
+
+`cmt` instructs Xen to enable/disable Cache Monitoring Technology.
+
+`rmid_max` indicates the max value for rmid.
+
### reboot
> `= t[riple] | k[bd] | a[cpi] | p[ci] | n[o] [, [w]arm | [c]old]`
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index c1e244d..86ca5f8 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -42,6 +42,7 @@ obj-y += numa.o
obj-y += pci.o
obj-y += percpu.o
obj-y += physdev.o
+obj-y += psr.o
obj-y += setup.o
obj-y += shutdown.o
obj-y += smp.o
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
new file mode 100644
index 0000000..a4cbc0a
--- /dev/null
+++ b/xen/arch/x86/psr.c
@@ -0,0 +1,114 @@
+/*
+ * pqos.c: Platform Shared Resource related service for guest.
+ *
+ * Copyright (c) 2014, Intel Corporation
+ * Author: Dongxiao Xu <dongxiao.xu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#include <xen/init.h>
+#include <xen/cpu.h>
+#include <asm/psr.h>
+
+#define PSR_CMT (1<<0)
+
+struct psr_cmt *__read_mostly psr_cmt;
+static bool_t __initdata opt_psr;
+static unsigned int __initdata opt_rmid_max = 255;
+
+static void __init parse_psr_param(char *s)
+{
+ char *ss, *val_str;
+
+ do {
+ ss = strchr(s, ',');
+ if ( ss )
+ *ss = '\0';
+
+ val_str = strchr(s, ':');
+ if ( val_str )
+ *val_str++ = '\0';
+
+ if ( !strcmp(s, "cmt") && ( !val_str || parse_bool(val_str) == 1 ) )
+ opt_psr |= PSR_CMT;
+ else if ( val_str && !strcmp(s, "rmid_max") )
+ opt_rmid_max = simple_strtoul(val_str, NULL, 0);
+
+ s = ss + 1;
+ } while ( ss );
+}
+custom_param("psr", parse_psr_param);
+
+static void __init init_psr_cmt(unsigned int rmid_max)
+{
+ unsigned int eax, ebx, ecx, edx;
+ unsigned int rmid;
+
+ if ( !boot_cpu_has(X86_FEATURE_CMT) )
+ return;
+
+ cpuid_count(0xf, 0, &eax, &ebx, &ecx, &edx);
+ if ( !edx )
+ return;
+
+ psr_cmt = xzalloc(struct psr_cmt);
+ if ( !psr_cmt )
+ return;
+
+ psr_cmt->features = edx;
+ psr_cmt->rmid_mask = ~(~0ull << get_count_order(ebx));
+ psr_cmt->rmid_max = min(rmid_max, ebx);
+
+ if ( psr_cmt->features & PSR_RESOURCE_TYPE_L3 )
+ {
+ cpuid_count(0xf, 1, &eax, &ebx, &ecx, &edx);
+ psr_cmt->l3.upscaling_factor = ebx;
+ psr_cmt->l3.rmid_max = ecx;
+ psr_cmt->l3.features = edx;
+ }
+
+ psr_cmt->rmid_max = min(rmid_max, psr_cmt->l3.rmid_max);
+ psr_cmt->rmid_to_dom = xmalloc_array(domid_t, psr_cmt->rmid_max + 1);
+ if ( !psr_cmt->rmid_to_dom )
+ {
+ xfree(psr_cmt);
+ psr_cmt = NULL;
+ return;
+ }
+
+ /*
+ * Once CMT is enabled each CPU will always require a RMID to associate
+ * with it. To reduce the waste of RMID, reserve RMID 0 for all CPUs that
+ * have no domain being monitored.
+ */
+ psr_cmt->rmid_to_dom[0] = DOMID_XEN;
+ for ( rmid = 1; rmid <= psr_cmt->rmid_max; rmid++ )
+ psr_cmt->rmid_to_dom[rmid] = DOMID_INVALID;
+
+ printk(XENLOG_INFO "Cache Monitoring Technology enabled\n");
+}
+
+static int __init init_psr(void)
+{
+ if ( opt_psr & PSR_CMT && opt_rmid_max )
+ init_psr_cmt(opt_rmid_max);
+ return 0;
+}
+__initcall(init_psr);
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 8c8b91f..3856fcc 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -49,6 +49,7 @@
#include <xen/cpu.h>
#include <asm/nmi.h>
#include <asm/alternative.h>
+#include <asm/psr.h>
/* opt_nosmp: If true, secondary processors are ignored. */
static bool_t __initdata opt_nosmp;
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 8014241..137d75c 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -148,6 +148,7 @@
#define X86_FEATURE_ERMS (7*32+ 9) /* Enhanced REP MOVSB/STOSB */
#define X86_FEATURE_INVPCID (7*32+10) /* Invalidate Process Context ID */
#define X86_FEATURE_RTM (7*32+11) /* Restricted Transactional Memory */
+#define X86_FEATURE_CMT (7*32+12) /* Cache Monitoring Technology */
#define X86_FEATURE_NO_FPU_SEL (7*32+13) /* FPU CS/DS stored as zero */
#define X86_FEATURE_MPX (7*32+14) /* Memory Protection Extensions */
#define X86_FEATURE_RDSEED (7*32+18) /* RDSEED instruction */
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
new file mode 100644
index 0000000..8a3803c
--- /dev/null
+++ b/xen/include/asm-x86/psr.h
@@ -0,0 +1,51 @@
+/*
+ * psr.h: Platform Shared Resource related service for guest.
+ *
+ * Copyright (c) 2014, Intel Corporation
+ * Author: Dongxiao Xu <dongxiao.xu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#ifndef __ASM_PSR_H__
+#define __ASM_PSR_H__
+
+/* Resource Type Enumeration */
+#define PSR_RESOURCE_TYPE_L3 0x2
+
+/* L3 Monitoring Features */
+#define PSR_CMT_L3_OCCUPANCY 0x1
+
+struct psr_cmt_l3 {
+ unsigned int features;
+ unsigned int upscaling_factor;
+ unsigned int rmid_max;
+};
+
+struct psr_cmt {
+ unsigned long rmid_mask;
+ unsigned int rmid_max;
+ unsigned int features;
+ domid_t *rmid_to_dom;
+ struct psr_cmt_l3 l3;
+};
+
+extern struct psr_cmt *psr_cmt;
+
+#endif /* __ASM_PSR_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 05/10] x86: dynamically attach/detach CMT service for a guest
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (3 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 06/10] x86: collect global CMT information Chao Peng
` (4 subsequent siblings)
9 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Add hypervisor side support for dynamically attach and detach
Cache Monitoring Technology(CMT) services for a certain guest.
When attach CMT service for a guest, system will allocate an
RMID for it. When detach or guest is shutdown, the RMID will be
recycled for future use.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/domain.c | 3 +++
xen/arch/x86/domctl.c | 29 ++++++++++++++++++++++++++
xen/arch/x86/psr.c | 46 ++++++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/domain.h | 2 ++
xen/include/asm-x86/psr.h | 10 +++++++++
xen/include/public/domctl.h | 12 +++++++++++
6 files changed, 102 insertions(+)
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 7b1dfe6..3cfd8f4 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -60,6 +60,7 @@
#include <xen/numa.h>
#include <xen/iommu.h>
#include <compat/vcpu.h>
+#include <asm/psr.h>
DEFINE_PER_CPU(struct vcpu *, curr_vcpu);
DEFINE_PER_CPU(unsigned long, cr4);
@@ -647,6 +648,8 @@ void arch_domain_destroy(struct domain *d)
free_xenheap_page(d->shared_info);
cleanup_domain_irq_mapping(d);
+
+ psr_free_rmid(d);
}
unsigned long pv_guest_cr4_fixup(const struct vcpu *v, unsigned long guest_cr4)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 7a5de43..6ed480a 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -35,6 +35,7 @@
#include <asm/mem_sharing.h>
#include <asm/xstate.h>
#include <asm/debugger.h>
+#include <asm/psr.h>
static int gdbsx_guest_mem_io(
domid_t domid, struct xen_domctl_gdbsx_memio *iop)
@@ -1319,6 +1320,34 @@ long arch_do_domctl(
}
break;
+ case XEN_DOMCTL_psr_cmt_op:
+ if ( !psr_cmt_enabled() )
+ {
+ ret = -ENODEV;
+ break;
+ }
+
+ switch ( domctl->u.psr_cmt_op.cmd )
+ {
+ case XEN_DOMCTL_PSR_CMT_OP_ATTACH:
+ ret = psr_alloc_rmid(d);
+ break;
+ case XEN_DOMCTL_PSR_CMT_OP_DETACH:
+ if ( d->arch.psr_rmid > 0 )
+ psr_free_rmid(d);
+ else
+ ret = -ENOENT;
+ break;
+ case XEN_DOMCTL_PSR_CMT_OP_QUERY_RMID:
+ domctl->u.psr_cmt_op.data = d->arch.psr_rmid;
+ copyback = 1;
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ break;
+
default:
ret = iommu_do_domctl(domctl, d, u_domctl);
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index a4cbc0a..4b4a4cf 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -15,6 +15,7 @@
*/
#include <xen/init.h>
#include <xen/cpu.h>
+#include <xen/sched.h>
#include <asm/psr.h>
#define PSR_CMT (1<<0)
@@ -103,6 +104,51 @@ static int __init init_psr(void)
}
__initcall(init_psr);
+/* Called with domain lock held, no psr specific lock needed */
+int psr_alloc_rmid(struct domain *d)
+{
+ unsigned int rmid;
+
+ ASSERT(psr_cmt_enabled());
+
+ if ( d->arch.psr_rmid > 0 )
+ return -EEXIST;
+
+ for ( rmid = 1; rmid <= psr_cmt->rmid_max; rmid++ )
+ {
+ if ( psr_cmt->rmid_to_dom[rmid] != DOMID_INVALID )
+ continue;
+
+ psr_cmt->rmid_to_dom[rmid] = d->domain_id;
+ break;
+ }
+
+ /* No RMID available, assign RMID=0 by default. */
+ if ( rmid > psr_cmt->rmid_max )
+ {
+ d->arch.psr_rmid = 0;
+ return -EUSERS;
+ }
+
+ d->arch.psr_rmid = rmid;
+
+ return 0;
+}
+
+/* Called with domain lock held, no psr specific lock needed */
+void psr_free_rmid(struct domain *d)
+{
+ unsigned int rmid;
+
+ rmid = d->arch.psr_rmid;
+ /* We do not free system reserved "RMID=0". */
+ if ( rmid == 0 )
+ return;
+
+ psr_cmt->rmid_to_dom[rmid] = DOMID_INVALID;
+ d->arch.psr_rmid = 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 7abe1b3..2be1d1e 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -314,6 +314,8 @@ struct arch_domain
/* Shared page for notifying that explicit PIRQ EOI is required. */
unsigned long *pirq_eoi_map;
unsigned long pirq_eoi_map_mfn;
+
+ unsigned int psr_rmid; /* RMID assigned to the domain for CMT */
} __cacheline_aligned;
#define has_arch_pdevs(d) (!list_empty(&(d)->arch.pdev_list))
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 8a3803c..98cfa6d 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -16,6 +16,8 @@
#ifndef __ASM_PSR_H__
#define __ASM_PSR_H__
+#include <xen/types.h>
+
/* Resource Type Enumeration */
#define PSR_RESOURCE_TYPE_L3 0x2
@@ -38,6 +40,14 @@ struct psr_cmt {
extern struct psr_cmt *psr_cmt;
+static inline bool_t psr_cmt_enabled(void)
+{
+ return !!psr_cmt;
+}
+
+int psr_alloc_rmid(struct domain *d);
+void psr_free_rmid(struct domain *d);
+
#endif /* __ASM_PSR_H__ */
/*
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index cfa39b3..59220ed 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -965,6 +965,16 @@ struct xen_domctl_vnuma {
typedef struct xen_domctl_vnuma xen_domctl_vnuma_t;
DEFINE_XEN_GUEST_HANDLE(xen_domctl_vnuma_t);
+struct xen_domctl_psr_cmt_op {
+#define XEN_DOMCTL_PSR_CMT_OP_DETACH 0
+#define XEN_DOMCTL_PSR_CMT_OP_ATTACH 1
+#define XEN_DOMCTL_PSR_CMT_OP_QUERY_RMID 2
+ uint32_t cmd;
+ uint32_t data;
+};
+typedef struct xen_domctl_psr_cmt_op xen_domctl_psr_cmt_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_cmt_op_t);
+
struct xen_domctl {
uint32_t cmd;
#define XEN_DOMCTL_createdomain 1
@@ -1038,6 +1048,7 @@ struct xen_domctl {
#define XEN_DOMCTL_get_vcpu_msrs 72
#define XEN_DOMCTL_set_vcpu_msrs 73
#define XEN_DOMCTL_setvnumainfo 74
+#define XEN_DOMCTL_psr_cmt_op 75
#define XEN_DOMCTL_gdbsx_guestmemio 1000
#define XEN_DOMCTL_gdbsx_pausevcpu 1001
#define XEN_DOMCTL_gdbsx_unpausevcpu 1002
@@ -1099,6 +1110,7 @@ struct xen_domctl {
struct xen_domctl_gdbsx_pauseunp_vcpu gdbsx_pauseunp_vcpu;
struct xen_domctl_gdbsx_domstatus gdbsx_domstatus;
struct xen_domctl_vnuma vnuma;
+ struct xen_domctl_psr_cmt_op psr_cmt_op;
uint8_t pad[128];
} u;
};
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 06/10] x86: collect global CMT information
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (4 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 05/10] x86: dynamically attach/detach CMT service for a guest Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 13:20 ` Jan Beulich
2014-09-30 10:49 ` [PATCH v18 07/10] x86: enable CMT for each domain RMID Chao Peng
` (3 subsequent siblings)
9 siblings, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
This implementation tries to put all policies into user space, thus some
global CMT information needs to be exposed, such as the total RMID count,
L3 upscaling factor, etc.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/cpu/intel_cacheinfo.c | 49 ++----------------------------------
xen/arch/x86/sysctl.c | 43 +++++++++++++++++++++++++++++++
xen/include/asm-x86/cpufeature.h | 45 +++++++++++++++++++++++++++++++++
xen/include/public/sysctl.h | 14 +++++++++++
4 files changed, 104 insertions(+), 47 deletions(-)
diff --git a/xen/arch/x86/cpu/intel_cacheinfo.c b/xen/arch/x86/cpu/intel_cacheinfo.c
index 430f939..48970c0 100644
--- a/xen/arch/x86/cpu/intel_cacheinfo.c
+++ b/xen/arch/x86/cpu/intel_cacheinfo.c
@@ -81,54 +81,9 @@ static struct _cache_table cache_table[] __cpuinitdata =
{ 0x00, 0, 0}
};
-
-enum _cache_type
-{
- CACHE_TYPE_NULL = 0,
- CACHE_TYPE_DATA = 1,
- CACHE_TYPE_INST = 2,
- CACHE_TYPE_UNIFIED = 3
-};
-
-union _cpuid4_leaf_eax {
- struct {
- enum _cache_type type:5;
- unsigned int level:3;
- unsigned int is_self_initializing:1;
- unsigned int is_fully_associative:1;
- unsigned int reserved:4;
- unsigned int num_threads_sharing:12;
- unsigned int num_cores_on_die:6;
- } split;
- u32 full;
-};
-
-union _cpuid4_leaf_ebx {
- struct {
- unsigned int coherency_line_size:12;
- unsigned int physical_line_partition:10;
- unsigned int ways_of_associativity:10;
- } split;
- u32 full;
-};
-
-union _cpuid4_leaf_ecx {
- struct {
- unsigned int number_of_sets:32;
- } split;
- u32 full;
-};
-
-struct _cpuid4_info {
- union _cpuid4_leaf_eax eax;
- union _cpuid4_leaf_ebx ebx;
- union _cpuid4_leaf_ecx ecx;
- unsigned long size;
-};
-
unsigned short num_cache_leaves;
-static int __cpuinit cpuid4_cache_lookup(int index, struct _cpuid4_info *this_leaf)
+int cpuid4_cache_lookup(int index, struct cpuid4_info *this_leaf)
{
union _cpuid4_leaf_eax eax;
union _cpuid4_leaf_ebx ebx;
@@ -185,7 +140,7 @@ unsigned int __cpuinit init_intel_cacheinfo(struct cpuinfo_x86 *c)
* parameters cpuid leaf to find the cache details
*/
for (i = 0; i < num_cache_leaves; i++) {
- struct _cpuid4_info this_leaf;
+ struct cpuid4_info this_leaf;
int retval;
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 15d4b91..b95408f 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -28,6 +28,7 @@
#include <xen/nodemask.h>
#include <xen/cpu.h>
#include <xsm/xsm.h>
+#include <asm/psr.h>
#define get_xen_guest_handle(val, hnd) do { val = (hnd).p; } while (0)
@@ -101,6 +102,48 @@ long arch_do_sysctl(
}
break;
+ case XEN_SYSCTL_psr_cmt_op:
+ if ( !psr_cmt_enabled() )
+ return -ENODEV;
+
+ if ( sysctl->u.psr_cmt_op.flags != 0 )
+ return -EINVAL;
+
+ switch ( sysctl->u.psr_cmt_op.cmd )
+ {
+ case XEN_SYSCTL_PSR_CMT_enabled:
+ sysctl->u.psr_cmt_op.data =
+ (psr_cmt->features & PSR_RESOURCE_TYPE_L3) &&
+ (psr_cmt->l3.features & PSR_CMT_L3_OCCUPANCY);
+ break;
+ case XEN_SYSCTL_PSR_CMT_get_total_rmid:
+ sysctl->u.psr_cmt_op.data = psr_cmt->rmid_max;
+ break;
+ case XEN_SYSCTL_PSR_CMT_get_l3_upscaling_factor:
+ sysctl->u.psr_cmt_op.data = psr_cmt->l3.upscaling_factor;
+ break;
+ case XEN_SYSCTL_PSR_CMT_get_l3_cache_size:
+ {
+ struct cpuid4_info info;
+
+ ret = cpuid4_cache_lookup(3, &info);
+ if ( ret < 0 )
+ break;
+
+ sysctl->u.psr_cmt_op.data = info.size / 1024; /* in KB unit */
+ }
+ break;
+ default:
+ sysctl->u.psr_cmt_op.data = 0;
+ ret = -ENOSYS;
+ break;
+ }
+
+ if ( __copy_to_guest(u_sysctl, sysctl, 1) )
+ ret = -EFAULT;
+
+ break;
+
default:
ret = -ENOSYS;
break;
diff --git a/xen/include/asm-x86/cpufeature.h b/xen/include/asm-x86/cpufeature.h
index 137d75c..d3bd14d 100644
--- a/xen/include/asm-x86/cpufeature.h
+++ b/xen/include/asm-x86/cpufeature.h
@@ -215,6 +215,51 @@
#define cpu_has_vmx boot_cpu_has(X86_FEATURE_VMXE)
#define cpu_has_cpuid_faulting boot_cpu_has(X86_FEATURE_CPUID_FAULTING)
+
+enum _cache_type {
+ CACHE_TYPE_NULL = 0,
+ CACHE_TYPE_DATA = 1,
+ CACHE_TYPE_INST = 2,
+ CACHE_TYPE_UNIFIED = 3
+};
+
+union _cpuid4_leaf_eax {
+ struct {
+ enum _cache_type type:5;
+ unsigned int level:3;
+ unsigned int is_self_initializing:1;
+ unsigned int is_fully_associative:1;
+ unsigned int reserved:4;
+ unsigned int num_threads_sharing:12;
+ unsigned int num_cores_on_die:6;
+ } split;
+ u32 full;
+};
+
+union _cpuid4_leaf_ebx {
+ struct {
+ unsigned int coherency_line_size:12;
+ unsigned int physical_line_partition:10;
+ unsigned int ways_of_associativity:10;
+ } split;
+ u32 full;
+};
+
+union _cpuid4_leaf_ecx {
+ struct {
+ unsigned int number_of_sets:32;
+ } split;
+ u32 full;
+};
+
+struct cpuid4_info {
+ union _cpuid4_leaf_eax eax;
+ union _cpuid4_leaf_ebx ebx;
+ union _cpuid4_leaf_ecx ecx;
+ unsigned long size;
+};
+
+int cpuid4_cache_lookup(int index, struct cpuid4_info *this_leaf);
#endif
#endif /* __ASM_I386_CPUFEATURE_H */
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 3588698..66b6e47 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -636,6 +636,18 @@ struct xen_sysctl_coverage_op {
typedef struct xen_sysctl_coverage_op xen_sysctl_coverage_op_t;
DEFINE_XEN_GUEST_HANDLE(xen_sysctl_coverage_op_t);
+#define XEN_SYSCTL_PSR_CMT_get_total_rmid 0
+#define XEN_SYSCTL_PSR_CMT_get_l3_upscaling_factor 1
+/* The L3 cache size is returned in KB unit */
+#define XEN_SYSCTL_PSR_CMT_get_l3_cache_size 2
+#define XEN_SYSCTL_PSR_CMT_enabled 3
+struct xen_sysctl_psr_cmt_op {
+ uint32_t cmd;
+ uint32_t flags; /* padding variable, may be extended for future use */
+ uint64_t data;
+};
+typedef struct xen_sysctl_psr_cmt_op xen_sysctl_psr_cmt_op_t;
+DEFINE_XEN_GUEST_HANDLE(xen_sysctl_psr_cmt_op_t);
struct xen_sysctl {
uint32_t cmd;
@@ -658,6 +670,7 @@ struct xen_sysctl {
#define XEN_SYSCTL_cpupool_op 18
#define XEN_SYSCTL_scheduler_op 19
#define XEN_SYSCTL_coverage_op 20
+#define XEN_SYSCTL_psr_cmt_op 21
uint32_t interface_version; /* XEN_SYSCTL_INTERFACE_VERSION */
union {
struct xen_sysctl_readconsole readconsole;
@@ -679,6 +692,7 @@ struct xen_sysctl {
struct xen_sysctl_cpupool_op cpupool_op;
struct xen_sysctl_scheduler_op scheduler_op;
struct xen_sysctl_coverage_op coverage_op;
+ struct xen_sysctl_psr_cmt_op psr_cmt_op;
uint8_t pad[128];
} u;
};
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 07/10] x86: enable CMT for each domain RMID
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (5 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 06/10] x86: collect global CMT information Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 08/10] x86: add CMT related MSRs in allowed list Chao Peng
` (2 subsequent siblings)
9 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
If the CMT service is attached to a domain, its related RMID
will be set to hardware for monitoring when the domain's vcpu is
scheduled in. When the domain's vcpu is scheduled out, RMID 0
(system reserved) will be set for monitoring.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/domain.c | 5 +++++
xen/arch/x86/psr.c | 27 +++++++++++++++++++++++++++
xen/include/asm-x86/msr-index.h | 3 +++
xen/include/asm-x86/psr.h | 1 +
4 files changed, 36 insertions(+)
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 3cfd8f4..04a6719 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1418,6 +1418,8 @@ static void __context_switch(void)
{
memcpy(&p->arch.user_regs, stack_regs, CTXT_SWITCH_STACK_BYTES);
vcpu_save_fpu(p);
+ if ( psr_cmt_enabled() )
+ psr_assoc_rmid(0);
p->arch.ctxt_switch_from(p);
}
@@ -1442,6 +1444,9 @@ static void __context_switch(void)
}
vcpu_restore_fpu_eager(n);
n->arch.ctxt_switch_to(n);
+
+ if ( psr_cmt_enabled() && n->domain->arch.psr_rmid > 0 )
+ psr_assoc_rmid(n->domain->arch.psr_rmid);
}
gdt = !is_pv_32on64_vcpu(n) ? per_cpu(gdt_table, cpu) :
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 4b4a4cf..c5ee16e 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -20,9 +20,15 @@
#define PSR_CMT (1<<0)
+struct pqr_assoc {
+ uint64_t val;
+ bool_t initialized;
+};
+
struct psr_cmt *__read_mostly psr_cmt;
static bool_t __initdata opt_psr;
static unsigned int __initdata opt_rmid_max = 255;
+static DEFINE_PER_CPU(struct pqr_assoc, pqr_assoc);
static void __init parse_psr_param(char *s)
{
@@ -149,6 +155,27 @@ void psr_free_rmid(struct domain *d)
d->arch.psr_rmid = 0;
}
+void psr_assoc_rmid(unsigned int rmid)
+{
+ uint64_t val;
+ uint64_t new_val;
+ struct pqr_assoc *pqr = &this_cpu(pqr_assoc);
+
+ if ( !pqr->initialized )
+ {
+ rdmsrl(MSR_IA32_PQR_ASSOC, pqr->val);
+ pqr->initialized = 1;
+ }
+ val = pqr->val;
+
+ new_val = (val & ~psr_cmt->rmid_mask) | (rmid & psr_cmt->rmid_mask);
+ if ( val != new_val )
+ {
+ wrmsrl(MSR_IA32_PQR_ASSOC, new_val);
+ pqr->val = new_val;
+ }
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 542222e..dcb2b87 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -323,6 +323,9 @@
#define MSR_IA32_TSC_DEADLINE 0x000006E0
#define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
+/* Platform Shared Resource MSRs */
+#define MSR_IA32_PQR_ASSOC 0x00000c8f
+
/* Intel Model 6 */
#define MSR_P6_PERFCTR(n) (0x000000c1 + (n))
#define MSR_P6_EVNTSEL(n) (0x00000186 + (n))
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 98cfa6d..b544306 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -47,6 +47,7 @@ static inline bool_t psr_cmt_enabled(void)
int psr_alloc_rmid(struct domain *d);
void psr_free_rmid(struct domain *d);
+void psr_assoc_rmid(unsigned int rmid);
#endif /* __ASM_PSR_H__ */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (6 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 07/10] x86: enable CMT for each domain RMID Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 13:26 ` Jan Beulich
2014-09-30 10:49 ` [PATCH v18 09/10] xsm: add CMT related xsm policies Chao Peng
2014-09-30 10:49 ` [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology Chao Peng
9 siblings, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Tool stack will try to access the two MSRs to perform CMT
related operations, thus added them in the allowed list.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
xen/arch/x86/platform_hypercall.c | 8 ++++++++
xen/include/asm-x86/msr-index.h | 2 ++
2 files changed, 10 insertions(+)
diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
index 3d873b5..ad6a96f 100644
--- a/xen/arch/x86/platform_hypercall.c
+++ b/xen/arch/x86/platform_hypercall.c
@@ -70,6 +70,14 @@ struct xen_resource_access {
static bool_t allow_access_msr(unsigned int msr)
{
+ switch ( msr )
+ {
+ /* MSR for CMT, refer to chapter 17.14 of Intel SDM. */
+ case MSR_IA32_QOSEVTSEL:
+ case MSR_IA32_QMC:
+ return 1;
+ }
+
return 0;
}
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index dcb2b87..ae089fb 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -324,6 +324,8 @@
#define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
/* Platform Shared Resource MSRs */
+#define MSR_IA32_QOSEVTSEL 0x00000c8d
+#define MSR_IA32_QMC 0x00000c8e
#define MSR_IA32_PQR_ASSOC 0x00000c8f
/* Intel Model 6 */
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 09/10] xsm: add CMT related xsm policies
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (7 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 08/10] x86: add CMT related MSRs in allowed list Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology Chao Peng
9 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Add xsm policies for CMT related hypercalls.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
tools/flask/policy/policy/modules/xen/xen.if | 2 +-
tools/flask/policy/policy/modules/xen/xen.te | 3 ++-
xen/xsm/flask/hooks.c | 6 ++++++
xen/xsm/flask/policy/access_vectors | 4 ++++
4 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/tools/flask/policy/policy/modules/xen/xen.if b/tools/flask/policy/policy/modules/xen/xen.if
index 32b51b6..641c797 100644
--- a/tools/flask/policy/policy/modules/xen/xen.if
+++ b/tools/flask/policy/policy/modules/xen/xen.if
@@ -49,7 +49,7 @@ define(`create_domain_common', `
getdomaininfo hypercall setvcpucontext setextvcpucontext
getscheduler getvcpuinfo getvcpuextstate getaddrsize
getaffinity setaffinity };
- allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim set_max_evtchn set_vnumainfo get_vnumainfo };
+ allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim set_max_evtchn set_vnumainfo get_vnumainfo psr_cmt_op };
allow $1 $2:security check_context;
allow $1 $2:shadow enable;
allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage mmuext_op };
diff --git a/tools/flask/policy/policy/modules/xen/xen.te b/tools/flask/policy/policy/modules/xen/xen.te
index 6cecf97..d214470 100644
--- a/tools/flask/policy/policy/modules/xen/xen.te
+++ b/tools/flask/policy/policy/modules/xen/xen.te
@@ -66,6 +66,7 @@ allow dom0_t xen_t:xen {
};
allow dom0_t xen_t:xen2 {
resource_op
+ psr_cmt_op
};
allow dom0_t xen_t:mmu memorymap;
@@ -79,7 +80,7 @@ allow dom0_t dom0_t:domain {
getpodtarget setpodtarget set_misc_info set_virq_handler
};
allow dom0_t dom0_t:domain2 {
- set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo get_vnumainfo
+ set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo get_vnumainfo psr_cmt_op
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 9f36503..1bb3c00 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -722,6 +722,8 @@ static int flask_domctl(struct domain *d, int cmd)
case XEN_DOMCTL_setvnumainfo:
return current_has_perm(d, SECCLASS_DOMAIN, DOMAIN2__SET_VNUMAINFO);
+ case XEN_DOMCTL_psr_cmt_op:
+ return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_CMT_OP);
default:
printk("flask_domctl: Unknown op %d\n", cmd);
@@ -778,6 +780,10 @@ static int flask_sysctl(int cmd)
case XEN_SYSCTL_numainfo:
return domain_has_xen(current->domain, XEN__PHYSINFO);
+ case XEN_SYSCTL_psr_cmt_op:
+ return avc_current_has_perm(SECINITSID_XEN, SECCLASS_XEN2,
+ XEN2__PSR_CMT_OP, NULL);
+
default:
printk("flask_sysctl: Unknown op %d\n", cmd);
return -EPERM;
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index daf0de5..de0c707 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -81,6 +81,8 @@ class xen2
{
# XENPF_resource_op
resource_op
+# XEN_SYSCTL_psr_cmt_op
+ psr_cmt_op
}
# Classes domain and domain2 consist of operations that a domain performs on
@@ -212,6 +214,8 @@ class domain2
set_vnumainfo
# XENMEM_getvnumainfo
get_vnumainfo
+# XEN_DOMCTL_psr_cmt_op
+ psr_cmt_op
}
# Similar to class domain, but primarily contains domctls related to HVM domains
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
` (8 preceding siblings ...)
2014-09-30 10:49 ` [PATCH v18 09/10] xsm: add CMT related xsm policies Chao Peng
@ 2014-09-30 10:49 ` Chao Peng
2014-10-02 14:57 ` Konrad Rzeszutek Wilk
9 siblings, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-09-30 10:49 UTC (permalink / raw)
To: xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, JBeulich, dgdegra
Introduced some new xl commands to enable/disable Cache Monitoring
Technology(CMT) feature.
The following two commands is to attach/detach the CMT feature
to/from a certain domain.
$ xl psr-cmt-attach domid
$ xl psr-cmt-detach domid
This command is to display the CMT information, such as L3 cache
occupancy.
$ xl psr-cmt-show cache_occupancy <domid>
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
---
docs/man/xl.pod.1 | 25 +++++
tools/libxc/Makefile | 1 +
tools/libxc/xc_msr_x86.h | 36 ++++++++
tools/libxc/xc_psr.c | 214 +++++++++++++++++++++++++++++++++++++++++++
tools/libxc/xenctrl.h | 17 ++++
tools/libxl/Makefile | 2 +-
tools/libxl/libxl.h | 19 ++++
tools/libxl/libxl_psr.c | 184 +++++++++++++++++++++++++++++++++++++
tools/libxl/libxl_types.idl | 4 +
tools/libxl/libxl_utils.c | 28 ++++++
tools/libxl/xl.h | 3 +
tools/libxl/xl_cmdimpl.c | 132 ++++++++++++++++++++++++++
tools/libxl/xl_cmdtable.c | 17 ++++
13 files changed, 681 insertions(+), 1 deletion(-)
create mode 100644 tools/libxc/xc_msr_x86.h
create mode 100644 tools/libxc/xc_psr.c
create mode 100644 tools/libxl/libxl_psr.c
diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index f1e95db..ef8cd24 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -1387,6 +1387,31 @@ Load FLASK policy from the given policy file. The initial policy is provided to
the hypervisor as a multiboot module; this command allows runtime updates to the
policy. Loading new security policy will reset runtime changes to device labels.
+=head1 Cache Monitoring Technology
+
+Some new hardware may offer monitoring capability in each logical processor to
+measure specific platform shared resource metric, for example, L3 cache
+occupancy. In Xen implementation, the monitoring granularity is domain level.
+To monitor a specific domain, just attach the domain id with the monitoring
+service. When the domain doesn't need to be monitored any more, detach the
+domain id from the monitoring service.
+
+=over 4
+
+=item B<psr-cmt-attach> [I<domain-id>]
+
+attach: Attach the platform shared resource monitoring service to a domain.
+
+=item B<psr-cmt-detach> [I<domain-id>]
+
+detach: Detach the platform shared resource monitoring service from a domain.
+
+=item B<psr-cmt-show> [I<psr-monitor-type>] [I<domain-id>]
+
+Show monitoring data for a certain domain or all domains. Current supported
+monitor types are:
+ - "cache-occupancy": showing the L3 cache occupancy.
+
=back
=head1 TO BE DOCUMENTED
diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index dde6109..d8cc21b 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -35,6 +35,7 @@ CTRL_SRCS-y += xc_kexec.c
CTRL_SRCS-y += xtl_core.c
CTRL_SRCS-y += xtl_logger_stdio.c
CTRL_SRCS-y += xc_resource.c
+CTRL_SRCS-y += xc_psr.c
CTRL_SRCS-$(CONFIG_X86) += xc_pagetab.c
CTRL_SRCS-$(CONFIG_Linux) += xc_linux.c xc_linux_osdep.c
CTRL_SRCS-$(CONFIG_FreeBSD) += xc_freebsd.c xc_freebsd_osdep.c
diff --git a/tools/libxc/xc_msr_x86.h b/tools/libxc/xc_msr_x86.h
new file mode 100644
index 0000000..1e0ee99
--- /dev/null
+++ b/tools/libxc/xc_msr_x86.h
@@ -0,0 +1,36 @@
+/*
+ * xc_msr_x86.h
+ *
+ * MSR definition macros
+ *
+ * Copyright (C) 2014 Intel Corporation
+ * Author Dongxiao Xu <dongxiao.xu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#ifndef XC_MSR_X86_H
+#define XC_MSR_X86_H
+
+#define MSR_IA32_QOSEVTSEL 0x00000c8d
+#define MSR_IA32_QMC 0x00000c8e
+
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
new file mode 100644
index 0000000..182ddd4
--- /dev/null
+++ b/tools/libxc/xc_psr.c
@@ -0,0 +1,214 @@
+/*
+ * xc_psr.c
+ *
+ * platform shared resource related API functions.
+ *
+ * Copyright (C) 2014 Intel Corporation
+ * Author Dongxiao Xu <dongxiao.xu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "xc_private.h"
+#include "xc_msr_x86.h"
+
+#define IA32_QM_CTR_ERROR_MASK (0x3ull << 62)
+
+#define EVTID_L3_OCCUPANCY 0x1
+
+int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid)
+{
+ DECLARE_DOMCTL;
+
+ domctl.cmd = XEN_DOMCTL_psr_cmt_op;
+ domctl.domain = (domid_t)domid;
+ domctl.u.psr_cmt_op.cmd = XEN_DOMCTL_PSR_CMT_OP_ATTACH;
+
+ return do_domctl(xch, &domctl);
+}
+
+int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid)
+{
+ DECLARE_DOMCTL;
+
+ domctl.cmd = XEN_DOMCTL_psr_cmt_op;
+ domctl.domain = (domid_t)domid;
+ domctl.u.psr_cmt_op.cmd = XEN_DOMCTL_PSR_CMT_OP_DETACH;
+
+ return do_domctl(xch, &domctl);
+}
+
+int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
+ uint32_t *rmid)
+{
+ int rc;
+ DECLARE_DOMCTL;
+
+ domctl.cmd = XEN_DOMCTL_psr_cmt_op;
+ domctl.domain = (domid_t)domid;
+ domctl.u.psr_cmt_op.cmd = XEN_DOMCTL_PSR_CMT_OP_QUERY_RMID;
+
+ rc = do_domctl(xch, &domctl);
+
+ if ( !rc )
+ *rmid = domctl.u.psr_cmt_op.data;
+
+ return rc;
+}
+
+int xc_psr_cmt_get_total_rmid(xc_interface *xch, uint32_t *total_rmid)
+{
+ static int val = 0;
+ int rc;
+ DECLARE_SYSCTL;
+
+ if ( val )
+ {
+ *total_rmid = val;
+ return 0;
+ }
+
+ sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
+ sysctl.u.psr_cmt_op.cmd = XEN_SYSCTL_PSR_CMT_get_total_rmid;
+ sysctl.u.psr_cmt_op.flags = 0;
+
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ val = *total_rmid = sysctl.u.psr_cmt_op.data;
+
+ return rc;
+}
+
+int xc_psr_cmt_get_l3_upscaling_factor(xc_interface *xch,
+ uint32_t *upscaling_factor)
+{
+ static int val = 0;
+ int rc;
+ DECLARE_SYSCTL;
+
+ if ( val )
+ {
+ *upscaling_factor = val;
+ return 0;
+ }
+
+ sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
+ sysctl.u.psr_cmt_op.cmd =
+ XEN_SYSCTL_PSR_CMT_get_l3_upscaling_factor;
+ sysctl.u.psr_cmt_op.flags = 0;
+
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ val = *upscaling_factor = sysctl.u.psr_cmt_op.data;
+
+ return rc;
+}
+
+int xc_psr_cmt_get_l3_cache_size(xc_interface *xch,
+ uint32_t *l3_cache_size)
+{
+ static int val = 0;
+ int rc;
+ DECLARE_SYSCTL;
+
+ if ( val )
+ {
+ *l3_cache_size = val;
+ return 0;
+ }
+
+ sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
+ sysctl.u.psr_cmt_op.cmd =
+ XEN_SYSCTL_PSR_CMT_get_l3_cache_size;
+ sysctl.u.psr_cmt_op.flags = 0;
+
+ rc = xc_sysctl(xch, &sysctl);
+ if ( !rc )
+ val = *l3_cache_size= sysctl.u.psr_cmt_op.data;
+
+ return rc;
+}
+
+int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
+ uint32_t cpu, xc_psr_cmt_type type, uint64_t *monitor_data)
+{
+ xc_resource_op_t op;
+ xc_resource_entry_t entries[2];
+ uint32_t evtid;
+ int rc;
+
+ switch ( type )
+ {
+ case XC_PSR_CMT_L3_OCCUPANCY:
+ evtid = EVTID_L3_OCCUPANCY;
+ break;
+ default:
+ return -1;
+ }
+
+ entries[0].u.cmd = XEN_RESOURCE_OP_MSR_WRITE;
+ entries[0].idx = MSR_IA32_QOSEVTSEL;
+ entries[0].val = (uint64_t)rmid << 32 | evtid;
+ entries[0].rsvd = 0;
+
+ entries[1].u.cmd = XEN_RESOURCE_OP_MSR_READ;
+ entries[1].idx = MSR_IA32_QMC;
+ entries[1].val = 0;
+ entries[1].rsvd = 0;
+
+ op.cpu = cpu;
+ op.nr_entries = 2;
+ op.entries = entries;
+
+ rc = xc_resource_op(xch, 1, &op);
+ if ( rc < 0 )
+ return rc;
+
+ if ( op.result !=2 || entries[1].val & IA32_QM_CTR_ERROR_MASK )
+ return -1;
+
+ *monitor_data = entries[1].val;
+
+ return 0;
+}
+
+int xc_psr_cmt_enabled(xc_interface *xch)
+{
+ static int val = -1;
+ int rc;
+ DECLARE_SYSCTL;
+
+ if ( val >= 0 )
+ return val;
+
+ sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
+ sysctl.u.psr_cmt_op.cmd = XEN_SYSCTL_PSR_CMT_enabled;
+ sysctl.u.psr_cmt_op.flags = 0;
+
+ rc = do_sysctl(xch, &sysctl);
+ if ( !rc )
+ {
+ val = sysctl.u.psr_cmt_op.data;
+ return val;
+ }
+
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index 6edb738..b0ab6f4 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -2672,6 +2672,23 @@ struct xc_resource_op {
typedef struct xc_resource_op xc_resource_op_t;
int xc_resource_op(xc_interface *xch, uint32_t nr_ops, xc_resource_op_t *ops);
+enum xc_psr_cmt_type {
+ XC_PSR_CMT_L3_OCCUPANCY,
+};
+typedef enum xc_psr_cmt_type xc_psr_cmt_type;
+int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
+int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
+int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
+ uint32_t *rmid);
+int xc_psr_cmt_get_total_rmid(xc_interface *xch, uint32_t *total_rmid);
+int xc_psr_cmt_get_l3_upscaling_factor(xc_interface *xch,
+ uint32_t *upscaling_factor);
+int xc_psr_cmt_get_l3_cache_size(xc_interface *xch,
+ uint32_t *l3_cache_size);
+int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
+ uint32_t cpu, uint32_t psr_cmt_type, uint64_t *monitor_data);
+int xc_psr_cmt_enabled(xc_interface *xch);
+
#endif /* XENCTRL_H */
/*
diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index 990414b..fd9ae28 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -43,7 +43,7 @@ LIBXL_OBJS-y += libxl_blktap2.o
else
LIBXL_OBJS-y += libxl_noblktap2.o
endif
-LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o
+LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o libxl_psr.o
LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_arm.o
ifeq ($(CONFIG_NetBSD),y)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index bc68cac..5acc933 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -640,6 +640,13 @@ typedef uint8_t libxl_mac[6];
#define LIBXL_MAC_BYTES(mac) mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]
void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, libxl_mac *src);
+/*
+ * LIBXL_HAVE_PSR_CMT
+ *
+ * If this is defined, the Cache Monitoring Technology feature is supported.
+ */
+#define LIBXL_HAVE_PSR_CMT 1
+
typedef char **libxl_string_list;
void libxl_string_list_dispose(libxl_string_list *sl);
int libxl_string_list_length(const libxl_string_list *sl);
@@ -1380,6 +1387,18 @@ bool libxl_ms_vm_genid_is_zero(const libxl_ms_vm_genid *id);
void libxl_ms_vm_genid_copy(libxl_ctx *ctx, libxl_ms_vm_genid *dst,
libxl_ms_vm_genid *src);
+int libxl_get_socket_cpu(libxl_ctx *ctx, uint32_t socketid);
+
+int libxl_psr_cmt_attach(libxl_ctx *ctx, uint32_t domid);
+int libxl_psr_cmt_detach(libxl_ctx *ctx, uint32_t domid);
+int libxl_psr_cmt_domain_attached(libxl_ctx *ctx, uint32_t domid);
+int libxl_psr_cmt_enabled(libxl_ctx *ctx);
+int libxl_psr_cmt_get_total_rmid(libxl_ctx *ctx, uint32_t *total_rmid);
+int libxl_psr_cmt_get_l3_cache_size(libxl_ctx *ctx,
+ uint32_t *l3_cache_size);
+int libxl_psr_cmt_get_cache_occupancy(libxl_ctx *ctx, uint32_t domid,
+ uint32_t socketid, uint32_t *l3_cache_occupancy);
+
/* misc */
/* Each of these sets or clears the flag according to whether the
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
new file mode 100644
index 0000000..5551be8
--- /dev/null
+++ b/tools/libxl/libxl_psr.c
@@ -0,0 +1,184 @@
+/*
+ * Copyright (C) 2014 Intel Corporation
+ * Author Dongxiao Xu <dongxiao.xu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+#include "libxl_internal.h"
+
+
+#define IA32_QM_CTR_ERROR_MASK (0x3ul << 62)
+
+static void libxl_psr_cmt_err_msg(libxl_ctx *ctx, int err)
+{
+ GC_INIT(ctx);
+
+ char *msg;
+
+ switch (err) {
+ case ENOSYS:
+ msg = "unsupported operation";
+ break;
+ case ENODEV:
+ msg = "Cache Monitoring Technology is not supported in this system";
+ break;
+ case EEXIST:
+ msg = "Cache Monitoring Technology is already attached to this domain";
+ break;
+ case ENOENT:
+ msg = "Cache Monitoring Technology is not attached to this domain";
+ break;
+ case EUSERS:
+ msg = "there is no free RMID available";
+ break;
+ case ESRCH:
+ msg = "is this Domain ID valid?";
+ break;
+ case EFAULT:
+ msg = "failed to exchange data with Xen";
+ break;
+ default:
+ msg = "unknown error";
+ break;
+ }
+
+ LOGE(ERROR, "%s", msg);
+
+ GC_FREE;
+}
+
+int libxl_psr_cmt_attach(libxl_ctx *ctx, uint32_t domid)
+{
+ int rc;
+
+ rc = xc_psr_cmt_attach(ctx->xch, domid);
+ if (rc < 0) {
+ libxl_psr_cmt_err_msg(ctx, errno);
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+int libxl_psr_cmt_detach(libxl_ctx *ctx, uint32_t domid)
+{
+ int rc;
+
+ rc = xc_psr_cmt_detach(ctx->xch, domid);
+ if (rc < 0) {
+ libxl_psr_cmt_err_msg(ctx, errno);
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+int libxl_psr_cmt_domain_attached(libxl_ctx *ctx, uint32_t domid)
+{
+ int rc;
+ uint32_t rmid;
+
+ rc = xc_psr_cmt_get_domain_rmid(ctx->xch, domid, &rmid);
+ if (rc < 0)
+ return 0;
+
+ return !!rmid;
+}
+
+int libxl_psr_cmt_enabled(libxl_ctx *ctx)
+{
+ return xc_psr_cmt_enabled(ctx->xch);
+}
+
+int libxl_psr_cmt_get_total_rmid(libxl_ctx *ctx, uint32_t *total_rmid)
+{
+ int rc;
+
+ rc = xc_psr_cmt_get_total_rmid(ctx->xch, total_rmid);
+ if (rc < 0) {
+ libxl_psr_cmt_err_msg(ctx, errno);
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+int libxl_psr_cmt_get_l3_cache_size(libxl_ctx *ctx,
+ uint32_t *l3_cache_size)
+{
+ int rc;
+
+ rc = xc_psr_cmt_get_l3_cache_size(ctx->xch, l3_cache_size);
+ if (rc < 0) {
+ libxl_psr_cmt_err_msg(ctx, errno);
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
+
+int libxl_psr_cmt_get_cache_occupancy(libxl_ctx *ctx, uint32_t domid,
+ uint32_t socketid, uint32_t *l3_cache_occupancy)
+{
+ GC_INIT(ctx);
+
+ unsigned int rmid;
+ uint32_t upscaling_factor;
+ uint64_t monitor_data;
+ int cpu, rc;
+ xc_psr_cmt_type type;
+
+ rc = xc_psr_cmt_get_domain_rmid(ctx->xch, domid, &rmid);
+ if (rc < 0 || rmid == 0) {
+ LOGE(ERROR, "fail to get the domain rmid, "
+ "or domain is not attached with platform QoS monitoring service");
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ cpu = libxl_get_socket_cpu(ctx, socketid);
+ if (cpu < 0) {
+ LOGE(ERROR, "failed to get socket cpu");
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ type = XC_PSR_CMT_L3_OCCUPANCY;
+ rc = xc_psr_cmt_get_data(ctx->xch, rmid, cpu, type, &monitor_data);
+ if (rc < 0) {
+ LOGE(ERROR, "failed to get monitoring data");
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ rc = xc_psr_cmt_get_l3_upscaling_factor(ctx->xch, &upscaling_factor);
+ if (rc < 0) {
+ LOGE(ERROR, "failed to get L3 upscaling factor");
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ *l3_cache_occupancy = upscaling_factor * monitor_data / 1024;
+ rc = 0;
+out:
+ GC_FREE;
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index f1fcbc3..27a5022 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -635,3 +635,7 @@ libxl_event = Struct("event",[
])),
("domain_create_console_available", None),
]))])
+
+libxl_psr_cmt_type = Enumeration("psr_cmt_type", [
+ (1, "CACHE_OCCUPANCY"),
+ ])
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 58df4f3..318a996 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -1065,6 +1065,34 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf, size_t len)
return ret;
}
+int libxl_get_socket_cpu(libxl_ctx *ctx, uint32_t socketid)
+{
+ int i, j, cpu, nr_cpus;
+ libxl_cputopology *topology;
+ int *socket_cpus;
+
+ topology = libxl_get_cpu_topology(ctx, &nr_cpus);
+ if (!topology)
+ return ERROR_FAIL;
+
+ socket_cpus = malloc(sizeof(int) * nr_cpus);
+ if (!socket_cpus) {
+ libxl_cputopology_list_free(topology, nr_cpus);
+ return ERROR_FAIL;
+ }
+
+ for (i = 0, j = 0; i < nr_cpus; i++)
+ if (topology[i].socket == socketid)
+ socket_cpus[j++] = i;
+
+ /* load balance among cpus in the same socket. */
+ cpu = socket_cpus[rand() % j];
+
+ free(socket_cpus);
+ libxl_cputopology_list_free(topology, nr_cpus);
+ return cpu;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
index 10a2e66..abb1fac 100644
--- a/tools/libxl/xl.h
+++ b/tools/libxl/xl.h
@@ -110,6 +110,9 @@ int main_loadpolicy(int argc, char **argv);
int main_remus(int argc, char **argv);
#endif
int main_devd(int argc, char **argv);
+int main_psr_cmt_attach(int argc, char **argv);
+int main_psr_cmt_detach(int argc, char **argv);
+int main_psr_cmt_show(int argc, char **argv);
void help(const char *command);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 698b3bc..77f19d7 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -7428,6 +7428,138 @@ out:
return ret;
}
+static int psr_cmt_show_cache_occupancy(uint32_t domid)
+{
+ uint32_t i, socketid, nr_sockets, total_rmid;
+ uint32_t l3_cache_size, l3_cache_occupancy;
+ libxl_physinfo info;
+ char *domain_name;
+ int rc, print_header, nr_domains;
+ libxl_dominfo *dominfo;
+
+ if (!libxl_psr_cmt_enabled(ctx)) {
+ printf("CMT is disabled in the system\n");
+ return -1;
+ }
+
+ rc = libxl_get_physinfo(ctx, &info);
+ if (rc < 0) {
+ printf("Failed getting physinfo, rc: %d\n", rc);
+ return -1;
+ }
+ nr_sockets = info.nr_cpus / info.threads_per_core / info.cores_per_socket;
+
+ rc = libxl_psr_cmt_get_total_rmid(ctx, &total_rmid);
+ if (rc < 0) {
+ printf("Failed to get max RMID value\n");
+ return -1;
+ }
+
+ rc = libxl_psr_cmt_get_l3_cache_size(ctx, &l3_cache_size);
+ if (rc < 0) {
+ printf("Failed to get system l3 cache size\n");
+ return -1;
+ }
+
+ printf("Total RMID: %d\n", total_rmid);
+ printf("Per-Socket L3 Cache Size: %d KB\n", l3_cache_size);
+
+ print_header = 1;
+ if (!(dominfo = libxl_list_domain(ctx, &nr_domains))) {
+ fprintf(stderr, "libxl_list_domain failed.\n");
+ return -1;
+ }
+ for (i = 0; i < nr_domains; i++) {
+ if (domid != ~0 && dominfo[i].domid != domid)
+ continue;
+ if (!libxl_psr_cmt_domain_attached(ctx, dominfo[i].domid))
+ continue;
+ if (print_header) {
+ printf("%-40s %5s", "Name", "ID");
+ for (socketid = 0; socketid < nr_sockets; socketid++)
+ printf("%14s %d", "Socket", socketid);
+ printf("\n");
+ print_header = 0;
+ }
+ domain_name = libxl_domid_to_name(ctx, dominfo[i].domid);
+ printf("%-40s %5d", domain_name, dominfo[i].domid);
+ free(domain_name);
+ for (socketid = 0; socketid < nr_sockets; socketid++) {
+ rc = libxl_psr_cmt_get_cache_occupancy(ctx, dominfo[i].domid,
+ socketid, &l3_cache_occupancy);
+ if ( !rc )
+ printf("%13u KB", l3_cache_occupancy);
+ }
+ printf("\n");
+ }
+ libxl_dominfo_list_free(dominfo, nr_domains);
+
+ return 0;
+}
+
+int main_psr_cmt_attach(int argc, char **argv)
+{
+ uint32_t domid;
+ int opt, ret = 0;
+
+ SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cmt-attach", 1) {
+ /* No options */
+ }
+
+ domid = find_domain(argv[optind]);
+ ret = libxl_psr_cmt_attach(ctx, domid);
+
+ return ret;
+}
+
+int main_psr_cmt_detach(int argc, char **argv)
+{
+ uint32_t domid;
+ int opt, ret = 0;
+
+ SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cmt-detach", 1) {
+ /* No options */
+ }
+
+ domid = find_domain(argv[optind]);
+ ret = libxl_psr_cmt_detach(ctx, domid);
+
+ return ret;
+}
+
+int main_psr_cmt_show(int argc, char **argv)
+{
+ int opt, ret = 0;
+ uint32_t domid;
+ libxl_psr_cmt_type type;
+
+ SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cmt-show", 1) {
+ /* No options */
+ }
+
+ libxl_psr_cmt_type_from_string(argv[optind], &type);
+
+ if (optind + 1 >= argc)
+ domid = ~0;
+ else if (optind + 1 == argc - 1)
+ domid = find_domain(argv[optind + 1]);
+ else {
+ help("psr-cmt-show");
+ return 2;
+ }
+
+ switch (type) {
+ case LIBXL_PSR_CMT_TYPE_CACHE_OCCUPANCY:
+ ret = psr_cmt_show_cache_occupancy(domid);
+ break;
+ default:
+ help("psr-cmt-show");
+ return 2;
+ }
+
+ return ret;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 35f02c4..da25c3f 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -502,6 +502,23 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-F Run in the foreground",
},
+ { "psr-cmt-attach",
+ &main_psr_cmt_attach, 0, 1,
+ "Attach Cache Monitoring Technology service to a domain",
+ "<Domain>",
+ },
+ { "psr-cmt-detach",
+ &main_psr_cmt_detach, 0, 1,
+ "Detach Cache Monitoring Technology service from a domain",
+ "<Domain>",
+ },
+ { "psr-cmt-show",
+ &main_psr_cmt_show, 0, 1,
+ "Show Cache Monitoring Technology information",
+ "<PSR-CMT-Type> <Domain>",
+ "Available monitor types:\n"
+ "\"cache_occupancy\": Show L3 cache occupancy\n",
+ },
};
int cmdtable_len = sizeof(cmd_table)/sizeof(struct cmd_spec);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-09-30 10:49 ` [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall Chao Peng
@ 2014-09-30 11:00 ` Andrew Cooper
2014-09-30 11:08 ` Andrew Cooper
2014-09-30 12:12 ` Jan Beulich
1 sibling, 1 reply; 31+ messages in thread
From: Andrew Cooper @ 2014-09-30 11:00 UTC (permalink / raw)
To: Chao Peng, xen-devel
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, JBeulich, dgdegra
On 30/09/14 11:49, Chao Peng wrote:
> Add a generic resource access hypercall for tool stack or other
> components, e.g., accessing MSR, port I/O, etc.
>
> The resource is abstracted as a resource address/value pair.
> The resource access can be any type of XEN_RESOURCE_OP_*(current
> only support MSR and it's white-listed). The resource operations
> are always runs on cpu that caller specified. If caller does not
> care this, it should use current cpu to eliminate the IPI overhead.
>
> Batch resource operations in one call are also supported but the
> max number currently is limited to 2. The operations in a batch are
> non-preemptible and execute in their original order. If preemptible
> batch is desirable, then multicall mechanism can be used.
>
> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
> ---
> xen/arch/x86/platform_hypercall.c | 157 ++++++++++++++++++++++++++++++
> xen/arch/x86/x86_64/platform_hypercall.c | 4 +
> xen/include/public/platform.h | 35 +++++++
> xen/include/xlat.lst | 1 +
> 4 files changed, 197 insertions(+)
>
> diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
> index 2162811..3d873b5 100644
> --- a/xen/arch/x86/platform_hypercall.c
> +++ b/xen/arch/x86/platform_hypercall.c
> @@ -61,6 +61,94 @@ long cpu_down_helper(void *data);
> long core_parking_helper(void *data);
> uint32_t get_cur_idle_nums(void);
>
> +#define RESOURCE_ACCESS_MAX_ENTRIES 2
> +struct xen_resource_access {
> + unsigned int ret;
> + unsigned int nr_entries;
> + xenpf_resource_entry_t *entries;
> +};
> +
> +static bool_t allow_access_msr(unsigned int msr)
> +{
> + return 0;
> +}
> +
> +static unsigned int check_resource_access(struct xen_resource_access *ra)
> +{
> + xenpf_resource_entry_t *entry;
> + int ret = 0;
> + unsigned int i;
> +
> + for ( i = 0; i < ra->nr_entries; i++ )
> + {
> + entry = ra->entries + i;
> +
> + if ( entry->rsvd )
> + {
> + entry->u.ret = -EINVAL;
> + break;
> + }
> +
> + switch ( entry->u.cmd )
> + {
> + case XEN_RESOURCE_OP_MSR_READ:
> + case XEN_RESOURCE_OP_MSR_WRITE:
> + if ( entry->idx >> 32 )
> + ret = -EINVAL;
> + else if ( !allow_access_msr(entry->idx) )
> + ret = -EACCES;
> + break;
> + default:
> + ret = -EINVAL;
> + break;
> + }
> +
> + if ( ret )
> + {
> + entry->u.ret = ret;
You must always set entry->u.ret even on success.
Breaking on error is still fine though.
> + break;
> + }
> + }
> +
> + /* Return the number of successes. */
> + return i;
> +}
> +
> +static void resource_access(void *info)
> +{
> + struct xen_resource_access *ra = info;
> + xenpf_resource_entry_t *entry;
> + int ret;
> + unsigned int i;
> +
> + for ( i = 0; i < ra->nr_entries; i++ )
> + {
> + entry = ra->entries + i;
> +
> + switch ( entry->u.cmd )
> + {
> + case XEN_RESOURCE_OP_MSR_READ:
> + ret = rdmsr_safe(entry->idx, entry->val);
> + break;
> + case XEN_RESOURCE_OP_MSR_WRITE:
> + ret = wrmsr_safe(entry->idx, entry->val);
> + break;
> + default:
> + ret = -EINVAL;
> + break;
> + }
> +
> + if ( ret )
> + {
> + entry->u.ret = ret;
> + break;
> + }
Same here
> + }
> +
> + /* Return the number of successes. */
> + ra->ret = i;
> +}
> +
> ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
> {
> ret_t ret = 0;
> @@ -601,6 +689,75 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
> }
> break;
>
> + case XENPF_resource_op:
> + {
> + struct xen_resource_access ra;
> + uint32_t cpu;
> + XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
> +
> + ra.nr_entries = op->u.resource_op.nr_entries;
> + if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
> + {
> + ret = -EINVAL;
> + break;
> + }
> +
> + ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
> + if ( !ra.entries )
> + {
> + ret = -ENOMEM;
> + break;
> + }
> +
> + guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
> +
> + if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
> + {
> + xfree(ra.entries);
> + ret = -EFAULT;
> + break;
> + }
> +
> + /* Do sanity check earlier to omit the potential IPI overhead. */
> + if ( check_resource_access(&ra) < ra.nr_entries )
> + {
> + /* Copy the return value for failed entry. */
> + if ( __copy_to_guest_offset(guest_entries, ret,
> + ra.entries + ret, 1) )
> + ret = -EFAULT;
> + else
> + ret = 0;
> +
> + xfree(ra.entries);
> + break;
> + }
This however is not ok. You have possibly signalled that entry 0 has
passed the permission check and entry 1 has failed the check, at which
point you leave entry 0 uninitialised in userspace and signal failure
for entry 1.
If some MSRs pass the permission check, you should go ahead and fill
them in to provide the partial good data to userspace.
> +
> + cpu = op->u.resource_op.cpu;
> + if ( cpu == smp_processor_id() )
> + resource_access(&ra);
> + else if ( cpu_online(cpu) )
You must check cpu < nr_cpu_ids, or cpu_online() could wander off the
end of the bitmap.
The correct check is something more like:
if ( (cpu >= nr_cpu_ids) || !cpu_online(cpu) )
ret = -ENODEV;
else if ( cpu == smp_processor_id() )
local()
else
on_selectected_cpus()
~Andrew
> + on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
> + else
> + {
> + xfree(ra.entries);
> + ret = -ENODEV;
> + break;
> + }
> +
> + /* Copy all if succeeded or up to the failed entry. */
> + if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
> + min(ra.nr_entries, ra.ret + 1)) )
> + {
> + xfree(ra.entries);
> + ret = -EFAULT;
> + break;
> + }
> +
> + ret = ra.ret;
> + xfree(ra.entries);
> + }
> + break;
> +
> default:
> ret = -ENOSYS;
> break;
> diff --git a/xen/arch/x86/x86_64/platform_hypercall.c b/xen/arch/x86/x86_64/platform_hypercall.c
> index b6f380e..ccfd30d 100644
> --- a/xen/arch/x86/x86_64/platform_hypercall.c
> +++ b/xen/arch/x86/x86_64/platform_hypercall.c
> @@ -32,6 +32,10 @@ CHECK_pf_pcpu_version;
> CHECK_pf_enter_acpi_sleep;
> #undef xen_pf_enter_acpi_sleep
>
> +#define xen_pf_resource_entry xenpf_resource_entry
> +CHECK_pf_resource_entry;
> +#undef xen_pf_resource_entry
> +
> #define COMPAT
> #define _XEN_GUEST_HANDLE(t) XEN_GUEST_HANDLE(t)
> #define _XEN_GUEST_HANDLE_PARAM(t) XEN_GUEST_HANDLE_PARAM(t)
> diff --git a/xen/include/public/platform.h b/xen/include/public/platform.h
> index 053b9fa..abf916c 100644
> --- a/xen/include/public/platform.h
> +++ b/xen/include/public/platform.h
> @@ -528,6 +528,40 @@ typedef struct xenpf_core_parking xenpf_core_parking_t;
> DEFINE_XEN_GUEST_HANDLE(xenpf_core_parking_t);
>
> /*
> + * Access generic platform resources(e.g., accessing MSR, port I/O, etc)
> + * in unified way. Batch resource operations in one call are supported and
> + * thay are always non-preemptible and executed in their original order.
> + * The batch itself returns a negative integer for general errors, or a
> + * non-negative integer for the number of successful operations. For latter
> + * case, the @ret in the failed entry(if have) indicates the exact error and
> + * it's meaningful only when it has a negative value.
> + */
> +#define XENPF_resource_op 61
> +
> +#define XEN_RESOURCE_OP_MSR_READ 0
> +#define XEN_RESOURCE_OP_MSR_WRITE 1
> +
> +struct xenpf_resource_entry {
> + union {
> + uint32_t cmd; /* IN: XEN_RESOURCE_OP_* */
> + int32_t ret; /* OUT: return value for this entry */
> + } u;
> + uint32_t rsvd; /* IN: padding and must be zero */
> + uint64_t idx; /* IN: resource address to access */
> + uint64_t val; /* IN/OUT: resource value to set/get */
> +};
> +typedef struct xenpf_resource_entry xenpf_resource_entry_t;
> +DEFINE_XEN_GUEST_HANDLE(xenpf_resource_entry_t);
> +
> +struct xenpf_resource_op {
> + uint32_t nr_entries; /* number of resource entry */
> + uint32_t cpu; /* which cpu to run */
> + XEN_GUEST_HANDLE(xenpf_resource_entry_t) entries;
> +};
> +typedef struct xenpf_resource_op xenpf_resource_op_t;
> +DEFINE_XEN_GUEST_HANDLE(xenpf_resource_op_t);
> +
> +/*
> * ` enum neg_errnoval
> * ` HYPERVISOR_platform_op(const struct xen_platform_op*);
> */
> @@ -553,6 +587,7 @@ struct xen_platform_op {
> struct xenpf_cpu_hotadd cpu_add;
> struct xenpf_mem_hotadd mem_add;
> struct xenpf_core_parking core_parking;
> + struct xenpf_resource_op resource_op;
> uint8_t pad[128];
> } u;
> };
> diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst
> index 9a35dd7..234b668 100644
> --- a/xen/include/xlat.lst
> +++ b/xen/include/xlat.lst
> @@ -88,6 +88,7 @@
> ? xenpf_enter_acpi_sleep platform.h
> ? xenpf_pcpuinfo platform.h
> ? xenpf_pcpu_version platform.h
> +? xenpf_resource_entry platform.h
> ! sched_poll sched.h
> ? sched_remote_shutdown sched.h
> ? sched_shutdown sched.h
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-09-30 11:00 ` Andrew Cooper
@ 2014-09-30 11:08 ` Andrew Cooper
2014-09-30 12:15 ` Jan Beulich
0 siblings, 1 reply; 31+ messages in thread
From: Andrew Cooper @ 2014-09-30 11:08 UTC (permalink / raw)
To: Chao Peng, xen-devel, JBeulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, dgdegra
On 30/09/14 12:00, Andrew Cooper wrote:
> On 30/09/14 11:49, Chao Peng wrote:
>> Add a generic resource access hypercall for tool stack or other
>> components, e.g., accessing MSR, port I/O, etc.
>>
>> The resource is abstracted as a resource address/value pair.
>> The resource access can be any type of XEN_RESOURCE_OP_*(current
>> only support MSR and it's white-listed). The resource operations
>> are always runs on cpu that caller specified. If caller does not
>> care this, it should use current cpu to eliminate the IPI overhead.
>>
>> Batch resource operations in one call are also supported but the
>> max number currently is limited to 2. The operations in a batch are
>> non-preemptible and execute in their original order. If preemptible
>> batch is desirable, then multicall mechanism can be used.
>>
>> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
>> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
>> ---
>> xen/arch/x86/platform_hypercall.c | 157 ++++++++++++++++++++++++++++++
>> xen/arch/x86/x86_64/platform_hypercall.c | 4 +
>> xen/include/public/platform.h | 35 +++++++
>> xen/include/xlat.lst | 1 +
>> 4 files changed, 197 insertions(+)
>>
>> diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c
>> index 2162811..3d873b5 100644
>> --- a/xen/arch/x86/platform_hypercall.c
>> +++ b/xen/arch/x86/platform_hypercall.c
>> @@ -61,6 +61,94 @@ long cpu_down_helper(void *data);
>> long core_parking_helper(void *data);
>> uint32_t get_cur_idle_nums(void);
>>
>> +#define RESOURCE_ACCESS_MAX_ENTRIES 2
>> +struct xen_resource_access {
>> + unsigned int ret;
>> + unsigned int nr_entries;
>> + xenpf_resource_entry_t *entries;
>> +};
>> +
>> +static bool_t allow_access_msr(unsigned int msr)
>> +{
>> + return 0;
>> +}
>> +
>> +static unsigned int check_resource_access(struct xen_resource_access *ra)
>> +{
>> + xenpf_resource_entry_t *entry;
>> + int ret = 0;
>> + unsigned int i;
>> +
>> + for ( i = 0; i < ra->nr_entries; i++ )
>> + {
>> + entry = ra->entries + i;
>> +
>> + if ( entry->rsvd )
>> + {
>> + entry->u.ret = -EINVAL;
>> + break;
>> + }
>> +
>> + switch ( entry->u.cmd )
>> + {
>> + case XEN_RESOURCE_OP_MSR_READ:
>> + case XEN_RESOURCE_OP_MSR_WRITE:
>> + if ( entry->idx >> 32 )
>> + ret = -EINVAL;
>> + else if ( !allow_access_msr(entry->idx) )
>> + ret = -EACCES;
>> + break;
>> + default:
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>> + if ( ret )
>> + {
>> + entry->u.ret = ret;
> You must always set entry->u.ret even on success.
>
> Breaking on error is still fine though.
>
>> + break;
>> + }
>> + }
>> +
>> + /* Return the number of successes. */
>> + return i;
>> +}
>> +
>> +static void resource_access(void *info)
>> +{
>> + struct xen_resource_access *ra = info;
>> + xenpf_resource_entry_t *entry;
>> + int ret;
>> + unsigned int i;
>> +
>> + for ( i = 0; i < ra->nr_entries; i++ )
>> + {
>> + entry = ra->entries + i;
>> +
>> + switch ( entry->u.cmd )
>> + {
>> + case XEN_RESOURCE_OP_MSR_READ:
>> + ret = rdmsr_safe(entry->idx, entry->val);
>> + break;
>> + case XEN_RESOURCE_OP_MSR_WRITE:
>> + ret = wrmsr_safe(entry->idx, entry->val);
>> + break;
>> + default:
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>> + if ( ret )
>> + {
>> + entry->u.ret = ret;
>> + break;
>> + }
> Same here
>
>> + }
>> +
>> + /* Return the number of successes. */
>> + ra->ret = i;
>> +}
>> +
>> ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
>> {
>> ret_t ret = 0;
>> @@ -601,6 +689,75 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
>> }
>> break;
>>
>> + case XENPF_resource_op:
>> + {
>> + struct xen_resource_access ra;
>> + uint32_t cpu;
>> + XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
>> +
>> + ra.nr_entries = op->u.resource_op.nr_entries;
>> + if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
>> + {
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>> + ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
>> + if ( !ra.entries )
>> + {
>> + ret = -ENOMEM;
>> + break;
>> + }
>> +
>> + guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
>> +
>> + if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
>> + {
>> + xfree(ra.entries);
>> + ret = -EFAULT;
>> + break;
>> + }
>> +
>> + /* Do sanity check earlier to omit the potential IPI overhead. */
>> + if ( check_resource_access(&ra) < ra.nr_entries )
>> + {
>> + /* Copy the return value for failed entry. */
>> + if ( __copy_to_guest_offset(guest_entries, ret,
>> + ra.entries + ret, 1) )
>> + ret = -EFAULT;
>> + else
>> + ret = 0;
>> +
>> + xfree(ra.entries);
>> + break;
>> + }
> This however is not ok. You have possibly signalled that entry 0 has
> passed the permission check and entry 1 has failed the check, at which
> point you leave entry 0 uninitialised in userspace and signal failure
> for entry 1.
>
> If some MSRs pass the permission check, you should go ahead and fill
> them in to provide the partial good data to userspace.
Actually, on second thoughts this is not correct, because of the union
of cmd and ret.
I cannot see a way of making this function correctly given the union, as
there is no way of signalling "permission ok" between
check_resource_access() and resource_access() without clobbering the
command.
My suggestion is still as before - make use of rsvd field for ret and
drop the union, but I would appreciate Jan's thoughts as he explicitly
suggested the union.
~Andrew
>
>> +
>> + cpu = op->u.resource_op.cpu;
>> + if ( cpu == smp_processor_id() )
>> + resource_access(&ra);
>> + else if ( cpu_online(cpu) )
> You must check cpu < nr_cpu_ids, or cpu_online() could wander off the
> end of the bitmap.
>
> The correct check is something more like:
>
> if ( (cpu >= nr_cpu_ids) || !cpu_online(cpu) )
> ret = -ENODEV;
> else if ( cpu == smp_processor_id() )
> local()
> else
> on_selectected_cpus()
>
> ~Andrew
>
>> + on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
>> + else
>> + {
>> + xfree(ra.entries);
>> + ret = -ENODEV;
>> + break;
>> + }
>> +
>> + /* Copy all if succeeded or up to the failed entry. */
>> + if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
>> + min(ra.nr_entries, ra.ret + 1)) )
>> + {
>> + xfree(ra.entries);
>> + ret = -EFAULT;
>> + break;
>> + }
>> +
>> + ret = ra.ret;
>> + xfree(ra.entries);
>> + }
>> + break;
>> +
>> default:
>> ret = -ENOSYS;
>> break;
>> diff --git a/xen/arch/x86/x86_64/platform_hypercall.c b/xen/arch/x86/x86_64/platform_hypercall.c
>> index b6f380e..ccfd30d 100644
>> --- a/xen/arch/x86/x86_64/platform_hypercall.c
>> +++ b/xen/arch/x86/x86_64/platform_hypercall.c
>> @@ -32,6 +32,10 @@ CHECK_pf_pcpu_version;
>> CHECK_pf_enter_acpi_sleep;
>> #undef xen_pf_enter_acpi_sleep
>>
>> +#define xen_pf_resource_entry xenpf_resource_entry
>> +CHECK_pf_resource_entry;
>> +#undef xen_pf_resource_entry
>> +
>> #define COMPAT
>> #define _XEN_GUEST_HANDLE(t) XEN_GUEST_HANDLE(t)
>> #define _XEN_GUEST_HANDLE_PARAM(t) XEN_GUEST_HANDLE_PARAM(t)
>> diff --git a/xen/include/public/platform.h b/xen/include/public/platform.h
>> index 053b9fa..abf916c 100644
>> --- a/xen/include/public/platform.h
>> +++ b/xen/include/public/platform.h
>> @@ -528,6 +528,40 @@ typedef struct xenpf_core_parking xenpf_core_parking_t;
>> DEFINE_XEN_GUEST_HANDLE(xenpf_core_parking_t);
>>
>> /*
>> + * Access generic platform resources(e.g., accessing MSR, port I/O, etc)
>> + * in unified way. Batch resource operations in one call are supported and
>> + * thay are always non-preemptible and executed in their original order.
>> + * The batch itself returns a negative integer for general errors, or a
>> + * non-negative integer for the number of successful operations. For latter
>> + * case, the @ret in the failed entry(if have) indicates the exact error and
>> + * it's meaningful only when it has a negative value.
>> + */
>> +#define XENPF_resource_op 61
>> +
>> +#define XEN_RESOURCE_OP_MSR_READ 0
>> +#define XEN_RESOURCE_OP_MSR_WRITE 1
>> +
>> +struct xenpf_resource_entry {
>> + union {
>> + uint32_t cmd; /* IN: XEN_RESOURCE_OP_* */
>> + int32_t ret; /* OUT: return value for this entry */
>> + } u;
>> + uint32_t rsvd; /* IN: padding and must be zero */
>> + uint64_t idx; /* IN: resource address to access */
>> + uint64_t val; /* IN/OUT: resource value to set/get */
>> +};
>> +typedef struct xenpf_resource_entry xenpf_resource_entry_t;
>> +DEFINE_XEN_GUEST_HANDLE(xenpf_resource_entry_t);
>> +
>> +struct xenpf_resource_op {
>> + uint32_t nr_entries; /* number of resource entry */
>> + uint32_t cpu; /* which cpu to run */
>> + XEN_GUEST_HANDLE(xenpf_resource_entry_t) entries;
>> +};
>> +typedef struct xenpf_resource_op xenpf_resource_op_t;
>> +DEFINE_XEN_GUEST_HANDLE(xenpf_resource_op_t);
>> +
>> +/*
>> * ` enum neg_errnoval
>> * ` HYPERVISOR_platform_op(const struct xen_platform_op*);
>> */
>> @@ -553,6 +587,7 @@ struct xen_platform_op {
>> struct xenpf_cpu_hotadd cpu_add;
>> struct xenpf_mem_hotadd mem_add;
>> struct xenpf_core_parking core_parking;
>> + struct xenpf_resource_op resource_op;
>> uint8_t pad[128];
>> } u;
>> };
>> diff --git a/xen/include/xlat.lst b/xen/include/xlat.lst
>> index 9a35dd7..234b668 100644
>> --- a/xen/include/xlat.lst
>> +++ b/xen/include/xlat.lst
>> @@ -88,6 +88,7 @@
>> ? xenpf_enter_acpi_sleep platform.h
>> ? xenpf_pcpuinfo platform.h
>> ? xenpf_pcpu_version platform.h
>> +? xenpf_resource_entry platform.h
>> ! sched_poll sched.h
>> ? sched_remote_shutdown sched.h
>> ? sched_shutdown sched.h
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-09-30 10:49 ` [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall Chao Peng
2014-09-30 11:00 ` Andrew Cooper
@ 2014-09-30 12:12 ` Jan Beulich
2014-10-01 10:04 ` Chao Peng
1 sibling, 1 reply; 31+ messages in thread
From: Jan Beulich @ 2014-09-30 12:12 UTC (permalink / raw)
To: Chao Peng
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> +static unsigned int check_resource_access(struct xen_resource_access *ra)
> +{
> + xenpf_resource_entry_t *entry;
> + int ret = 0;
> + unsigned int i;
> +
> + for ( i = 0; i < ra->nr_entries; i++ )
> + {
> + entry = ra->entries + i;
> +
> + if ( entry->rsvd )
> + {
> + entry->u.ret = -EINVAL;
> + break;
> + }
> +
> + switch ( entry->u.cmd )
> + {
> + case XEN_RESOURCE_OP_MSR_READ:
> + case XEN_RESOURCE_OP_MSR_WRITE:
> + if ( entry->idx >> 32 )
> + ret = -EINVAL;
> + else if ( !allow_access_msr(entry->idx) )
> + ret = -EACCES;
> + break;
> + default:
> + ret = -EINVAL;
-EOPNOTSUPP or any other suitable but more specific error code
than -EINVAL.
> + break;
> + }
> +
> + if ( ret )
> + {
> + entry->u.ret = ret;
> + break;
> + }
Other than Andrew said, this is okay (thus retaining the ->u.cmd
value for the success case).
> + }
> +
> + /* Return the number of successes. */
> + return i;
> +}
> +
> +static void resource_access(void *info)
> +{
> + struct xen_resource_access *ra = info;
> + xenpf_resource_entry_t *entry;
> + int ret;
> + unsigned int i;
> +
> + for ( i = 0; i < ra->nr_entries; i++ )
> + {
> + entry = ra->entries + i;
> +
> + switch ( entry->u.cmd )
> + {
> + case XEN_RESOURCE_OP_MSR_READ:
> + ret = rdmsr_safe(entry->idx, entry->val);
> + break;
> + case XEN_RESOURCE_OP_MSR_WRITE:
> + ret = wrmsr_safe(entry->idx, entry->val);
> + break;
> + default:
> + ret = -EINVAL;
> + break;
BUG(). You checked invalid cmd-s already above.
> + }
> +
> + if ( ret )
> + {
> + entry->u.ret = ret;
> + break;
> + }
Here you indeed should update ->u.ret unconditionally if we really
want success to be indicated here too. I'm not sure this is needed
though, since the return value of the hypercall should indicate the
slot where to look for the op-specific error code.
> @@ -601,6 +689,75 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
> }
> break;
>
> + case XENPF_resource_op:
> + {
> + struct xen_resource_access ra;
> + uint32_t cpu;
> + XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
> +
> + ra.nr_entries = op->u.resource_op.nr_entries;
> + if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
> + {
> + ret = -EINVAL;
I don't think ra.nr_entries == 0 is a reason to fail the hypercall.
> + break;
> + }
> +
> + ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
> + if ( !ra.entries )
> + {
> + ret = -ENOMEM;
> + break;
> + }
> +
> + guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
> +
> + if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
> + {
> + xfree(ra.entries);
> + ret = -EFAULT;
> + break;
> + }
> +
> + /* Do sanity check earlier to omit the potential IPI overhead. */
> + if ( check_resource_access(&ra) < ra.nr_entries )
> + {
> + /* Copy the return value for failed entry. */
> + if ( __copy_to_guest_offset(guest_entries, ret,
> + ra.entries + ret, 1) )
> + ret = -EFAULT;
> + else
> + ret = 0;
This should be the index of the failed entry. I guess it would be
easier and more consistent if check_resource_access() too used
ra.ret for passing back the failed index (which btw should be
renamed to e.g. "done" - "ret" is no longer a suitable name).
> +
> + xfree(ra.entries);
> + break;
> + }
> +
> + cpu = op->u.resource_op.cpu;
> + if ( cpu == smp_processor_id() )
> + resource_access(&ra);
> + else if ( cpu_online(cpu) )
This continues to be wrong.
> + on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
> + else
> + {
> + xfree(ra.entries);
> + ret = -ENODEV;
> + break;
> + }
> +
> + /* Copy all if succeeded or up to the failed entry. */
> + if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
> + min(ra.nr_entries, ra.ret + 1)) )
I don't see a need for min() here - ra.ret mustn't be out of range.
If you're concerned, add an ASSERT().
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-09-30 11:08 ` Andrew Cooper
@ 2014-09-30 12:15 ` Jan Beulich
0 siblings, 0 replies; 31+ messages in thread
From: Jan Beulich @ 2014-09-30 12:15 UTC (permalink / raw)
To: Andrew Cooper
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, xen-devel, Chao Peng, dgdegra
>>> On 30.09.14 at 13:08, <andrew.cooper3@citrix.com> wrote:
> On 30/09/14 12:00, Andrew Cooper wrote:
>> On 30/09/14 11:49, Chao Peng wrote:
>>> + /* Do sanity check earlier to omit the potential IPI overhead. */
>>> + if ( check_resource_access(&ra) < ra.nr_entries )
>>> + {
>>> + /* Copy the return value for failed entry. */
>>> + if ( __copy_to_guest_offset(guest_entries, ret,
>>> + ra.entries + ret, 1) )
>>> + ret = -EFAULT;
>>> + else
>>> + ret = 0;
>>> +
>>> + xfree(ra.entries);
>>> + break;
>>> + }
>> This however is not ok. You have possibly signalled that entry 0 has
>> passed the permission check and entry 1 has failed the check, at which
>> point you leave entry 0 uninitialised in userspace and signal failure
>> for entry 1.
>>
>> If some MSRs pass the permission check, you should go ahead and fill
>> them in to provide the partial good data to userspace.
>
> Actually, on second thoughts this is not correct, because of the union
> of cmd and ret.
>
> I cannot see a way of making this function correctly given the union, as
> there is no way of signalling "permission ok" between
> check_resource_access() and resource_access() without clobbering the
> command.
>
> My suggestion is still as before - make use of rsvd field for ret and
> drop the union, but I would appreciate Jan's thoughts as he explicitly
> suggested the union.
See the other reply - I don't currently see the union to cause
any problem. We simply don't need to signal "permission okay"
for each individual entry, all we need is "permission okay up to
here".
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature
2014-09-30 10:49 ` [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature Chao Peng
@ 2014-09-30 13:13 ` Jan Beulich
2014-10-01 10:49 ` Chao Peng
0 siblings, 1 reply; 31+ messages in thread
From: Jan Beulich @ 2014-09-30 13:13 UTC (permalink / raw)
To: Chao Peng
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> --- /dev/null
> +++ b/xen/arch/x86/psr.c
> @@ -0,0 +1,114 @@
> +/*
> + * pqos.c: Platform Shared Resource related service for guest.
Stale filename.
> +static void __init parse_psr_param(char *s)
> +{
> + char *ss, *val_str;
> +
> + do {
> + ss = strchr(s, ',');
> + if ( ss )
> + *ss = '\0';
> +
> + val_str = strchr(s, ':');
> + if ( val_str )
> + *val_str++ = '\0';
> +
> + if ( !strcmp(s, "cmt") && ( !val_str || parse_bool(val_str) == 1 ) )
> + opt_psr |= PSR_CMT;
Do you really mean to ignore e.g. "psr=cmt:xyz"?
> +static void __init init_psr_cmt(unsigned int rmid_max)
> +{
> + unsigned int eax, ebx, ecx, edx;
> + unsigned int rmid;
> +
> + if ( !boot_cpu_has(X86_FEATURE_CMT) )
> + return;
> +
> + cpuid_count(0xf, 0, &eax, &ebx, &ecx, &edx);
> + if ( !edx )
> + return;
> +
> + psr_cmt = xzalloc(struct psr_cmt);
> + if ( !psr_cmt )
> + return;
> +
> + psr_cmt->features = edx;
> + psr_cmt->rmid_mask = ~(~0ull << get_count_order(ebx));
> + psr_cmt->rmid_max = min(rmid_max, ebx);
The value written here gets replaced further down without taking
the value computed here into account - that's likely not what you
want. The two rmid_max values being recorded are kind of
confusing anyway - do you really need both at some point (other
than here)?
> +
> + if ( psr_cmt->features & PSR_RESOURCE_TYPE_L3 )
> + {
> + cpuid_count(0xf, 1, &eax, &ebx, &ecx, &edx);
> + psr_cmt->l3.upscaling_factor = ebx;
> + psr_cmt->l3.rmid_max = ecx;
> + psr_cmt->l3.features = edx;
> + }
> +
> + psr_cmt->rmid_max = min(rmid_max, psr_cmt->l3.rmid_max);
> + psr_cmt->rmid_to_dom = xmalloc_array(domid_t, psr_cmt->rmid_max + 1);
This still degenerates to allocating zero bytes (and then corrupting
memory) when psr_cmt->rmid_max is 0xffffffff. At the very least
use 1UL as addend.
> +static int __init init_psr(void)
> +{
> + if ( opt_psr & PSR_CMT && opt_rmid_max )
Please parenthesize the &.
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -49,6 +49,7 @@
> #include <xen/cpu.h>
> #include <asm/nmi.h>
> #include <asm/alternative.h>
> +#include <asm/psr.h>
Why?
> +struct psr_cmt {
> + unsigned long rmid_mask;
When in patch 7 finally this field gets used for something it is to mask
a value read from an MSR. Hence its type should be uint64_t. And
of course it would have helped if the field got added there rather
than here (perhaps also for the l3 field below).
> + unsigned int rmid_max;
> + unsigned int features;
> + domid_t *rmid_to_dom;
> + struct psr_cmt_l3 l3;
> +};
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 06/10] x86: collect global CMT information
2014-09-30 10:49 ` [PATCH v18 06/10] x86: collect global CMT information Chao Peng
@ 2014-09-30 13:20 ` Jan Beulich
0 siblings, 0 replies; 31+ messages in thread
From: Jan Beulich @ 2014-09-30 13:20 UTC (permalink / raw)
To: Chao Peng
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> + case XEN_SYSCTL_psr_cmt_op:
> + if ( !psr_cmt_enabled() )
> + return -ENODEV;
> +
> + if ( sysctl->u.psr_cmt_op.flags != 0 )
> + return -EINVAL;
> +
> + switch ( sysctl->u.psr_cmt_op.cmd )
> + {
> + case XEN_SYSCTL_PSR_CMT_enabled:
> + sysctl->u.psr_cmt_op.data =
> + (psr_cmt->features & PSR_RESOURCE_TYPE_L3) &&
> + (psr_cmt->l3.features & PSR_CMT_L3_OCCUPANCY);
> + break;
> + case XEN_SYSCTL_PSR_CMT_get_total_rmid:
> + sysctl->u.psr_cmt_op.data = psr_cmt->rmid_max;
> + break;
> + case XEN_SYSCTL_PSR_CMT_get_l3_upscaling_factor:
> + sysctl->u.psr_cmt_op.data = psr_cmt->l3.upscaling_factor;
> + break;
> + case XEN_SYSCTL_PSR_CMT_get_l3_cache_size:
> + {
> + struct cpuid4_info info;
> +
> + ret = cpuid4_cache_lookup(3, &info);
> + if ( ret < 0 )
> + break;
> +
> + sysctl->u.psr_cmt_op.data = info.size / 1024; /* in KB unit */
> + }
Now that I look at this again - is this really a useful operation without
specifying a socket of CPU number?
> + break;
> + default:
Please move the break inside the braces or separate it by a blank
line from "default:".
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-09-30 10:49 ` [PATCH v18 08/10] x86: add CMT related MSRs in allowed list Chao Peng
@ 2014-09-30 13:26 ` Jan Beulich
2014-10-01 10:59 ` Chao Peng
2014-10-06 14:01 ` Chao Peng
0 siblings, 2 replies; 31+ messages in thread
From: Jan Beulich @ 2014-09-30 13:26 UTC (permalink / raw)
To: Chao Peng
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> --- a/xen/include/asm-x86/msr-index.h
> +++ b/xen/include/asm-x86/msr-index.h
> @@ -324,6 +324,8 @@
> #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
>
> /* Platform Shared Resource MSRs */
> +#define MSR_IA32_QOSEVTSEL 0x00000c8d
Is this still an appropriate name with then name change "QOS" -> "CMT"?
> +#define MSR_IA32_QMC 0x00000c8e
The Q here may similarly be questionable now.
> #define MSR_IA32_PQR_ASSOC 0x00000c8f
And maybe even the one here?
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-09-30 12:12 ` Jan Beulich
@ 2014-10-01 10:04 ` Chao Peng
2014-10-01 10:33 ` Jan Beulich
0 siblings, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-10-01 10:04 UTC (permalink / raw)
To: Jan Beulich, andrew.cooper3
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, xen-devel, dgdegra
On Tue, Sep 30, 2014 at 01:12:42PM +0100, Jan Beulich wrote:
> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> > +static unsigned int check_resource_access(struct xen_resource_access *ra)
> > +{
> > + xenpf_resource_entry_t *entry;
> > + int ret = 0;
> > + unsigned int i;
> > +
> > + for ( i = 0; i < ra->nr_entries; i++ )
> > + {
> > + entry = ra->entries + i;
> > +
> > + if ( entry->rsvd )
> > + {
> > + entry->u.ret = -EINVAL;
> > + break;
> > + }
> > +
> > + switch ( entry->u.cmd )
> > + {
> > + case XEN_RESOURCE_OP_MSR_READ:
> > + case XEN_RESOURCE_OP_MSR_WRITE:
> > + if ( entry->idx >> 32 )
> > + ret = -EINVAL;
> > + else if ( !allow_access_msr(entry->idx) )
> > + ret = -EACCES;
> > + break;
> > + default:
> > + ret = -EINVAL;
>
> -EOPNOTSUPP or any other suitable but more specific error code
> than -EINVAL.
Thanks Jan.
>
> > + break;
> > + }
> > +
> > + if ( ret )
> > + {
> > + entry->u.ret = ret;
> > + break;
> > + }
>
> Other than Andrew said, this is okay (thus retaining the ->u.cmd
> value for the success case).
>
> > + }
> > +
> > + /* Return the number of successes. */
> > + return i;
> > +}
> > +
> > +static void resource_access(void *info)
> > +{
> > + struct xen_resource_access *ra = info;
> > + xenpf_resource_entry_t *entry;
> > + int ret;
> > + unsigned int i;
> > +
> > + for ( i = 0; i < ra->nr_entries; i++ )
> > + {
> > + entry = ra->entries + i;
> > +
> > + switch ( entry->u.cmd )
> > + {
> > + case XEN_RESOURCE_OP_MSR_READ:
> > + ret = rdmsr_safe(entry->idx, entry->val);
> > + break;
> > + case XEN_RESOURCE_OP_MSR_WRITE:
> > + ret = wrmsr_safe(entry->idx, entry->val);
> > + break;
> > + default:
> > + ret = -EINVAL;
> > + break;
>
> BUG(). You checked invalid cmd-s already above.
OK
>
> > + }
> > +
> > + if ( ret )
> > + {
> > + entry->u.ret = ret;
> > + break;
> > + }
>
> Here you indeed should update ->u.ret unconditionally if we really
> want success to be indicated here too. I'm not sure this is needed
> though, since the return value of the hypercall should indicate the
> slot where to look for the op-specific error code.
I didn't update ->u.ret for succesful one is what I wanted to keep consistent
with check_resource_access().
>
> > @@ -601,6 +689,75 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)
> > }
> > break;
> >
> > + case XENPF_resource_op:
> > + {
> > + struct xen_resource_access ra;
> > + uint32_t cpu;
> > + XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
> > +
> > + ra.nr_entries = op->u.resource_op.nr_entries;
> > + if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
> > + {
> > + ret = -EINVAL;
>
> I don't think ra.nr_entries == 0 is a reason to fail the hypercall.
Do you mean 'ret = 0' ?
>
> > + break;
> > + }
> > +
> > + ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
> > + if ( !ra.entries )
> > + {
> > + ret = -ENOMEM;
> > + break;
> > + }
> > +
> > + guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
> > +
> > + if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
> > + {
> > + xfree(ra.entries);
> > + ret = -EFAULT;
> > + break;
> > + }
> > +
> > + /* Do sanity check earlier to omit the potential IPI overhead. */
> > + if ( check_resource_access(&ra) < ra.nr_entries )
> > + {
> > + /* Copy the return value for failed entry. */
> > + if ( __copy_to_guest_offset(guest_entries, ret,
> > + ra.entries + ret, 1) )
> > + ret = -EFAULT;
> > + else
> > + ret = 0;
>
> This should be the index of the failed entry. I guess it would be
> easier and more consistent if check_resource_access() too used
> ra.ret for passing back the failed index (which btw should be
> renamed to e.g. "done" - "ret" is no longer a suitable name).
I agree to use ra.ret for check_resource_access(). But I insist that return 0
here is more reasonable. As we use positive return value to indicate the
number of successful operations. But here we just passed some check and
have even not performed the access. Return index of the failed entry
will lead the caller to think the data for entries earlier is valid.
>
> > +
> > + xfree(ra.entries);
> > + break;
> > + }
> > +
> > + cpu = op->u.resource_op.cpu;
> > + if ( cpu == smp_processor_id() )
> > + resource_access(&ra);
> > + else if ( cpu_online(cpu) )
>
> This continues to be wrong.
It's not lucky that I had sent out v18 before I saw your guys discussion.
I will use Andrew's suggestion.
>
> > + on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
> > + else
> > + {
> > + xfree(ra.entries);
> > + ret = -ENODEV;
> > + break;
> > + }
> > +
> > + /* Copy all if succeeded or up to the failed entry. */
> > + if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
> > + min(ra.nr_entries, ra.ret + 1)) )
>
> I don't see a need for min() here - ra.ret mustn't be out of range.
> If you're concerned, add an ASSERT().
For fully-succeeded case, ra.ret will be ra.nr_entries and ra.ret + 1 is out
of range.
Chao
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-10-01 10:04 ` Chao Peng
@ 2014-10-01 10:33 ` Jan Beulich
2014-10-02 10:26 ` Chao Peng
0 siblings, 1 reply; 31+ messages in thread
From: Jan Beulich @ 2014-10-01 10:33 UTC (permalink / raw)
To: Chao Peng
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
>>> On 01.10.14 at 12:04, <chao.p.peng@linux.intel.com> wrote:
> On Tue, Sep 30, 2014 at 01:12:42PM +0100, Jan Beulich wrote:
>> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
>> > + case XENPF_resource_op:
>> > + {
>> > + struct xen_resource_access ra;
>> > + uint32_t cpu;
>> > + XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
>> > +
>> > + ra.nr_entries = op->u.resource_op.nr_entries;
>> > + if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
>> > + {
>> > + ret = -EINVAL;
>>
>> I don't think ra.nr_entries == 0 is a reason to fail the hypercall.
> Do you mean 'ret = 0' ?
Sure.
>> > + break;
>> > + }
>> > +
>> > + ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
>> > + if ( !ra.entries )
>> > + {
>> > + ret = -ENOMEM;
>> > + break;
>> > + }
>> > +
>> > + guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
>> > +
>> > + if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
>> > + {
>> > + xfree(ra.entries);
>> > + ret = -EFAULT;
>> > + break;
>> > + }
>> > +
>> > + /* Do sanity check earlier to omit the potential IPI overhead. */
>> > + if ( check_resource_access(&ra) < ra.nr_entries )
>> > + {
>> > + /* Copy the return value for failed entry. */
>> > + if ( __copy_to_guest_offset(guest_entries, ret,
>> > + ra.entries + ret, 1) )
>> > + ret = -EFAULT;
>> > + else
>> > + ret = 0;
>>
>> This should be the index of the failed entry. I guess it would be
>> easier and more consistent if check_resource_access() too used
>> ra.ret for passing back the failed index (which btw should be
>> renamed to e.g. "done" - "ret" is no longer a suitable name).
>
> I agree to use ra.ret for check_resource_access(). But I insist that return
> 0
> here is more reasonable. As we use positive return value to indicate the
> number of successful operations. But here we just passed some check and
> have even not performed the access. Return index of the failed entry
> will lead the caller to think the data for entries earlier is valid.
No - really correct behavior would be to carry out the operations
that passed the validation checks rather than exiting early here.
>> > + on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
>> > + else
>> > + {
>> > + xfree(ra.entries);
>> > + ret = -ENODEV;
>> > + break;
>> > + }
>> > +
>> > + /* Copy all if succeeded or up to the failed entry. */
>> > + if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
>> > + min(ra.nr_entries, ra.ret + 1)) )
>>
>> I don't see a need for min() here - ra.ret mustn't be out of range.
>> If you're concerned, add an ASSERT().
>
> For fully-succeeded case, ra.ret will be ra.nr_entries and ra.ret + 1 is out
> of range.
I don't think I understand what you're trying to tell me.
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature
2014-09-30 13:13 ` Jan Beulich
@ 2014-10-01 10:49 ` Chao Peng
0 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-10-01 10:49 UTC (permalink / raw)
To: Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
On Tue, Sep 30, 2014 at 02:13:18PM +0100, Jan Beulich wrote:
> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> > --- /dev/null
> > +++ b/xen/arch/x86/psr.c
> > @@ -0,0 +1,114 @@
> > +/*
> > + * pqos.c: Platform Shared Resource related service for guest.
>
> Stale filename.
Good catch, thanks.
>
> > +static void __init parse_psr_param(char *s)
> > +{
> > + char *ss, *val_str;
> > +
> > + do {
> > + ss = strchr(s, ',');
> > + if ( ss )
> > + *ss = '\0';
> > +
> > + val_str = strchr(s, ':');
> > + if ( val_str )
> > + *val_str++ = '\0';
> > +
> > + if ( !strcmp(s, "cmt") && ( !val_str || parse_bool(val_str) == 1 ) )
> > + opt_psr |= PSR_CMT;
>
> Do you really mean to ignore e.g. "psr=cmt:xyz"?
Yes, CMT is disabled silently for such case. But I feel comfortable to
add some msg like this:
if ( parse_bool(val_str) == -1 )
printk("PSR: unknown cmt value: %s - CMT disabled!\n", val_str);
>
> > +static void __init init_psr_cmt(unsigned int rmid_max)
> > +{
> > + unsigned int eax, ebx, ecx, edx;
> > + unsigned int rmid;
> > +
> > + if ( !boot_cpu_has(X86_FEATURE_CMT) )
> > + return;
> > +
> > + cpuid_count(0xf, 0, &eax, &ebx, &ecx, &edx);
> > + if ( !edx )
> > + return;
> > +
> > + psr_cmt = xzalloc(struct psr_cmt);
> > + if ( !psr_cmt )
> > + return;
> > +
> > + psr_cmt->features = edx;
> > + psr_cmt->rmid_mask = ~(~0ull << get_count_order(ebx));
> > + psr_cmt->rmid_max = min(rmid_max, ebx);
>
> The value written here gets replaced further down without taking
> the value computed here into account - that's likely not what you
> want. The two rmid_max values being recorded are kind of
> confusing anyway - do you really need both at some point (other
> than here)?
This is actually a good catch. The line below should be:
psr_cmt->rmid_max = min(psr_cmt->rmid_max, psr_cmt->l3.rmid_max);
>
> > +
> > + if ( psr_cmt->features & PSR_RESOURCE_TYPE_L3 )
> > + {
> > + cpuid_count(0xf, 1, &eax, &ebx, &ecx, &edx);
> > + psr_cmt->l3.upscaling_factor = ebx;
> > + psr_cmt->l3.rmid_max = ecx;
> > + psr_cmt->l3.features = edx;
> > + }
> > +
> > + psr_cmt->rmid_max = min(rmid_max, psr_cmt->l3.rmid_max);
> > + psr_cmt->rmid_to_dom = xmalloc_array(domid_t, psr_cmt->rmid_max + 1);
>
> This still degenerates to allocating zero bytes (and then corrupting
> memory) when psr_cmt->rmid_max is 0xffffffff. At the very least
> use 1UL as addend.
Indeed, xmalloc will not return NULL as we expected for such case.
Then the following can be picked up again:
BUG_ON(pqosm->rmid_max < 0xffffffff);
>
> > +static int __init init_psr(void)
> > +{
> > + if ( opt_psr & PSR_CMT && opt_rmid_max )
>
> Please parenthesize the &.
>
> > --- a/xen/arch/x86/setup.c
> > +++ b/xen/arch/x86/setup.c
> > @@ -49,6 +49,7 @@
> > #include <xen/cpu.h>
> > #include <asm/nmi.h>
> > #include <asm/alternative.h>
> > +#include <asm/psr.h>
>
> Why?
>
> > +struct psr_cmt {
> > + unsigned long rmid_mask;
>
> When in patch 7 finally this field gets used for something it is to mask
> a value read from an MSR. Hence its type should be uint64_t. And
> of course it would have helped if the field got added there rather
> than here (perhaps also for the l3 field below).
Will change rmid_mask to uint64_t and move it to .c file. But will leave
l3 field here as it's needed for sysctl.c.
Chao
>
> > + unsigned int rmid_max;
> > + unsigned int features;
> > + domid_t *rmid_to_dom;
> > + struct psr_cmt_l3 l3;
> > +};
>
> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-09-30 13:26 ` Jan Beulich
@ 2014-10-01 10:59 ` Chao Peng
2014-10-01 11:17 ` Jan Beulich
2014-10-06 14:01 ` Chao Peng
1 sibling, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-10-01 10:59 UTC (permalink / raw)
To: Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> > --- a/xen/include/asm-x86/msr-index.h
> > +++ b/xen/include/asm-x86/msr-index.h
> > @@ -324,6 +324,8 @@
> > #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
> >
> > /* Platform Shared Resource MSRs */
> > +#define MSR_IA32_QOSEVTSEL 0x00000c8d
>
> Is this still an appropriate name with then name change "QOS" -> "CMT"?
>
> > +#define MSR_IA32_QMC 0x00000c8e
>
> The Q here may similarly be questionable now.
>
> > #define MSR_IA32_PQR_ASSOC 0x00000c8f
>
> And maybe even the one here?
>
>
As the SDM still use the old names, I left these names unchanged so that
no surprise for someone don't know the history to look into the spec.
So it's more like a spec issue.
On the other hand, I also feel the code here looks ugly. So maybe we can
still change them to 'correct' ones?
Chao
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-10-01 10:59 ` Chao Peng
@ 2014-10-01 11:17 ` Jan Beulich
2014-10-02 10:27 ` Chao Peng
0 siblings, 1 reply; 31+ messages in thread
From: Jan Beulich @ 2014-10-01 11:17 UTC (permalink / raw)
To: Chao Peng
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
>>> On 01.10.14 at 12:59, <chao.p.peng@linux.intel.com> wrote:
> On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
>> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
>> > --- a/xen/include/asm-x86/msr-index.h
>> > +++ b/xen/include/asm-x86/msr-index.h
>> > @@ -324,6 +324,8 @@
>> > #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
>> >
>> > /* Platform Shared Resource MSRs */
>> > +#define MSR_IA32_QOSEVTSEL 0x00000c8d
>>
>> Is this still an appropriate name with then name change "QOS" -> "CMT"?
>>
>> > +#define MSR_IA32_QMC 0x00000c8e
>>
>> The Q here may similarly be questionable now.
>>
>> > #define MSR_IA32_PQR_ASSOC 0x00000c8f
>>
>> And maybe even the one here?
>>
>>
>
> As the SDM still use the old names, I left these names unchanged so that
> no surprise for someone don't know the history to look into the spec.
>
> So it's more like a spec issue.
>
> On the other hand, I also feel the code here looks ugly. So maybe we can
> still change them to 'correct' ones?
If you can first internally agree on stable names going forward?
Jan
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall
2014-10-01 10:33 ` Jan Beulich
@ 2014-10-02 10:26 ` Chao Peng
0 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-10-02 10:26 UTC (permalink / raw)
To: Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
On Wed, Oct 01, 2014 at 11:33:41AM +0100, Jan Beulich wrote:
> >>> On 01.10.14 at 12:04, <chao.p.peng@linux.intel.com> wrote:
> > On Tue, Sep 30, 2014 at 01:12:42PM +0100, Jan Beulich wrote:
> >> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> >> > + case XENPF_resource_op:
> >> > + {
> >> > + struct xen_resource_access ra;
> >> > + uint32_t cpu;
> >> > + XEN_GUEST_HANDLE(xenpf_resource_entry_t) guest_entries;
> >> > +
> >> > + ra.nr_entries = op->u.resource_op.nr_entries;
> >> > + if ( ra.nr_entries == 0 || ra.nr_entries > RESOURCE_ACCESS_MAX_ENTRIES )
> >> > + {
> >> > + ret = -EINVAL;
> >>
> >> I don't think ra.nr_entries == 0 is a reason to fail the hypercall.
> > Do you mean 'ret = 0' ?
>
> Sure.
>
> >> > + break;
> >> > + }
> >> > +
> >> > + ra.entries = xmalloc_array(xenpf_resource_entry_t, ra.nr_entries);
> >> > + if ( !ra.entries )
> >> > + {
> >> > + ret = -ENOMEM;
> >> > + break;
> >> > + }
> >> > +
> >> > + guest_from_compat_handle(guest_entries, op->u.resource_op.entries);
> >> > +
> >> > + if ( copy_from_guest(ra.entries, guest_entries, ra.nr_entries) )
> >> > + {
> >> > + xfree(ra.entries);
> >> > + ret = -EFAULT;
> >> > + break;
> >> > + }
> >> > +
> >> > + /* Do sanity check earlier to omit the potential IPI overhead. */
> >> > + if ( check_resource_access(&ra) < ra.nr_entries )
> >> > + {
> >> > + /* Copy the return value for failed entry. */
> >> > + if ( __copy_to_guest_offset(guest_entries, ret,
> >> > + ra.entries + ret, 1) )
> >> > + ret = -EFAULT;
> >> > + else
> >> > + ret = 0;
> >>
> >> This should be the index of the failed entry. I guess it would be
> >> easier and more consistent if check_resource_access() too used
> >> ra.ret for passing back the failed index (which btw should be
> >> renamed to e.g. "done" - "ret" is no longer a suitable name).
> >
> > I agree to use ra.ret for check_resource_access(). But I insist that return
> > 0
> > here is more reasonable. As we use positive return value to indicate the
> > number of successful operations. But here we just passed some check and
> > have even not performed the access. Return index of the failed entry
> > will lead the caller to think the data for entries earlier is valid.
>
> No - really correct behavior would be to carry out the operations
> that passed the validation checks rather than exiting early here.
Then make sense. Thank you for clarification.
>
> >> > + on_selected_cpus(cpumask_of(cpu), resource_access, &ra, 1);
> >> > + else
> >> > + {
> >> > + xfree(ra.entries);
> >> > + ret = -ENODEV;
> >> > + break;
> >> > + }
> >> > +
> >> > + /* Copy all if succeeded or up to the failed entry. */
> >> > + if ( __copy_to_guest_offset(guest_entries, 0, ra.entries,
> >> > + min(ra.nr_entries, ra.ret + 1)) )
> >>
> >> I don't see a need for min() here - ra.ret mustn't be out of range.
> >> If you're concerned, add an ASSERT().
> >
> > For fully-succeeded case, ra.ret will be ra.nr_entries and ra.ret + 1 is out
> > of range.
>
> I don't think I understand what you're trying to tell me.
OK, as you have said, the ra.ret is the index for the failed entry.
Here we want to copy to guest the failed entry together with the entries
before. So 'ra.ret + 1' entries should be copied.
How about all the entries are successful? In this case ra.ret will
actully be ra.nr_entries. The number we should copy here is ra.nr_entries
but not 'ra.ret + 1'.
The min() is for this purpose then.
Chao
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-10-01 11:17 ` Jan Beulich
@ 2014-10-02 10:27 ` Chao Peng
0 siblings, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-10-02 10:27 UTC (permalink / raw)
To: Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
On Wed, Oct 01, 2014 at 12:17:13PM +0100, Jan Beulich wrote:
> >>> On 01.10.14 at 12:59, <chao.p.peng@linux.intel.com> wrote:
> > On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
> >> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> >> > --- a/xen/include/asm-x86/msr-index.h
> >> > +++ b/xen/include/asm-x86/msr-index.h
> >> > @@ -324,6 +324,8 @@
> >> > #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
> >> >
> >> > /* Platform Shared Resource MSRs */
> >> > +#define MSR_IA32_QOSEVTSEL 0x00000c8d
> >>
> >> Is this still an appropriate name with then name change "QOS" -> "CMT"?
> >>
> >> > +#define MSR_IA32_QMC 0x00000c8e
> >>
> >> The Q here may similarly be questionable now.
> >>
> >> > #define MSR_IA32_PQR_ASSOC 0x00000c8f
> >>
> >> And maybe even the one here?
> >>
> >>
> >
> > As the SDM still use the old names, I left these names unchanged so that
> > no surprise for someone don't know the history to look into the spec.
> >
> > So it's more like a spec issue.
> >
> > On the other hand, I also feel the code here looks ugly. So maybe we can
> > still change them to 'correct' ones?
>
> If you can first internally agree on stable names going forward?
Yes, I'm pushing for this.
Chao
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology
2014-09-30 10:49 ` [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology Chao Peng
@ 2014-10-02 14:57 ` Konrad Rzeszutek Wilk
2014-10-02 16:49 ` Wei Liu
0 siblings, 1 reply; 31+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-10-02 14:57 UTC (permalink / raw)
To: Chao Peng, ian.jackson, ian.campbell, wei.liu2
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, JBeulich, dgdegra
On Tue, Sep 30, 2014 at 06:49:12PM +0800, Chao Peng wrote:
> Introduced some new xl commands to enable/disable Cache Monitoring
> Technology(CMT) feature.
>
> The following two commands is to attach/detach the CMT feature
> to/from a certain domain.
>
> $ xl psr-cmt-attach domid
> $ xl psr-cmt-detach domid
>
> This command is to display the CMT information, such as L3 cache
> occupancy.
>
> $ xl psr-cmt-show cache_occupancy <domid>
You need to CC the toolstack maintainers. You can
use scripts/get_maintainers.pl file to find them.
I am CC-ing them for you.
>
> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
> ---
> docs/man/xl.pod.1 | 25 +++++
> tools/libxc/Makefile | 1 +
> tools/libxc/xc_msr_x86.h | 36 ++++++++
> tools/libxc/xc_psr.c | 214 +++++++++++++++++++++++++++++++++++++++++++
> tools/libxc/xenctrl.h | 17 ++++
> tools/libxl/Makefile | 2 +-
> tools/libxl/libxl.h | 19 ++++
> tools/libxl/libxl_psr.c | 184 +++++++++++++++++++++++++++++++++++++
> tools/libxl/libxl_types.idl | 4 +
> tools/libxl/libxl_utils.c | 28 ++++++
> tools/libxl/xl.h | 3 +
> tools/libxl/xl_cmdimpl.c | 132 ++++++++++++++++++++++++++
> tools/libxl/xl_cmdtable.c | 17 ++++
> 13 files changed, 681 insertions(+), 1 deletion(-)
> create mode 100644 tools/libxc/xc_msr_x86.h
> create mode 100644 tools/libxc/xc_psr.c
> create mode 100644 tools/libxl/libxl_psr.c
>
> diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> index f1e95db..ef8cd24 100644
> --- a/docs/man/xl.pod.1
> +++ b/docs/man/xl.pod.1
> @@ -1387,6 +1387,31 @@ Load FLASK policy from the given policy file. The initial policy is provided to
> the hypervisor as a multiboot module; this command allows runtime updates to the
> policy. Loading new security policy will reset runtime changes to device labels.
>
> +=head1 Cache Monitoring Technology
> +
> +Some new hardware may offer monitoring capability in each logical processor to
> +measure specific platform shared resource metric, for example, L3 cache
> +occupancy. In Xen implementation, the monitoring granularity is domain level.
> +To monitor a specific domain, just attach the domain id with the monitoring
> +service. When the domain doesn't need to be monitored any more, detach the
> +domain id from the monitoring service.
> +
> +=over 4
> +
> +=item B<psr-cmt-attach> [I<domain-id>]
> +
> +attach: Attach the platform shared resource monitoring service to a domain.
> +
> +=item B<psr-cmt-detach> [I<domain-id>]
> +
> +detach: Detach the platform shared resource monitoring service from a domain.
> +
> +=item B<psr-cmt-show> [I<psr-monitor-type>] [I<domain-id>]
> +
> +Show monitoring data for a certain domain or all domains. Current supported
> +monitor types are:
> + - "cache-occupancy": showing the L3 cache occupancy.
> +
> =back
>
> =head1 TO BE DOCUMENTED
> diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
> index dde6109..d8cc21b 100644
> --- a/tools/libxc/Makefile
> +++ b/tools/libxc/Makefile
> @@ -35,6 +35,7 @@ CTRL_SRCS-y += xc_kexec.c
> CTRL_SRCS-y += xtl_core.c
> CTRL_SRCS-y += xtl_logger_stdio.c
> CTRL_SRCS-y += xc_resource.c
> +CTRL_SRCS-y += xc_psr.c
> CTRL_SRCS-$(CONFIG_X86) += xc_pagetab.c
> CTRL_SRCS-$(CONFIG_Linux) += xc_linux.c xc_linux_osdep.c
> CTRL_SRCS-$(CONFIG_FreeBSD) += xc_freebsd.c xc_freebsd_osdep.c
> diff --git a/tools/libxc/xc_msr_x86.h b/tools/libxc/xc_msr_x86.h
> new file mode 100644
> index 0000000..1e0ee99
> --- /dev/null
> +++ b/tools/libxc/xc_msr_x86.h
> @@ -0,0 +1,36 @@
> +/*
> + * xc_msr_x86.h
> + *
> + * MSR definition macros
> + *
> + * Copyright (C) 2014 Intel Corporation
> + * Author Dongxiao Xu <dongxiao.xu@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#ifndef XC_MSR_X86_H
> +#define XC_MSR_X86_H
> +
> +#define MSR_IA32_QOSEVTSEL 0x00000c8d
> +#define MSR_IA32_QMC 0x00000c8e
> +
> +#endif
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> new file mode 100644
> index 0000000..182ddd4
> --- /dev/null
> +++ b/tools/libxc/xc_psr.c
> @@ -0,0 +1,214 @@
> +/*
> + * xc_psr.c
> + *
> + * platform shared resource related API functions.
> + *
> + * Copyright (C) 2014 Intel Corporation
> + * Author Dongxiao Xu <dongxiao.xu@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "xc_private.h"
> +#include "xc_msr_x86.h"
> +
> +#define IA32_QM_CTR_ERROR_MASK (0x3ull << 62)
> +
> +#define EVTID_L3_OCCUPANCY 0x1
> +
> +int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid)
> +{
> + DECLARE_DOMCTL;
> +
> + domctl.cmd = XEN_DOMCTL_psr_cmt_op;
> + domctl.domain = (domid_t)domid;
> + domctl.u.psr_cmt_op.cmd = XEN_DOMCTL_PSR_CMT_OP_ATTACH;
> +
> + return do_domctl(xch, &domctl);
> +}
> +
> +int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid)
> +{
> + DECLARE_DOMCTL;
> +
> + domctl.cmd = XEN_DOMCTL_psr_cmt_op;
> + domctl.domain = (domid_t)domid;
> + domctl.u.psr_cmt_op.cmd = XEN_DOMCTL_PSR_CMT_OP_DETACH;
> +
> + return do_domctl(xch, &domctl);
> +}
> +
> +int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
> + uint32_t *rmid)
> +{
> + int rc;
> + DECLARE_DOMCTL;
> +
> + domctl.cmd = XEN_DOMCTL_psr_cmt_op;
> + domctl.domain = (domid_t)domid;
> + domctl.u.psr_cmt_op.cmd = XEN_DOMCTL_PSR_CMT_OP_QUERY_RMID;
> +
> + rc = do_domctl(xch, &domctl);
> +
> + if ( !rc )
> + *rmid = domctl.u.psr_cmt_op.data;
> +
> + return rc;
> +}
> +
> +int xc_psr_cmt_get_total_rmid(xc_interface *xch, uint32_t *total_rmid)
> +{
> + static int val = 0;
> + int rc;
> + DECLARE_SYSCTL;
> +
> + if ( val )
> + {
> + *total_rmid = val;
> + return 0;
> + }
> +
> + sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
> + sysctl.u.psr_cmt_op.cmd = XEN_SYSCTL_PSR_CMT_get_total_rmid;
> + sysctl.u.psr_cmt_op.flags = 0;
> +
> + rc = xc_sysctl(xch, &sysctl);
> + if ( !rc )
> + val = *total_rmid = sysctl.u.psr_cmt_op.data;
> +
> + return rc;
> +}
> +
> +int xc_psr_cmt_get_l3_upscaling_factor(xc_interface *xch,
> + uint32_t *upscaling_factor)
> +{
> + static int val = 0;
> + int rc;
> + DECLARE_SYSCTL;
> +
> + if ( val )
> + {
> + *upscaling_factor = val;
> + return 0;
> + }
> +
> + sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
> + sysctl.u.psr_cmt_op.cmd =
> + XEN_SYSCTL_PSR_CMT_get_l3_upscaling_factor;
> + sysctl.u.psr_cmt_op.flags = 0;
> +
> + rc = xc_sysctl(xch, &sysctl);
> + if ( !rc )
> + val = *upscaling_factor = sysctl.u.psr_cmt_op.data;
> +
> + return rc;
> +}
> +
> +int xc_psr_cmt_get_l3_cache_size(xc_interface *xch,
> + uint32_t *l3_cache_size)
> +{
> + static int val = 0;
> + int rc;
> + DECLARE_SYSCTL;
> +
> + if ( val )
> + {
> + *l3_cache_size = val;
> + return 0;
> + }
> +
> + sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
> + sysctl.u.psr_cmt_op.cmd =
> + XEN_SYSCTL_PSR_CMT_get_l3_cache_size;
> + sysctl.u.psr_cmt_op.flags = 0;
> +
> + rc = xc_sysctl(xch, &sysctl);
> + if ( !rc )
> + val = *l3_cache_size= sysctl.u.psr_cmt_op.data;
> +
> + return rc;
> +}
> +
> +int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
> + uint32_t cpu, xc_psr_cmt_type type, uint64_t *monitor_data)
> +{
> + xc_resource_op_t op;
> + xc_resource_entry_t entries[2];
> + uint32_t evtid;
> + int rc;
> +
> + switch ( type )
> + {
> + case XC_PSR_CMT_L3_OCCUPANCY:
> + evtid = EVTID_L3_OCCUPANCY;
> + break;
> + default:
> + return -1;
> + }
> +
> + entries[0].u.cmd = XEN_RESOURCE_OP_MSR_WRITE;
> + entries[0].idx = MSR_IA32_QOSEVTSEL;
> + entries[0].val = (uint64_t)rmid << 32 | evtid;
> + entries[0].rsvd = 0;
> +
> + entries[1].u.cmd = XEN_RESOURCE_OP_MSR_READ;
> + entries[1].idx = MSR_IA32_QMC;
> + entries[1].val = 0;
> + entries[1].rsvd = 0;
> +
> + op.cpu = cpu;
> + op.nr_entries = 2;
> + op.entries = entries;
> +
> + rc = xc_resource_op(xch, 1, &op);
> + if ( rc < 0 )
> + return rc;
> +
> + if ( op.result !=2 || entries[1].val & IA32_QM_CTR_ERROR_MASK )
> + return -1;
> +
> + *monitor_data = entries[1].val;
> +
> + return 0;
> +}
> +
> +int xc_psr_cmt_enabled(xc_interface *xch)
> +{
> + static int val = -1;
> + int rc;
> + DECLARE_SYSCTL;
> +
> + if ( val >= 0 )
> + return val;
> +
> + sysctl.cmd = XEN_SYSCTL_psr_cmt_op;
> + sysctl.u.psr_cmt_op.cmd = XEN_SYSCTL_PSR_CMT_enabled;
> + sysctl.u.psr_cmt_op.flags = 0;
> +
> + rc = do_sysctl(xch, &sysctl);
> + if ( !rc )
> + {
> + val = sysctl.u.psr_cmt_op.data;
> + return val;
> + }
> +
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
> index 6edb738..b0ab6f4 100644
> --- a/tools/libxc/xenctrl.h
> +++ b/tools/libxc/xenctrl.h
> @@ -2672,6 +2672,23 @@ struct xc_resource_op {
> typedef struct xc_resource_op xc_resource_op_t;
> int xc_resource_op(xc_interface *xch, uint32_t nr_ops, xc_resource_op_t *ops);
>
> +enum xc_psr_cmt_type {
> + XC_PSR_CMT_L3_OCCUPANCY,
> +};
> +typedef enum xc_psr_cmt_type xc_psr_cmt_type;
> +int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
> +int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
> +int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
> + uint32_t *rmid);
> +int xc_psr_cmt_get_total_rmid(xc_interface *xch, uint32_t *total_rmid);
> +int xc_psr_cmt_get_l3_upscaling_factor(xc_interface *xch,
> + uint32_t *upscaling_factor);
> +int xc_psr_cmt_get_l3_cache_size(xc_interface *xch,
> + uint32_t *l3_cache_size);
> +int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
> + uint32_t cpu, uint32_t psr_cmt_type, uint64_t *monitor_data);
> +int xc_psr_cmt_enabled(xc_interface *xch);
> +
> #endif /* XENCTRL_H */
>
> /*
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> index 990414b..fd9ae28 100644
> --- a/tools/libxl/Makefile
> +++ b/tools/libxl/Makefile
> @@ -43,7 +43,7 @@ LIBXL_OBJS-y += libxl_blktap2.o
> else
> LIBXL_OBJS-y += libxl_noblktap2.o
> endif
> -LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o
> +LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o libxl_psr.o
> LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_arm.o
>
> ifeq ($(CONFIG_NetBSD),y)
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index bc68cac..5acc933 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -640,6 +640,13 @@ typedef uint8_t libxl_mac[6];
> #define LIBXL_MAC_BYTES(mac) mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]
> void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, libxl_mac *src);
>
> +/*
> + * LIBXL_HAVE_PSR_CMT
> + *
> + * If this is defined, the Cache Monitoring Technology feature is supported.
> + */
> +#define LIBXL_HAVE_PSR_CMT 1
> +
> typedef char **libxl_string_list;
> void libxl_string_list_dispose(libxl_string_list *sl);
> int libxl_string_list_length(const libxl_string_list *sl);
> @@ -1380,6 +1387,18 @@ bool libxl_ms_vm_genid_is_zero(const libxl_ms_vm_genid *id);
> void libxl_ms_vm_genid_copy(libxl_ctx *ctx, libxl_ms_vm_genid *dst,
> libxl_ms_vm_genid *src);
>
> +int libxl_get_socket_cpu(libxl_ctx *ctx, uint32_t socketid);
> +
> +int libxl_psr_cmt_attach(libxl_ctx *ctx, uint32_t domid);
> +int libxl_psr_cmt_detach(libxl_ctx *ctx, uint32_t domid);
> +int libxl_psr_cmt_domain_attached(libxl_ctx *ctx, uint32_t domid);
> +int libxl_psr_cmt_enabled(libxl_ctx *ctx);
> +int libxl_psr_cmt_get_total_rmid(libxl_ctx *ctx, uint32_t *total_rmid);
> +int libxl_psr_cmt_get_l3_cache_size(libxl_ctx *ctx,
> + uint32_t *l3_cache_size);
> +int libxl_psr_cmt_get_cache_occupancy(libxl_ctx *ctx, uint32_t domid,
> + uint32_t socketid, uint32_t *l3_cache_occupancy);
> +
> /* misc */
>
> /* Each of these sets or clears the flag according to whether the
> diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> new file mode 100644
> index 0000000..5551be8
> --- /dev/null
> +++ b/tools/libxl/libxl_psr.c
> @@ -0,0 +1,184 @@
> +/*
> + * Copyright (C) 2014 Intel Corporation
> + * Author Dongxiao Xu <dongxiao.xu@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "libxl_osdeps.h" /* must come before any other headers */
> +#include "libxl_internal.h"
> +
> +
> +#define IA32_QM_CTR_ERROR_MASK (0x3ul << 62)
> +
> +static void libxl_psr_cmt_err_msg(libxl_ctx *ctx, int err)
> +{
> + GC_INIT(ctx);
> +
> + char *msg;
> +
> + switch (err) {
> + case ENOSYS:
> + msg = "unsupported operation";
> + break;
> + case ENODEV:
> + msg = "Cache Monitoring Technology is not supported in this system";
> + break;
> + case EEXIST:
> + msg = "Cache Monitoring Technology is already attached to this domain";
> + break;
> + case ENOENT:
> + msg = "Cache Monitoring Technology is not attached to this domain";
> + break;
> + case EUSERS:
> + msg = "there is no free RMID available";
> + break;
> + case ESRCH:
> + msg = "is this Domain ID valid?";
> + break;
> + case EFAULT:
> + msg = "failed to exchange data with Xen";
> + break;
> + default:
> + msg = "unknown error";
> + break;
> + }
> +
> + LOGE(ERROR, "%s", msg);
> +
> + GC_FREE;
> +}
> +
> +int libxl_psr_cmt_attach(libxl_ctx *ctx, uint32_t domid)
> +{
> + int rc;
> +
> + rc = xc_psr_cmt_attach(ctx->xch, domid);
> + if (rc < 0) {
> + libxl_psr_cmt_err_msg(ctx, errno);
> + return ERROR_FAIL;
> + }
> +
> + return 0;
> +}
> +
> +int libxl_psr_cmt_detach(libxl_ctx *ctx, uint32_t domid)
> +{
> + int rc;
> +
> + rc = xc_psr_cmt_detach(ctx->xch, domid);
> + if (rc < 0) {
> + libxl_psr_cmt_err_msg(ctx, errno);
> + return ERROR_FAIL;
> + }
> +
> + return 0;
> +}
> +
> +int libxl_psr_cmt_domain_attached(libxl_ctx *ctx, uint32_t domid)
> +{
> + int rc;
> + uint32_t rmid;
> +
> + rc = xc_psr_cmt_get_domain_rmid(ctx->xch, domid, &rmid);
> + if (rc < 0)
> + return 0;
> +
> + return !!rmid;
> +}
> +
> +int libxl_psr_cmt_enabled(libxl_ctx *ctx)
> +{
> + return xc_psr_cmt_enabled(ctx->xch);
> +}
> +
> +int libxl_psr_cmt_get_total_rmid(libxl_ctx *ctx, uint32_t *total_rmid)
> +{
> + int rc;
> +
> + rc = xc_psr_cmt_get_total_rmid(ctx->xch, total_rmid);
> + if (rc < 0) {
> + libxl_psr_cmt_err_msg(ctx, errno);
> + return ERROR_FAIL;
> + }
> +
> + return 0;
> +}
> +
> +int libxl_psr_cmt_get_l3_cache_size(libxl_ctx *ctx,
> + uint32_t *l3_cache_size)
> +{
> + int rc;
> +
> + rc = xc_psr_cmt_get_l3_cache_size(ctx->xch, l3_cache_size);
> + if (rc < 0) {
> + libxl_psr_cmt_err_msg(ctx, errno);
> + return ERROR_FAIL;
> + }
> +
> + return 0;
> +}
> +
> +int libxl_psr_cmt_get_cache_occupancy(libxl_ctx *ctx, uint32_t domid,
> + uint32_t socketid, uint32_t *l3_cache_occupancy)
> +{
> + GC_INIT(ctx);
> +
> + unsigned int rmid;
> + uint32_t upscaling_factor;
> + uint64_t monitor_data;
> + int cpu, rc;
> + xc_psr_cmt_type type;
> +
> + rc = xc_psr_cmt_get_domain_rmid(ctx->xch, domid, &rmid);
> + if (rc < 0 || rmid == 0) {
> + LOGE(ERROR, "fail to get the domain rmid, "
> + "or domain is not attached with platform QoS monitoring service");
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + cpu = libxl_get_socket_cpu(ctx, socketid);
> + if (cpu < 0) {
> + LOGE(ERROR, "failed to get socket cpu");
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + type = XC_PSR_CMT_L3_OCCUPANCY;
> + rc = xc_psr_cmt_get_data(ctx->xch, rmid, cpu, type, &monitor_data);
> + if (rc < 0) {
> + LOGE(ERROR, "failed to get monitoring data");
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + rc = xc_psr_cmt_get_l3_upscaling_factor(ctx->xch, &upscaling_factor);
> + if (rc < 0) {
> + LOGE(ERROR, "failed to get L3 upscaling factor");
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + *l3_cache_occupancy = upscaling_factor * monitor_data / 1024;
> + rc = 0;
> +out:
> + GC_FREE;
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index f1fcbc3..27a5022 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -635,3 +635,7 @@ libxl_event = Struct("event",[
> ])),
> ("domain_create_console_available", None),
> ]))])
> +
> +libxl_psr_cmt_type = Enumeration("psr_cmt_type", [
> + (1, "CACHE_OCCUPANCY"),
> + ])
> diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
> index 58df4f3..318a996 100644
> --- a/tools/libxl/libxl_utils.c
> +++ b/tools/libxl/libxl_utils.c
> @@ -1065,6 +1065,34 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf, size_t len)
> return ret;
> }
>
> +int libxl_get_socket_cpu(libxl_ctx *ctx, uint32_t socketid)
> +{
> + int i, j, cpu, nr_cpus;
> + libxl_cputopology *topology;
> + int *socket_cpus;
> +
> + topology = libxl_get_cpu_topology(ctx, &nr_cpus);
> + if (!topology)
> + return ERROR_FAIL;
> +
> + socket_cpus = malloc(sizeof(int) * nr_cpus);
> + if (!socket_cpus) {
> + libxl_cputopology_list_free(topology, nr_cpus);
> + return ERROR_FAIL;
> + }
> +
> + for (i = 0, j = 0; i < nr_cpus; i++)
> + if (topology[i].socket == socketid)
> + socket_cpus[j++] = i;
> +
> + /* load balance among cpus in the same socket. */
> + cpu = socket_cpus[rand() % j];
> +
> + free(socket_cpus);
> + libxl_cputopology_list_free(topology, nr_cpus);
> + return cpu;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
> index 10a2e66..abb1fac 100644
> --- a/tools/libxl/xl.h
> +++ b/tools/libxl/xl.h
> @@ -110,6 +110,9 @@ int main_loadpolicy(int argc, char **argv);
> int main_remus(int argc, char **argv);
> #endif
> int main_devd(int argc, char **argv);
> +int main_psr_cmt_attach(int argc, char **argv);
> +int main_psr_cmt_detach(int argc, char **argv);
> +int main_psr_cmt_show(int argc, char **argv);
>
> void help(const char *command);
>
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 698b3bc..77f19d7 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -7428,6 +7428,138 @@ out:
> return ret;
> }
>
> +static int psr_cmt_show_cache_occupancy(uint32_t domid)
> +{
> + uint32_t i, socketid, nr_sockets, total_rmid;
> + uint32_t l3_cache_size, l3_cache_occupancy;
> + libxl_physinfo info;
> + char *domain_name;
> + int rc, print_header, nr_domains;
> + libxl_dominfo *dominfo;
> +
> + if (!libxl_psr_cmt_enabled(ctx)) {
> + printf("CMT is disabled in the system\n");
> + return -1;
> + }
> +
> + rc = libxl_get_physinfo(ctx, &info);
> + if (rc < 0) {
> + printf("Failed getting physinfo, rc: %d\n", rc);
> + return -1;
> + }
> + nr_sockets = info.nr_cpus / info.threads_per_core / info.cores_per_socket;
> +
> + rc = libxl_psr_cmt_get_total_rmid(ctx, &total_rmid);
> + if (rc < 0) {
> + printf("Failed to get max RMID value\n");
> + return -1;
> + }
> +
> + rc = libxl_psr_cmt_get_l3_cache_size(ctx, &l3_cache_size);
> + if (rc < 0) {
> + printf("Failed to get system l3 cache size\n");
> + return -1;
> + }
> +
> + printf("Total RMID: %d\n", total_rmid);
> + printf("Per-Socket L3 Cache Size: %d KB\n", l3_cache_size);
> +
> + print_header = 1;
> + if (!(dominfo = libxl_list_domain(ctx, &nr_domains))) {
> + fprintf(stderr, "libxl_list_domain failed.\n");
> + return -1;
> + }
> + for (i = 0; i < nr_domains; i++) {
> + if (domid != ~0 && dominfo[i].domid != domid)
> + continue;
> + if (!libxl_psr_cmt_domain_attached(ctx, dominfo[i].domid))
> + continue;
> + if (print_header) {
> + printf("%-40s %5s", "Name", "ID");
> + for (socketid = 0; socketid < nr_sockets; socketid++)
> + printf("%14s %d", "Socket", socketid);
> + printf("\n");
> + print_header = 0;
> + }
> + domain_name = libxl_domid_to_name(ctx, dominfo[i].domid);
> + printf("%-40s %5d", domain_name, dominfo[i].domid);
> + free(domain_name);
> + for (socketid = 0; socketid < nr_sockets; socketid++) {
> + rc = libxl_psr_cmt_get_cache_occupancy(ctx, dominfo[i].domid,
> + socketid, &l3_cache_occupancy);
> + if ( !rc )
> + printf("%13u KB", l3_cache_occupancy);
> + }
> + printf("\n");
> + }
> + libxl_dominfo_list_free(dominfo, nr_domains);
> +
> + return 0;
> +}
> +
> +int main_psr_cmt_attach(int argc, char **argv)
> +{
> + uint32_t domid;
> + int opt, ret = 0;
> +
> + SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cmt-attach", 1) {
> + /* No options */
> + }
> +
> + domid = find_domain(argv[optind]);
> + ret = libxl_psr_cmt_attach(ctx, domid);
> +
> + return ret;
> +}
> +
> +int main_psr_cmt_detach(int argc, char **argv)
> +{
> + uint32_t domid;
> + int opt, ret = 0;
> +
> + SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cmt-detach", 1) {
> + /* No options */
> + }
> +
> + domid = find_domain(argv[optind]);
> + ret = libxl_psr_cmt_detach(ctx, domid);
> +
> + return ret;
> +}
> +
> +int main_psr_cmt_show(int argc, char **argv)
> +{
> + int opt, ret = 0;
> + uint32_t domid;
> + libxl_psr_cmt_type type;
> +
> + SWITCH_FOREACH_OPT(opt, "", NULL, "psr-cmt-show", 1) {
> + /* No options */
> + }
> +
> + libxl_psr_cmt_type_from_string(argv[optind], &type);
> +
> + if (optind + 1 >= argc)
> + domid = ~0;
> + else if (optind + 1 == argc - 1)
> + domid = find_domain(argv[optind + 1]);
> + else {
> + help("psr-cmt-show");
> + return 2;
> + }
> +
> + switch (type) {
> + case LIBXL_PSR_CMT_TYPE_CACHE_OCCUPANCY:
> + ret = psr_cmt_show_cache_occupancy(domid);
> + break;
> + default:
> + help("psr-cmt-show");
> + return 2;
> + }
> +
> + return ret;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
> index 35f02c4..da25c3f 100644
> --- a/tools/libxl/xl_cmdtable.c
> +++ b/tools/libxl/xl_cmdtable.c
> @@ -502,6 +502,23 @@ struct cmd_spec cmd_table[] = {
> "[options]",
> "-F Run in the foreground",
> },
> + { "psr-cmt-attach",
> + &main_psr_cmt_attach, 0, 1,
> + "Attach Cache Monitoring Technology service to a domain",
> + "<Domain>",
> + },
> + { "psr-cmt-detach",
> + &main_psr_cmt_detach, 0, 1,
> + "Detach Cache Monitoring Technology service from a domain",
> + "<Domain>",
> + },
> + { "psr-cmt-show",
> + &main_psr_cmt_show, 0, 1,
> + "Show Cache Monitoring Technology information",
> + "<PSR-CMT-Type> <Domain>",
> + "Available monitor types:\n"
> + "\"cache_occupancy\": Show L3 cache occupancy\n",
> + },
> };
>
> int cmdtable_len = sizeof(cmd_table)/sizeof(struct cmd_spec);
> --
> 1.7.9.5
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology
2014-10-02 14:57 ` Konrad Rzeszutek Wilk
@ 2014-10-02 16:49 ` Wei Liu
0 siblings, 0 replies; 31+ messages in thread
From: Wei Liu @ 2014-10-02 16:49 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: wei.liu2, ian.campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, ian.jackson, xen-devel, JBeulich, Chao Peng,
dgdegra, keir
On Thu, Oct 02, 2014 at 10:57:50AM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, Sep 30, 2014 at 06:49:12PM +0800, Chao Peng wrote:
> > Introduced some new xl commands to enable/disable Cache Monitoring
> > Technology(CMT) feature.
> >
> > The following two commands is to attach/detach the CMT feature
> > to/from a certain domain.
> >
> > $ xl psr-cmt-attach domid
> > $ xl psr-cmt-detach domid
> >
> > This command is to display the CMT information, such as L3 cache
> > occupancy.
> >
> > $ xl psr-cmt-show cache_occupancy <domid>
>
> You need to CC the toolstack maintainers. You can
> use scripts/get_maintainers.pl file to find them.
>
> I am CC-ing them for you.
>
Thanks Konrad.
> >
> > Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> > Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
> > ---
> > docs/man/xl.pod.1 | 25 +++++
> > tools/libxc/Makefile | 1 +
> > tools/libxc/xc_msr_x86.h | 36 ++++++++
> > tools/libxc/xc_psr.c | 214 +++++++++++++++++++++++++++++++++++++++++++
> > tools/libxc/xenctrl.h | 17 ++++
> > tools/libxl/Makefile | 2 +-
> > tools/libxl/libxl.h | 19 ++++
> > tools/libxl/libxl_psr.c | 184 +++++++++++++++++++++++++++++++++++++
> > tools/libxl/libxl_types.idl | 4 +
> > tools/libxl/libxl_utils.c | 28 ++++++
> > tools/libxl/xl.h | 3 +
> > tools/libxl/xl_cmdimpl.c | 132 ++++++++++++++++++++++++++
> > tools/libxl/xl_cmdtable.c | 17 ++++
> > 13 files changed, 681 insertions(+), 1 deletion(-)
> > create mode 100644 tools/libxc/xc_msr_x86.h
> > create mode 100644 tools/libxc/xc_psr.c
> > create mode 100644 tools/libxl/libxl_psr.c
> >
> > diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> > index f1e95db..ef8cd24 100644
> > --- a/docs/man/xl.pod.1
> > +++ b/docs/man/xl.pod.1
> > @@ -1387,6 +1387,31 @@ Load FLASK policy from the given policy file. The initial policy is provided to
> > the hypervisor as a multiboot module; this command allows runtime updates to the
> > policy. Loading new security policy will reset runtime changes to device labels.
> >
> > +=head1 Cache Monitoring Technology
> > +
> > +Some new hardware may offer monitoring capability in each logical processor to
You've specified some models in other patches, can you list them here as
well?
> > +measure specific platform shared resource metric, for example, L3 cache
> > +occupancy. In Xen implementation, the monitoring granularity is domain level.
> > +To monitor a specific domain, just attach the domain id with the monitoring
> > +service. When the domain doesn't need to be monitored any more, detach the
> > +domain id from the monitoring service.
> > +
> > +=over 4
> > +
> > +=item B<psr-cmt-attach> [I<domain-id>]
> > +
> > +attach: Attach the platform shared resource monitoring service to a domain.
> > +
> > +=item B<psr-cmt-detach> [I<domain-id>]
> > +
> > +detach: Detach the platform shared resource monitoring service from a domain.
> > +
> > +=item B<psr-cmt-show> [I<psr-monitor-type>] [I<domain-id>]
> > +
> > +Show monitoring data for a certain domain or all domains. Current supported
> > +monitor types are:
> > + - "cache-occupancy": showing the L3 cache occupancy.
> > +
> > =back
> >
> > =head1 TO BE DOCUMENTED
> > diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
> > index dde6109..d8cc21b 100644
> > --- a/tools/libxc/Makefile
> > +++ b/tools/libxc/Makefile
> > @@ -35,6 +35,7 @@ CTRL_SRCS-y += xc_kexec.c
> > CTRL_SRCS-y += xtl_core.c
> > CTRL_SRCS-y += xtl_logger_stdio.c
> > CTRL_SRCS-y += xc_resource.c
> > +CTRL_SRCS-y += xc_psr.c
Shouldn't this be CTRL_SRCS-$(CONFIG_X86)?
Since you've done it for libxl changes I'm assuming it is.
> > CTRL_SRCS-$(CONFIG_X86) += xc_pagetab.c
> > CTRL_SRCS-$(CONFIG_Linux) += xc_linux.c xc_linux_osdep.c
> > CTRL_SRCS-$(CONFIG_FreeBSD) += xc_freebsd.c xc_freebsd_osdep.c
> > diff --git a/tools/libxc/xc_msr_x86.h b/tools/libxc/xc_msr_x86.h
> > new file mode 100644
> > index 0000000..1e0ee99
> > --- /dev/null
> > +++ b/tools/libxc/xc_msr_x86.h
> > @@ -0,0 +1,36 @@
> > +/*
> > + * xc_msr_x86.h
> > + *
> > + * MSR definition macros
> > + *
> > + * Copyright (C) 2014 Intel Corporation
> > + * Author Dongxiao Xu <dongxiao.xu@intel.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU Lesser General Public License as published
> > + * by the Free Software Foundation; version 2.1 only. with the special
> > + * exception on linking described in file LICENSE.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU Lesser General Public License for more details.
> > + */
> > +
> > +#ifndef XC_MSR_X86_H
> > +#define XC_MSR_X86_H
> > +
> > +#define MSR_IA32_QOSEVTSEL 0x00000c8d
> > +#define MSR_IA32_QMC 0x00000c8e
> > +
> > +#endif
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-file-style: "BSD"
> > + * c-basic-offset: 4
> > + * tab-width: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> > new file mode 100644
> > index 0000000..182ddd4
> > --- /dev/null
> > +++ b/tools/libxc/xc_psr.c
> > @@ -0,0 +1,214 @@
> > +/*
> > + * xc_psr.c
> > + *
Assuming this file contains mostly plumbing code and the hypervisor
interface is agreed upon, I just skim this file and don't spot obvious
problem.
> > diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
> > index 6edb738..b0ab6f4 100644
> > --- a/tools/libxc/xenctrl.h
> > +++ b/tools/libxc/xenctrl.h
> > @@ -2672,6 +2672,23 @@ struct xc_resource_op {
> > typedef struct xc_resource_op xc_resource_op_t;
> > int xc_resource_op(xc_interface *xch, uint32_t nr_ops, xc_resource_op_t *ops);
> >
> > +enum xc_psr_cmt_type {
> > + XC_PSR_CMT_L3_OCCUPANCY,
> > +};
> > +typedef enum xc_psr_cmt_type xc_psr_cmt_type;
> > +int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
> > +int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
> > +int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
> > + uint32_t *rmid);
> > +int xc_psr_cmt_get_total_rmid(xc_interface *xch, uint32_t *total_rmid);
> > +int xc_psr_cmt_get_l3_upscaling_factor(xc_interface *xch,
> > + uint32_t *upscaling_factor);
> > +int xc_psr_cmt_get_l3_cache_size(xc_interface *xch,
> > + uint32_t *l3_cache_size);
> > +int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
> > + uint32_t cpu, uint32_t psr_cmt_type, uint64_t *monitor_data);
> > +int xc_psr_cmt_enabled(xc_interface *xch);
> > +
Make them x86 only?
> > #endif /* XENCTRL_H */
> >
> > /*
> > diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> > index 990414b..fd9ae28 100644
> > --- a/tools/libxl/Makefile
> > +++ b/tools/libxl/Makefile
> > @@ -43,7 +43,7 @@ LIBXL_OBJS-y += libxl_blktap2.o
> > else
> > LIBXL_OBJS-y += libxl_noblktap2.o
> > endif
> > -LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o
> > +LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o libxl_psr.o
> > LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_arm.o
> >
> > ifeq ($(CONFIG_NetBSD),y)
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index bc68cac..5acc933 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -640,6 +640,13 @@ typedef uint8_t libxl_mac[6];
> > #define LIBXL_MAC_BYTES(mac) mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]
> > void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, libxl_mac *src);
> >
> > +/*
> > + * LIBXL_HAVE_PSR_CMT
> > + *
> > + * If this is defined, the Cache Monitoring Technology feature is supported.
> > + */
> > +#define LIBXL_HAVE_PSR_CMT 1
> > +
This should also be made x86 only.
> > typedef char **libxl_string_list;
> > void libxl_string_list_dispose(libxl_string_list *sl);
> > int libxl_string_list_length(const libxl_string_list *sl);
> > @@ -1380,6 +1387,18 @@ bool libxl_ms_vm_genid_is_zero(const libxl_ms_vm_genid *id);
> > void libxl_ms_vm_genid_copy(libxl_ctx *ctx, libxl_ms_vm_genid *dst,
> > libxl_ms_vm_genid *src);
> >
> > +int libxl_get_socket_cpu(libxl_ctx *ctx, uint32_t socketid);
> > +
This can be made internal in libxl_internal.{c,h}. Have a look at other
internal functions.
> > +int libxl_psr_cmt_attach(libxl_ctx *ctx, uint32_t domid);
> > +int libxl_psr_cmt_detach(libxl_ctx *ctx, uint32_t domid);
> > +int libxl_psr_cmt_domain_attached(libxl_ctx *ctx, uint32_t domid);
> > +int libxl_psr_cmt_enabled(libxl_ctx *ctx);
> > +int libxl_psr_cmt_get_total_rmid(libxl_ctx *ctx, uint32_t *total_rmid);
> > +int libxl_psr_cmt_get_l3_cache_size(libxl_ctx *ctx,
> > + uint32_t *l3_cache_size);
> > +int libxl_psr_cmt_get_cache_occupancy(libxl_ctx *ctx, uint32_t domid,
> > + uint32_t socketid, uint32_t *l3_cache_occupancy);
> > +
> > /* misc */
> >
> > /* Each of these sets or clears the flag according to whether the
> > diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> > new file mode 100644
> > index 0000000..5551be8
> > --- /dev/null
> > +++ b/tools/libxl/libxl_psr.c
> > @@ -0,0 +1,184 @@
> > +/*
> > + * Copyright (C) 2014 Intel Corporation
> > + * Author Dongxiao Xu <dongxiao.xu@intel.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU Lesser General Public License as published
> > + * by the Free Software Foundation; version 2.1 only. with the special
> > + * exception on linking described in file LICENSE.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU Lesser General Public License for more details.
> > + */
> > +
> > +#include "libxl_osdeps.h" /* must come before any other headers */
> > +#include "libxl_internal.h"
> > +
> > +
> > +#define IA32_QM_CTR_ERROR_MASK (0x3ul << 62)
> > +
> > +static void libxl_psr_cmt_err_msg(libxl_ctx *ctx, int err)
Internal functions are normally named libxl__FOO and just take a gc.
> > +{
> > + GC_INIT(ctx);
Redundant as this is never used.
> > +
> > + char *msg;
> > +
> > + switch (err) {
> > + case ENOSYS:
> > + msg = "unsupported operation";
> > + break;
> > + case ENODEV:
> > + msg = "Cache Monitoring Technology is not supported in this system";
> > + break;
> > + case EEXIST:
> > + msg = "Cache Monitoring Technology is already attached to this domain";
> > + break;
> > + case ENOENT:
> > + msg = "Cache Monitoring Technology is not attached to this domain";
> > + break;
> > + case EUSERS:
> > + msg = "there is no free RMID available";
> > + break;
> > + case ESRCH:
> > + msg = "is this Domain ID valid?";
Just use "Domain ID not valid".
> > + break;
> > + case EFAULT:
> > + msg = "failed to exchange data with Xen";
> > + break;
> > + default:
> > + msg = "unknown error";
> > + break;
> > + }
> > +
> > + LOGE(ERROR, "%s", msg);
> > +
If you have to do the logging here, this function should be renamed to
libxl__psc_cmt_log_err_msg.
> > + GC_FREE;
> > +}
> > +
[...]
> > +int libxl_psr_cmt_get_cache_occupancy(libxl_ctx *ctx, uint32_t domid,
> > + uint32_t socketid, uint32_t *l3_cache_occupancy)
> > +{
> > + GC_INIT(ctx);
> > +
> > + unsigned int rmid;
> > + uint32_t upscaling_factor;
> > + uint64_t monitor_data;
> > + int cpu, rc;
> > + xc_psr_cmt_type type;
> > +
> > + rc = xc_psr_cmt_get_domain_rmid(ctx->xch, domid, &rmid);
> > + if (rc < 0 || rmid == 0) {
> > + LOGE(ERROR, "fail to get the domain rmid, "
> > + "or domain is not attached with platform QoS monitoring service");
> > + rc = ERROR_FAIL;
> > + goto out;
> > + }
> > +
> > + cpu = libxl_get_socket_cpu(ctx, socketid);
> > + if (cpu < 0) {
> > + LOGE(ERROR, "failed to get socket cpu");
> > + rc = ERROR_FAIL;
> > + goto out;
> > + }
> > +
> > + type = XC_PSR_CMT_L3_OCCUPANCY;
> > + rc = xc_psr_cmt_get_data(ctx->xch, rmid, cpu, type, &monitor_data);
> > + if (rc < 0) {
> > + LOGE(ERROR, "failed to get monitoring data");
> > + rc = ERROR_FAIL;
> > + goto out;
> > + }
> > +
> > + rc = xc_psr_cmt_get_l3_upscaling_factor(ctx->xch, &upscaling_factor);
> > + if (rc < 0) {
> > + LOGE(ERROR, "failed to get L3 upscaling factor");
> > + rc = ERROR_FAIL;
> > + goto out;
> > + }
> > +
> > + *l3_cache_occupancy = upscaling_factor * monitor_data / 1024;
> > + rc = 0;
> > +out:
> > + GC_FREE;
> > + return 0;
return rc;
> > +}
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-basic-offset: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index f1fcbc3..27a5022 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -635,3 +635,7 @@ libxl_event = Struct("event",[
> > ])),
> > ("domain_create_console_available", None),
> > ]))])
> > +
> > +libxl_psr_cmt_type = Enumeration("psr_cmt_type", [
> > + (1, "CACHE_OCCUPANCY"),
> > + ])
> > diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
> > index 58df4f3..318a996 100644
> > --- a/tools/libxl/libxl_utils.c
> > +++ b/tools/libxl/libxl_utils.c
> > @@ -1065,6 +1065,34 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf, size_t len)
> > return ret;
> > }
> >
> > +int libxl_get_socket_cpu(libxl_ctx *ctx, uint32_t socketid)
To be honest from look of its name I cannot tell immediately what this
function does.
Also reading the code it does some kind of balancing that needs to be
explained.
Maybe you can tell us the purpose of this function so that we can either
have better code comment or better name.
> > +{
> > + int i, j, cpu, nr_cpus;
> > + libxl_cputopology *topology;
> > + int *socket_cpus;
> > +
> > + topology = libxl_get_cpu_topology(ctx, &nr_cpus);
You might also want to check libxl.h for paradigm on how to use a type.
Normally that means _init at the beginning then _dispose at the end.
> > + if (!topology)
> > + return ERROR_FAIL;
> > +
> > + socket_cpus = malloc(sizeof(int) * nr_cpus);
Please use libxl__malloc(gc...) and friends for all memory allocations.
> > + if (!socket_cpus) {
> > + libxl_cputopology_list_free(topology, nr_cpus);
> > + return ERROR_FAIL;
> > + }
... then you can remove these three lines.
> > +
> > + for (i = 0, j = 0; i < nr_cpus; i++)
> > + if (topology[i].socket == socketid)
> > + socket_cpus[j++] = i;
> > +
> > + /* load balance among cpus in the same socket. */
> > + cpu = socket_cpus[rand() % j];
> > +
> > + free(socket_cpus);
And remove this as well..
> > + libxl_cputopology_list_free(topology, nr_cpus);
> > + return cpu;
> > +}
> > +
> > /*
> > * Local variables:
> > * mode: C
> > diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
> > index 10a2e66..abb1fac 100644
> > --- a/tools/libxl/xl.h
> > +++ b/tools/libxl/xl.h
> > @@ -110,6 +110,9 @@ int main_loadpolicy(int argc, char **argv);
> > int main_remus(int argc, char **argv);
> > #endif
> > int main_devd(int argc, char **argv);
> > +int main_psr_cmt_attach(int argc, char **argv);
> > +int main_psr_cmt_detach(int argc, char **argv);
> > +int main_psr_cmt_show(int argc, char **argv);
> >
> > void help(const char *command);
> >
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 698b3bc..77f19d7 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -7428,6 +7428,138 @@ out:
> > return ret;
> > }
> >
> > +static int psr_cmt_show_cache_occupancy(uint32_t domid)
> > +{
> > + uint32_t i, socketid, nr_sockets, total_rmid;
> > + uint32_t l3_cache_size, l3_cache_occupancy;
> > + libxl_physinfo info;
> > + char *domain_name;
> > + int rc, print_header, nr_domains;
> > + libxl_dominfo *dominfo;
> > +
> > + if (!libxl_psr_cmt_enabled(ctx)) {
> > + printf("CMT is disabled in the system\n");
> > + return -1;
> > + }
Failure message should go to stderr, please use
fprintf(stderr, BLAH);
> > +
> > + rc = libxl_get_physinfo(ctx, &info);
Again, check libxl.h for paradigm of using a type.
Or have a look at libxl_domains() as an example.
> > + if (rc < 0) {
> > + printf("Failed getting physinfo, rc: %d\n", rc);
> > + return -1;
> > + }
> > + nr_sockets = info.nr_cpus / info.threads_per_core / info.cores_per_socket;
> > +
> > + rc = libxl_psr_cmt_get_total_rmid(ctx, &total_rmid);
> > + if (rc < 0) {
> > + printf("Failed to get max RMID value\n");
fprintf(stderr, BLAH);
> > + return -1;
> > + }
> > +
> > + rc = libxl_psr_cmt_get_l3_cache_size(ctx, &l3_cache_size);
> > + if (rc < 0) {
> > + printf("Failed to get system l3 cache size\n");
fprintf(stderr, BLAH);
> > + return -1;
> > + }
> > +
> > + printf("Total RMID: %d\n", total_rmid);
> > + printf("Per-Socket L3 Cache Size: %d KB\n", l3_cache_size);
> > +
> > + print_header = 1;
> > + if (!(dominfo = libxl_list_domain(ctx, &nr_domains))) {
> > + fprintf(stderr, "libxl_list_domain failed.\n");
> > + return -1;
> > + }
As far as I can tell, this function can be used to either list all
domains or a single one.
In the latter case you still ask for information for every living domain
on this host. This is not optimal.
> > + for (i = 0; i < nr_domains; i++) {
> > + if (domid != ~0 && dominfo[i].domid != domid)
What is this domid != ~0 for? Need a comment on this.
> > + continue;
> > + if (!libxl_psr_cmt_domain_attached(ctx, dominfo[i].domid))
> > + continue;
> > + if (print_header) {
> > + printf("%-40s %5s", "Name", "ID");
> > + for (socketid = 0; socketid < nr_sockets; socketid++)
> > + printf("%14s %d", "Socket", socketid);
> > + printf("\n");
> > + print_header = 0;
> > + }
> > + domain_name = libxl_domid_to_name(ctx, dominfo[i].domid);
> > + printf("%-40s %5d", domain_name, dominfo[i].domid);
> > + free(domain_name);
> > + for (socketid = 0; socketid < nr_sockets; socketid++) {
> > + rc = libxl_psr_cmt_get_cache_occupancy(ctx, dominfo[i].domid,
> > + socketid, &l3_cache_occupancy);
> > + if ( !rc )
Coding style. Please remove extra spaces.
> > + printf("%13u KB", l3_cache_occupancy);
> > + }
> > + printf("\n");
> > + }
> > + libxl_dominfo_list_free(dominfo, nr_domains);
> > +
> > + return 0;
> > +}
> > +
> > +int main_psr_cmt_attach(int argc, char **argv)
[...]
> > +
> > +int main_psr_cmt_detach(int argc, char **argv)
[...]
> > +int main_psr_cmt_show(int argc, char **argv)
[...]
Assuming they are mostly copy-n-paste of existing commands they look
good to me.
Wei.
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-09-30 13:26 ` Jan Beulich
2014-10-01 10:59 ` Chao Peng
@ 2014-10-06 14:01 ` Chao Peng
2014-10-06 14:06 ` Andrew Cooper
1 sibling, 1 reply; 31+ messages in thread
From: Chao Peng @ 2014-10-06 14:01 UTC (permalink / raw)
To: Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
andrew.cooper3, Ian.Jackson, xen-devel, dgdegra
On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
> >>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> > --- a/xen/include/asm-x86/msr-index.h
> > +++ b/xen/include/asm-x86/msr-index.h
> > @@ -324,6 +324,8 @@
> > #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
> >
We discussed this internally and finally we decide to continue to use
the names in current SDM. Which is slightly different with what we used
here and also different with what you suggested here.
It looks still a little confusing but these MSRs maybe used in features
than just CMT so narrowing down to CMT is not OK.
I will fix this in a new version and let you know.
Thanks,
Chao
> > /* Platform Shared Resource MSRs */
> > +#define MSR_IA32_QOSEVTSEL 0x00000c8d
Will be MSR_IA32_QM_EVTSEL
>
> Is this still an appropriate name with then name change "QOS" -> "CMT"?
>
> > +#define MSR_IA32_QMC 0x00000c8e
Will be MSR_IA32_QM_CTR
>
> The Q here may similarly be questionable now.
>
> > #define MSR_IA32_PQR_ASSOC 0x00000c8f
Stay unchange.
>
> And maybe even the one here?
>
> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-10-06 14:01 ` Chao Peng
@ 2014-10-06 14:06 ` Andrew Cooper
2014-10-06 14:23 ` Chao Peng
2014-10-08 7:58 ` Chao Peng
0 siblings, 2 replies; 31+ messages in thread
From: Andrew Cooper @ 2014-10-06 14:06 UTC (permalink / raw)
To: Chao Peng, Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, xen-devel, dgdegra
On 06/10/14 15:01, Chao Peng wrote:
> On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
>>>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
>>> --- a/xen/include/asm-x86/msr-index.h
>>> +++ b/xen/include/asm-x86/msr-index.h
>>> @@ -324,6 +324,8 @@
>>> #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
>>>
> We discussed this internally and finally we decide to continue to use
> the names in current SDM. Which is slightly different with what we used
> here and also different with what you suggested here.
>
> It looks still a little confusing but these MSRs maybe used in features
> than just CMT so narrowing down to CMT is not OK.
>
> I will fix this in a new version and let you know.
> Thanks,
> Chao
Be aware that most of your series has been committed to staging.
It will be fine to start a brand new series against staging containing
any outstanding patches, and new changes
~Andrew
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-10-06 14:06 ` Andrew Cooper
@ 2014-10-06 14:23 ` Chao Peng
2014-10-08 7:58 ` Chao Peng
1 sibling, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-10-06 14:23 UTC (permalink / raw)
To: Andrew Cooper
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, xen-devel, Jan Beulich, dgdegra
On Mon, Oct 06, 2014 at 03:06:21PM +0100, Andrew Cooper wrote:
> On 06/10/14 15:01, Chao Peng wrote:
> > On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
> >>>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> >>> --- a/xen/include/asm-x86/msr-index.h
> >>> +++ b/xen/include/asm-x86/msr-index.h
> >>> @@ -324,6 +324,8 @@
> >>> #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
> >>>
> > We discussed this internally and finally we decide to continue to use
> > the names in current SDM. Which is slightly different with what we used
> > here and also different with what you suggested here.
> >
> > It looks still a little confusing but these MSRs maybe used in features
> > than just CMT so narrowing down to CMT is not OK.
> >
> > I will fix this in a new version and let you know.
> > Thanks,
> > Chao
>
> Be aware that most of your series has been committed to staging.
>
> It will be fine to start a brand new series against staging containing
> any outstanding patches, and new changes
>
Got it, thanks Andrew.
Chao
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH v18 08/10] x86: add CMT related MSRs in allowed list
2014-10-06 14:06 ` Andrew Cooper
2014-10-06 14:23 ` Chao Peng
@ 2014-10-08 7:58 ` Chao Peng
1 sibling, 0 replies; 31+ messages in thread
From: Chao Peng @ 2014-10-08 7:58 UTC (permalink / raw)
To: Andrew Cooper, Jan Beulich
Cc: keir, Ian.Campbell, stefano.stabellini, George.Dunlap,
Ian.Jackson, xen-devel, dgdegra
On Mon, Oct 06, 2014 at 03:06:21PM +0100, Andrew Cooper wrote:
> On 06/10/14 15:01, Chao Peng wrote:
> > On Tue, Sep 30, 2014 at 02:26:35PM +0100, Jan Beulich wrote:
> >>>>> On 30.09.14 at 12:49, <chao.p.peng@linux.intel.com> wrote:
> >>> --- a/xen/include/asm-x86/msr-index.h
> >>> +++ b/xen/include/asm-x86/msr-index.h
> >>> @@ -324,6 +324,8 @@
> >>> #define MSR_IA32_ENERGY_PERF_BIAS 0x000001b0
> >>>
> > We discussed this internally and finally we decide to continue to use
> > the names in current SDM. Which is slightly different with what we used
> > here and also different with what you suggested here.
> >
> > It looks still a little confusing but these MSRs maybe used in features
> > than just CMT so narrowing down to CMT is not OK.
> >
> > I will fix this in a new version and let you know.
> > Thanks,
> > Chao
>
> Be aware that most of your series has been committed to staging.
>
> It will be fine to start a brand new series against staging containing
> any outstanding patches, and new changes
>
> ~Andrew
I'd prefer not to fix this at this time as the patch serial has already
gone into master branch. After all it'just naming issue and current names
are consistent with current CMT code. If other feature coming in the future
makes these names ambiguous then we can change at that time.
Chao
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2014-10-08 7:58 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-30 10:49 [PATCH v18 00/10] enable Cache Monitoring Technology (CMT) feature Chao Peng
2014-09-30 10:49 ` [PATCH v18 01/10] x86: add generic resource (e.g. MSR) access hypercall Chao Peng
2014-09-30 11:00 ` Andrew Cooper
2014-09-30 11:08 ` Andrew Cooper
2014-09-30 12:15 ` Jan Beulich
2014-09-30 12:12 ` Jan Beulich
2014-10-01 10:04 ` Chao Peng
2014-10-01 10:33 ` Jan Beulich
2014-10-02 10:26 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 02/10] xsm: add resource operation related xsm policy Chao Peng
2014-09-30 10:49 ` [PATCH v18 03/10] tools: provide interface for generic resource access Chao Peng
2014-09-30 10:49 ` [PATCH v18 04/10] x86: detect and initialize Cache Monitoring Technology feature Chao Peng
2014-09-30 13:13 ` Jan Beulich
2014-10-01 10:49 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 05/10] x86: dynamically attach/detach CMT service for a guest Chao Peng
2014-09-30 10:49 ` [PATCH v18 06/10] x86: collect global CMT information Chao Peng
2014-09-30 13:20 ` Jan Beulich
2014-09-30 10:49 ` [PATCH v18 07/10] x86: enable CMT for each domain RMID Chao Peng
2014-09-30 10:49 ` [PATCH v18 08/10] x86: add CMT related MSRs in allowed list Chao Peng
2014-09-30 13:26 ` Jan Beulich
2014-10-01 10:59 ` Chao Peng
2014-10-01 11:17 ` Jan Beulich
2014-10-02 10:27 ` Chao Peng
2014-10-06 14:01 ` Chao Peng
2014-10-06 14:06 ` Andrew Cooper
2014-10-06 14:23 ` Chao Peng
2014-10-08 7:58 ` Chao Peng
2014-09-30 10:49 ` [PATCH v18 09/10] xsm: add CMT related xsm policies Chao Peng
2014-09-30 10:49 ` [PATCH v18 10/10] tools: CMDs and APIs for Cache Monitoring Technology Chao Peng
2014-10-02 14:57 ` Konrad Rzeszutek Wilk
2014-10-02 16:49 ` Wei Liu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).