From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: [PATCH v2 0/8] enable Cache QoS Monitoring (CQM) feature Date: Thu, 21 Nov 2013 14:36:32 +0000 Message-ID: <528E1A70.1060203@citrix.com> References: <1385018444-104477-1-git-send-email-dongxiao.xu@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1385018444-104477-1-git-send-email-dongxiao.xu@intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: dongxiao.xu@intel.com Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 21/11/13 07:20, dongxiao.xu@intel.com wrote: > From: Dongxiao Xu You will need to CC the relevant maintainers. This would be Jan and Keir for the Hypervisor changes, and IanC, IanJ and Stefano for the tools changes. I would also suggest that you group the tools changes together at the end of the series rather than having them interspersed. ~Andrew > > Changes from v1: > - Address comments from Andrew Cooper, including: > * Change function names, e.g., alloc_cqm_rmid(), system_supports_cqm(), etc. > * Change some structure element order to save packing cost. > * Correct some function's return value. > * Some programming styles change. > * ... > > Future generations of Intel Xeon processor may offer monitoring capability in > each logical processor to measure specific quality-of-service metric, > for example, the Cache QoS Monitoring to get L3 cache occupancy. > Detailed information please refer to Intel SDM chapter 17.14. > > Cache QoS Monitoring provides a layer of abstraction between applications and > logical processors through the use of Resource Monitoring IDs (RMIDs). > In Xen design, each guest in the system can be assigned an RMID independently, > while RMID=0 is reserved for monitoring domains that doesn't enable CQM service. > When any of the domain's vcpu is scheduled on a logical processor, the domain's > RMID will be activated by programming the value into one specific MSR, and when > the vcpu is scheduled out, a RMID=0 will be programmed into that MSR. > The Cache QoS Hardware tracks cache utilization of memory accesses according to > the RMIDs and reports monitored data via a counter register. With this solution, > we can get the knowledge how much L3 cache is used by a certain guest. > > To attach CQM service to a certain guest, two approaches are provided: > 1) Create the guest with "pqos_cqm=1" set in configuration file. > 2) Use "xl pqos-attach cqm domid" for a running guest. > > To detached CQM service from a guest, users can: > 1) Use "xl pqos-detach cqm domid" for a running guest. > 2) Also destroying a guest will detach the CQM service. > > To get the L3 cache usage, users can use the command of: > $ xl pqos-list cqm (domid) > > The below data is just an example showing how the CQM related data is exposed to > end user. > > [root@localhost]# xl pqos-list cqm > RMID count 56 RMID available 53 > Name ID SocketID L3C_Usage SocketID L3C_Usage > Domain-0 0 0 20127744 1 25231360 > ExampleHVMDomain 1 0 3211264 1 10551296 > > Dongxiao Xu (8): > x86: detect and initialize Cache QoS Monitoring feature > x86: handle CQM resource when creating/destroying guests > tools: handle CQM resource when creating/destroying guests > x86: dynamically attach/detach CQM service for a guest > tools: dynamically attach/detach CQM service for a guest > x86: get per domain CQM information > tools: get per domain CQM information > x86: enable CQM monitoring for each domain RMID > > tools/libxc/xc_domain.c | 51 +++++++++ > tools/libxc/xenctrl.h | 14 +++ > tools/libxl/Makefile | 3 +- > tools/libxl/libxl.h | 8 ++ > tools/libxl/libxl_create.c | 3 + > tools/libxl/libxl_pqos.c | 102 +++++++++++++++++ > tools/libxl/libxl_types.idl | 1 + > tools/libxl/xl.h | 3 + > tools/libxl/xl_cmdimpl.c | 129 ++++++++++++++++++++++ > tools/libxl/xl_cmdtable.c | 15 +++ > xen/arch/x86/Makefile | 1 + > xen/arch/x86/cpu/intel.c | 6 + > xen/arch/x86/domain.c | 14 +++ > xen/arch/x86/domctl.c | 40 +++++++ > xen/arch/x86/pqos.c | 224 ++++++++++++++++++++++++++++++++++++++ > xen/arch/x86/setup.c | 3 + > xen/arch/x86/sysctl.c | 64 +++++++++++ > xen/common/domctl.c | 5 +- > xen/include/asm-x86/cpufeature.h | 1 + > xen/include/asm-x86/domain.h | 2 + > xen/include/asm-x86/msr-index.h | 5 + > xen/include/asm-x86/pqos.h | 56 ++++++++++ > xen/include/public/domctl.h | 31 ++++++ > xen/include/public/sysctl.h | 15 +++ > xen/include/xen/sched.h | 3 + > 25 files changed, 797 insertions(+), 2 deletions(-) > create mode 100644 tools/libxl/libxl_pqos.c > create mode 100644 xen/arch/x86/pqos.c > create mode 100644 xen/include/asm-x86/pqos.h >