From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chao Peng Subject: Re: [PATCH v4 12/12] docs: add xl-psr.markdown Date: Fri, 10 Apr 2015 15:45:10 +0800 Message-ID: <20150410074510.GG3417@pengc-linux.bj.intel.com> References: <1428571105-3604-1-git-send-email-chao.p.peng@linux.intel.com> <1428571105-3604-13-git-send-email-chao.p.peng@linux.intel.com> <552662A2.3050506@citrix.com> Reply-To: Chao Peng Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <552662A2.3050506@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andrew Cooper Cc: keir@xen.org, Ian.Campbell@citrix.com, stefano.stabellini@eu.citrix.com, Ian.Jackson@eu.citrix.com, xen-devel@lists.xen.org, will.auld@intel.com, JBeulich@suse.com, wei.liu2@citrix.com, dgdegra@tycho.nsa.gov List-Id: xen-devel@lists.xenproject.org On Thu, Apr 09, 2015 at 12:29:38PM +0100, Andrew Cooper wrote: > On 09/04/15 10:18, Chao Peng wrote: > > Add document to introduce basic concepts and terms in PSR family > > techonologies and the xl/libxl interfaces. > > Very nice! A few minor comments... > > > > > Signed-off-by: Chao Peng > > --- > > docs/man/xl.pod.1 | 7 +++ > > docs/misc/xl-psr.markdown | 111 ++++++++++++++++++++++++++++++++++++++++++++++ > > 2 files changed, 118 insertions(+) > > create mode 100644 docs/misc/xl-psr.markdown > > > > +## Cache Monitoring Technology (CMT) > > + > > +Cache Monitoring Technology (CMT) is a new feature available on Intel Haswell > > +and later server platforms that allows an OS or Hypervisor/VMM to determine > > +the usage of cache(currently only L3 cache supported) > > L3, or LLC? This appears to be used ambiguously, but does have a > material impact for system with L4 caches. As there is no CMT+L4 working actually, I'd keep it as L3 right now. > > > by applications running > > +on the platform. A Resource Monitoring ID (RMID) is the abstraction of the > > +application(s) that will be monitored for its cache usage. The CMT hardware > > +tracks cache utilization of memory accesses according to the RMID and reports > > +monitored data via a counter register. > > + > > +Detailed information please refer to Intel SDM chapter 17.14. > > Please put the chapter title as well, as the numbering does alter slowly > over time. Sure. > > > + > > +In Xen's implementation, each domain in the system can be assigned a RMID > > +independently, while RMID=0 is reserved for monitoring domains that doesn't > > +enable CMT service. RMID is opaque for xl/libxl and is only used in > > +hypervisor. > > + > > +### xl interfaces > > + > > +A domain is assigned a RMID implicitly by attaching it to CMT service: > > + > > +xl psr-cmt-attach domid > > + > > +After that, cache usage for the domain can be showed by: > > + > > +xl psr-cmt-show cache_occupancy > > + > > +Once monitoring is not needed any more, the domain can be detached from the > > +CMT service by: > > + > > +xl psr-cmt-detach domid > > + > > +The attaching may fail because of no free RMID available. In such case > > +unused RMID(s) can be freed by detaching corresponding domains from CMT > > +services. Maximum COS number in the system can also be obtained by: > > You have not yet introduced COS as a term. Perhaps this bit is better > moving down to the CAT section? Good catch. Thanks. > > > + > > +xl psr_cmt-show > > "psr-cmt-show" > > I am not sure how wise it is to dump information like max rmid/max cos > into cmt-show. > > Is it perhaps worth having an `xl psr-hwinfo` (or equivalent) which will > dump the hardware capabilities, per-socket limits etc, as a consise way > to obtain all relevant information? Sounds reasonable. I will do it. > > > + > > +## Memory Bandwidth Monitoring (MBM) > > + > > +Memory Bandwidth Monitoring(MBM) is a new hardware feature available on Intel > > +Broadwell and later server platforms which builds on the CMT infrastructure to > > +allow monitoring of system memory bandwidth. It introduces two new monitoring > > +event type to monitor system total/local memory bandwidth. The same RMID can > > +be used to monitor both cache usage and memory bandwidth at the same time. > > + > > +Detailed information please refer to Intel SDM chapter 17.14. > > + > > +In Xen's implementation, MBM shares the same set of underlying monitoring > > +service with CMT and can be used to monitor memory bandwidth on domain basis. > > + > > +The xl/libxl interface is the same with that of CMT. The difference is the > > +monitor type is corresponding memory monitoring type(local_mem_bandwidth/ > > +total_mem_bandwidth) but not cache_occupancy. > > + > > +## Cache Allocation Technology (CAT) > > + > > +Cache Allocation Technology (CAT) is a new feature available on Intel > > +Broadwell and later server platforms that allows an OS or Hypervisor/VMM to > > +partition cache allocation(i.e. L3 cache) based on application priority or > > +Class of Service(COS). Each COS is configured using capacity bitmasks (CBM) > > +which represent cache capacity and indicate the degree of overlap and > > +isolation between classes. System cache resource is divided into numbers of > > +minimum portions which is then made up into subset for cache partition. Each > > +portion corresponds to a bit in CBM and the set bit represents the > > +corresponding cache portion is available. > > + > > +Detailed information please refer to Intel SDM chapter 17.15. > > + > > +In Xen's implementation, CBM can be set/get with libxl/xl interfaces but COS > > Strictly speaking that should be "set/got" in english, but "configured" > would be a better alternative. Exactly, thanks. Chao