From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3DF28EB64DA for ; Fri, 7 Jul 2023 21:31:46 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F0A1510E5F0; Fri, 7 Jul 2023 21:31:45 +0000 (UTC) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9DEAF10E600 for ; Fri, 7 Jul 2023 21:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688765504; x=1720301504; h=date:message-id:from:to:cc:subject:in-reply-to: references:mime-version; bh=JbjmY5DhpUiIXU+9/cIxBayPgqA086ypMarlHN4Jgos=; b=Dj9UoWjsMcMT0Y1eVYiSkoi42oJQscpIQx41LvIVLJvGptgK9JUblsnX lymvoi7ItDVUOS50LlQytZHIj046O10VmyICfIdc0BiH6Zr9GXz+5cHS4 w548NDb4nebbLwEDsCXaofDGqU59VmfoOWW6bsUraAYFclWxvR40mntXV MGCLAQOWbcvZvoJyrCer26xiYYhOaQ639PtdINkWhmlnbBuE0Z30oe5gk 44pXogKdE30yg7fnc8yXTOX3osXLrtzEHwczyXAK+LGq8eg9wJWcAt8oT iB6GA/b0SUbTjKlpu6I+efkEBqgmckeJ/c2JN2fC+68Or83KoxQJusdjS g==; X-IronPort-AV: E=McAfee;i="6600,9927,10764"; a="367477794" X-IronPort-AV: E=Sophos;i="6.01,189,1684825200"; d="scan'208";a="367477794" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2023 14:31:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10764"; a="864687502" X-IronPort-AV: E=Sophos;i="6.01,189,1684825200"; d="scan'208";a="864687502" Received: from adixit-mobl.amr.corp.intel.com (HELO adixit-arch.intel.com) ([10.209.63.31]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2023 14:31:43 -0700 Date: Fri, 07 Jul 2023 14:25:17 -0700 Message-ID: <877crbct36.wl-ashutosh.dixit@intel.com> From: "Dixit, Ashutosh" To: "Iddamsetty, Aravind" In-Reply-To: <3c4bec6c-7af0-ade7-91bc-bbd3bfa46357@intel.com> References: <20230627122113.1472532-1-aravind.iddamsetty@intel.com> <20230627122113.1472532-3-aravind.iddamsetty@intel.com> <87mt09daqu.wl-ashutosh.dixit@intel.com> <87a5w8cvmr.wl-ashutosh.dixit@intel.com> <04ae6811-9bc1-c66d-6cb8-640bfd8a9c7b@intel.com> <878rbsckzf.wl-ashutosh.dixit@intel.com> <3c4bec6c-7af0-ade7-91bc-bbd3bfa46357@intel.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Subject: Re: [Intel-xe] [PATCH v2 2/2] drm/xe/pmu: Enable PMU interface X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Bommu Krishnaiah , intel-xe@lists.freedesktop.org, Tvrtko Ursulin Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" On Fri, 07 Jul 2023 03:42:36 -0700, Iddamsetty, Aravind wrote: > Hi Aravind, > On 07-07-2023 11:38, Dixit, Ashutosh wrote: > > On Thu, 06 Jul 2023 20:53:47 -0700, Iddamsetty, Aravind wrote: > > I will look at the timing stuff later but one further question about the > > requirement: > > > >>> Also, could you please explain where the requirement to expose these OAG > >>> group busy/free registers via the PMU is coming from? Since these are OA > >>> registers presumably they can be collected using the OA subsystem. > >> > >> L0 sysman needs this > >> https://spec.oneapi.io/level-zero/latest/sysman/api.html#zes-engine-properties-t > >> and xpumanager uses this > >> https://github.com/intel/xpumanager/blob/master/core/src/device/gpu/gpu_device.cpp > > > > So fine these are UMD requirements, but why do these quantities (everything > > in this patch) have to exposed via PMU? I could just create sysfs or an > > ioctl to provide these to userland, right? > > PMU is enhanced interface to present the metrics, it provides low > latency reads compared to sysfs Why lower latency compared to sysfs? In both cases we have user to kernel transitions and then register reads etc. > and one can read multiple events in a single shot Yes, this PMU can do and sysfs can't, though ioctl's can do this. > and it will give timestamps as well which sysfs cannot provide and which > is one of the requirements of UMD. Ioctl's can do this if implement (counter, timestamp) pairs, but I agree this may look strange so PMU does seem to have an advantage here. But are these timestamps needed? The spec talks about different timestamp bases but in this case we have already converted to ns and I am wondering if the UMD can use it's own timestamps (maybe average of the ioctl call and return from ioctl) if UMD needs timestamps. > Also UMDs/ observability tools do not want to have any open handles to > get these info so ioctl is dropped out. Why? This also I don't follow. And UMD has an perf pmu fd open. See igt@perf_pmu@module-unload e.g. which tests that module unload should fail if the perf pmu fd is open (which takes a ref count on the module). > the other motivation to use PMU in xe is the existing tools like > intel_gpu_top will work with just a minor change. Not too concerned about userspace tools. They can be changed to use a different interface. So I am still not convinced xe needs to expose a PMU interface with these sort of "software events/counters". So my question is why can't we just have an ioctl to expose these things, why PMU? Incidentally if you look at amdgpu_pmu.c, they seem to exposing some hardware sort of events through the PMU, not our kind of software stuff. Another interesting thing is if we have ftrace statements they seem to automatically be exposed by PMU (https://perf.wiki.kernel.org/index.php/Tutorial), e.g.: i915:i915_request_add [Tracepoint event] i915:i915_request_queue [Tracepoint event] i915:i915_request_retire [Tracepoint event] i915:i915_request_wait_begin [Tracepoint event] i915:i915_request_wait_end [Tracepoint event] So I am wondering if this might be an option? So anyway let's try to understand the need for the PMU interface a bit more before deciding on this. Once we introduce the interface (a) people will willy nilly start exposing random stuff through that inteface (b) same stuff will get exposed via multiple interfaces (e.g. frequency and rc6 residency in i915) etc. I am speaking on the basis of what I saw in i915. Let's see if Tvrtko responds, otherwise I will try to get him on irc or something. It will be good to have some input from maybe one of the architects too about this. Thanks. -- Ashutosh > > I had this same question about i915 PMU which was never answered. i915 PMU > > IMO does truly strange things like sample freq's every 5 ms and provides > > software averages which I thought userspace can easily do. > > that is a different thing nothing to do with PMU interface > > Thanks, > Aravind. > > > > I don't think it's the timestamps, maybe there is some convention related > > to the cpu pmu (which I am not familiar with). > > > > Let's see, maybe Tvrtko can also answer why these things were exposed via > > i915 PMU. > > > > Thanks. > > -- > > Ashutosh > > > > > >>> > >>> The i915 PMU I believe deduces busyness by sampling the RING_CTL register > >>> using a timer. So these registers look better since you can get these > >>> busyness values directly. On the other hand you can only get busyness for > >>> an engine group and things like compute seem to be missing? > >> > >> The per engine busyness is a different thing we still need that and it > >> has different implementation with GuC enabled, I believe Umesh is > >> looking into that. > >> > >> compute group will still be accounted in XE_OAG_RENDER_BUSY_FREE and > >> also under XE_OAG_RC0_ANY_ENGINE_BUSY_FREE. > >>> > >>> Also, would you know about plans to expose other kinds of busyness-es? I > >>> think we may be exposing per-VF and also per-client busyness via PMU. Not > >>> sure what else GuC can expose. Knowing all this we can better understand > >>> how these particular busyness values will be used. > >> > >> ya, that shall be coming next probably from Umesh but per client > >> busyness is through fdinfo.