From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D103EDE9BA for ; Tue, 10 Sep 2024 20:28:34 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A796910E002; Tue, 10 Sep 2024 20:28:33 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="DtGCtv5N"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id E691810E002 for ; Tue, 10 Sep 2024 20:28:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726000113; x=1757536113; h=date:message-id:from:to:cc:subject:in-reply-to: references:mime-version; bh=YxbLuEMOSyUB2k70XhYwT5z/vEHJDUKR9HkwzGoQ/NM=; b=DtGCtv5NPy7yCs0r0ktWwYjBBttOOLf05nJ6Xt+jab1c6czXBtFfe8ZD MOx0NEx03RWn2/lUQLTMgH3Mz840Mb9uFEOjKIIuAx2uL07wMSsYIrxFE xE9GtF1rOOKzgSIMsLM87vkyH0wq37PCsNrDR97M1eoAsfuhtvQS6TALM 6aurxGSj0vuotv4/k4NDe4Kh6QJ6KNzRWOAWXKuJ+F+PkNq1apDhI3Ilh Wxex7ZO4q5QGdBxFA5tMyKyEs7DFPEOyxM8Nxry6aaS24HEHNE8MnoF0L AX/a0aeHwK+NVFNCRdLtr4DU6L2NezNqQRNJBm6sb5ShWLmUk+f8RsBbe Q==; X-CSE-ConnectionGUID: cC1iiOzjRMep+Yuxiv3YMQ== X-CSE-MsgGUID: +hJ7VX00Tk6sSbji2t0tqA== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="35915353" X-IronPort-AV: E=Sophos;i="6.10,218,1719903600"; d="scan'208";a="35915353" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 13:28:33 -0700 X-CSE-ConnectionGUID: ondtBIOMRwaMtArh7FXv3A== X-CSE-MsgGUID: 90fonNSdQGG4bmuWZrTVRA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,218,1719903600"; d="scan'208";a="71529149" Received: from gcorreax-mobl1.amr.corp.intel.com (HELO adixit-arch.intel.com) ([10.125.33.211]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 13:28:32 -0700 Date: Tue, 10 Sep 2024 13:28:30 -0700 Message-ID: <87zfofz21d.wl-ashutosh.dixit@intel.com> From: "Dixit, Ashutosh" To: Matthew Brost Cc: intel-xe@lists.freedesktop.org, Umesh Nerlige Ramappa Subject: Re: [PATCH] drm/xe/oa: Insert wmb/sfence before enabling OA In-Reply-To: References: <20240910025907.2667875-1-ashutosh.dixit@intel.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" On Tue, 10 Sep 2024 09:14:11 -0700, Matthew Brost wrote: > Hi Matt, > On Mon, Sep 09, 2024 at 07:59:07PM -0700, Ashutosh Dixit wrote: > > We are occasionally seeing that OA Buffer register is not programmed (has > > value 0) when OA is enabled. This means OA has been enabled before it has > > been fully configured. Or, the register write enabling OA has overtaken > > previous OA configuration register writes. > > > > Therefore, insert a wmb/sfence to preserve OA register write ordering > > before enabling OA. > > > > v2: s/wmb()/xe_device_wmb()/ > > > > Fixes: e936f885f1e9 ("drm/xe/oa/uapi: Expose OA stream fd") > > Reported-by: Guy Zadicario > > Signed-off-by: Ashutosh Dixit > > Cc: stable@vger.kernel.org > > --- > > drivers/gpu/drm/xe/xe_oa.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c > > index 63286ed8457fa..4fb7aae37a94f 100644 > > --- a/drivers/gpu/drm/xe/xe_oa.c > > +++ b/drivers/gpu/drm/xe/xe_oa.c > > @@ -440,6 +440,9 @@ static void xe_oa_enable(struct xe_oa_stream *stream) > > val = __format_to_oactrl(format, regs->oa_ctrl_counter_select_mask) | > > __oa_ccs_select(stream) | OAG_OACONTROL_OA_COUNTER_ENABLE; > > > > + /* Flush previous writes to HW before enabling OA */ > > + xe_device_wmb(stream->oa->xe); > > + > > So is preventing the xe_mmio_write32 passing the memset in > xe_oa_init_oa_buffer. Indeed a xe_device_wmb is needed. Actually, what seems to be happening is that the final register write below which enables OA is passing previous register writes in xe_oa_init_oa_buffer() which are configuring OA. So in some environments OA gets enabled before it is fully configured (specifically the OA buffer ggtt address register set in xe_oa_init_oa_buffer() is occasionally being seen as 0). xe_device_wmb() is supposed to order these writes. (There is a spinlock in xe_oa_init_oa_buffer(), but afaik spinlock would only ensure a smp_wmb() but not a full wmb(), so a wmb() is still needed). I am still trying to verify if this patch actually fixes the issue and won't merge this till this is verified. The uncertainty I have is if I should also do a mmio read to also flush PCI write transactions in addition to the wmb(). > > I will say it a bit goofy that stream->oa_buffer.vaddr is accessed > directly in the OA code rather than using the xe_map layer which made it > a little harder to reach the above conclusion. You might want to fixup > the OA to use the xe_map layer rather than directly accessing pointers > if that is possible. Will look into this too. > > Anyways, this LGTM: > Reviewed-by: Matthew Brost > > > xe_mmio_write32(stream->gt, regs->oa_ctrl, val); > > } > > > > -- > > 2.41.0 > > Thanks. -- Ashutosh