From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DFD58CD98FD for ; Wed, 11 Oct 2023 08:22:32 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 329AA10E4C0; Wed, 11 Oct 2023 08:22:29 +0000 (UTC) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3D96610E4BB; Wed, 11 Oct 2023 08:22:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697012547; x=1728548547; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=rop2SO3ybzEk4DVuIiU1MtKYn2SMe0Tm8G/1drFuF44=; b=RjwKArkKyoJataX/EgLYTvxKNkeUV1+J/hMRIYPKflfIEtVokXqLfPix eAfaD/SIxUE6z6xk1jfH9RgKaFEQQkMFu/IRsVjwY52PM31H9R8Ms5M9W AwGBeo626AOSUHd1hS18FuBNH4mXNuUPo069nK96OIN4APf6DPayKrkzT 9stAlks8olT47OE3LYn7jQmcUInO2mgfeoTCbpRRsrh4+1gYkRVKnw9EA Yf5L9Iy+NmlDHc0T2AKSw1P4EMldSLEN4c2JBT03qqhwAHjoTAhYPbjtD SsLjxN0YQs6sAleHE4X/slBw02JLXRKEv6/s18WdlUB1nuk2L2U+Ti4mX Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10859"; a="448810102" X-IronPort-AV: E=Sophos;i="6.03,214,1694761200"; d="scan'208";a="448810102" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Oct 2023 01:22:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10859"; a="877580799" X-IronPort-AV: E=Sophos;i="6.03,214,1694761200"; d="scan'208";a="877580799" Received: from clanggaa-mobl.ger.corp.intel.com (HELO [10.249.254.169]) ([10.249.254.169]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Oct 2023 01:22:22 -0700 Message-ID: <925e4c2d15161ce8115409b6af97dab9a2996bf7.camel@linux.intel.com> From: Thomas =?ISO-8859-1?Q?Hellstr=F6m?= To: Dave Airlie , Thomas =?ISO-8859-1?Q?Hellstr=F6m?= "(Intel)" Date: Wed, 11 Oct 2023 10:22:20 +0200 In-Reply-To: References: <20230820215320.4187-1-dakr@redhat.com> <20230820215320.4187-3-dakr@redhat.com> <0c50ff22-0f11-1e27-c32e-694ce2b1e6c5@shipmail.org> <88c45fe6-0942-707c-9ea7-8486c177fcd7@shipmail.org> <736b6b6d-9e04-a27d-7d60-0c45d696b304@shipmail.org> <8a8253ae-0b85-df90-b480-64eeebfafc6d@shipmail.org> Organization: Intel Sweden AB, Registration Number: 556189-6027 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.46.4 (3.46.4-1.fc37) MIME-Version: 1.0 Subject: Re: [Nouveau] [PATCH drm-misc-next 2/3] drm/gpuva_mgr: generalize dma_resv/extobj handling and GEM validation X-BeenThere: nouveau@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Nouveau development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.brost@intel.com, sarah.walker@imgtec.com, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Liam.Howlett@oracle.com, boris.brezillon@collabora.com, donald.robson@imgtec.com, daniel@ffwll.ch, christian.koenig@amd.com, faith.ekstrand@collabora.com, bskeggs@redhat.com Errors-To: nouveau-bounces@lists.freedesktop.org Sender: "Nouveau" On Wed, 2023-10-11 at 06:23 +1000, Dave Airlie wrote: > > I think we're then optimizing for different scenarios. Our compute > > driver will use mostly external objects only, and if shared, I > > don't > > forsee them bound to many VMs. What saves us currently here is that > > in > > compute mode we only really traverse the extobj list after a > > preempt > > fence wait, or when a vm is using a new context for the first time. > > So > > vm's extobj list is pretty large. Each bo's vma list will typically > > be > > pretty small. >=20 > Can I ask why we are optimising for this userspace, this seems > incredibly broken. First Judging from the discussion with Christian this is not really uncommon. There *are* ways that we can play tricks in KMD of assorted cleverness to reduce the extobj list size, but doing that in KMD that wouldn't be much different than accepting a large extobj list size and do what we can to reduce overhead of iterating over it. Second the discussion here really was about whether we should be using a lower level lock to allow for async state updates, with a rather complex mechanism with weak reference counting and a requirement to drop the locks within the loop to avoid locking inversion. If that were a simplification with little or no overhead all fine, but IMO it's not a simplification? >=20 > We've has this sort of problem in the past with Intel letting the > tail > wag the horse, does anyone remember optimising relocations for a > userspace that didn't actually need to use relocations? >=20 > We need to ask why this userspace is doing this, can we get some > pointers to it? compute driver should have no reason to use mostly > external objects, the OpenCL and level0 APIs should be good enough to > figure this out. TBH for the compute UMD case, I'd be prepared to drop the *performance* argument of fine-grained locking the extobj list since it's really only traversed on new contexts and preemption. But as Christian mentions there might be other cases. We should perhaps figure those out and document? /Thoams >=20 > Dave. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75FBACD98FF for ; Wed, 11 Oct 2023 08:22:29 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C666610E4BB; Wed, 11 Oct 2023 08:22:28 +0000 (UTC) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3D96610E4BB; Wed, 11 Oct 2023 08:22:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697012547; x=1728548547; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=rop2SO3ybzEk4DVuIiU1MtKYn2SMe0Tm8G/1drFuF44=; b=RjwKArkKyoJataX/EgLYTvxKNkeUV1+J/hMRIYPKflfIEtVokXqLfPix eAfaD/SIxUE6z6xk1jfH9RgKaFEQQkMFu/IRsVjwY52PM31H9R8Ms5M9W AwGBeo626AOSUHd1hS18FuBNH4mXNuUPo069nK96OIN4APf6DPayKrkzT 9stAlks8olT47OE3LYn7jQmcUInO2mgfeoTCbpRRsrh4+1gYkRVKnw9EA Yf5L9Iy+NmlDHc0T2AKSw1P4EMldSLEN4c2JBT03qqhwAHjoTAhYPbjtD SsLjxN0YQs6sAleHE4X/slBw02JLXRKEv6/s18WdlUB1nuk2L2U+Ti4mX Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10859"; a="448810102" X-IronPort-AV: E=Sophos;i="6.03,214,1694761200"; d="scan'208";a="448810102" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Oct 2023 01:22:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10859"; a="877580799" X-IronPort-AV: E=Sophos;i="6.03,214,1694761200"; d="scan'208";a="877580799" Received: from clanggaa-mobl.ger.corp.intel.com (HELO [10.249.254.169]) ([10.249.254.169]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Oct 2023 01:22:22 -0700 Message-ID: <925e4c2d15161ce8115409b6af97dab9a2996bf7.camel@linux.intel.com> Subject: Re: [PATCH drm-misc-next 2/3] drm/gpuva_mgr: generalize dma_resv/extobj handling and GEM validation From: Thomas =?ISO-8859-1?Q?Hellstr=F6m?= To: Dave Airlie , Thomas =?ISO-8859-1?Q?Hellstr=F6m?= "(Intel)" Date: Wed, 11 Oct 2023 10:22:20 +0200 In-Reply-To: References: <20230820215320.4187-1-dakr@redhat.com> <20230820215320.4187-3-dakr@redhat.com> <0c50ff22-0f11-1e27-c32e-694ce2b1e6c5@shipmail.org> <88c45fe6-0942-707c-9ea7-8486c177fcd7@shipmail.org> <736b6b6d-9e04-a27d-7d60-0c45d696b304@shipmail.org> <8a8253ae-0b85-df90-b480-64eeebfafc6d@shipmail.org> Organization: Intel Sweden AB, Registration Number: 556189-6027 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.46.4 (3.46.4-1.fc37) MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.brost@intel.com, sarah.walker@imgtec.com, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Liam.Howlett@oracle.com, boris.brezillon@collabora.com, Danilo Krummrich , donald.robson@imgtec.com, christian.koenig@amd.com, faith.ekstrand@collabora.com, bskeggs@redhat.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Wed, 2023-10-11 at 06:23 +1000, Dave Airlie wrote: > > I think we're then optimizing for different scenarios. Our compute > > driver will use mostly external objects only, and if shared, I > > don't > > forsee them bound to many VMs. What saves us currently here is that > > in > > compute mode we only really traverse the extobj list after a > > preempt > > fence wait, or when a vm is using a new context for the first time. > > So > > vm's extobj list is pretty large. Each bo's vma list will typically > > be > > pretty small. >=20 > Can I ask why we are optimising for this userspace, this seems > incredibly broken. First Judging from the discussion with Christian this is not really uncommon. There *are* ways that we can play tricks in KMD of assorted cleverness to reduce the extobj list size, but doing that in KMD that wouldn't be much different than accepting a large extobj list size and do what we can to reduce overhead of iterating over it. Second the discussion here really was about whether we should be using a lower level lock to allow for async state updates, with a rather complex mechanism with weak reference counting and a requirement to drop the locks within the loop to avoid locking inversion. If that were a simplification with little or no overhead all fine, but IMO it's not a simplification? >=20 > We've has this sort of problem in the past with Intel letting the > tail > wag the horse, does anyone remember optimising relocations for a > userspace that didn't actually need to use relocations? >=20 > We need to ask why this userspace is doing this, can we get some > pointers to it? compute driver should have no reason to use mostly > external objects, the OpenCL and level0 APIs should be good enough to > figure this out. TBH for the compute UMD case, I'd be prepared to drop the *performance* argument of fine-grained locking the extobj list since it's really only traversed on new contexts and preemption. But as Christian mentions there might be other cases. We should perhaps figure those out and document? /Thoams >=20 > Dave.