From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF188C433F5 for ; Tue, 7 Sep 2021 17:38:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6DA2D61100 for ; Tue, 7 Sep 2021 17:38:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6DA2D61100 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C96D56E037; Tue, 7 Sep 2021 17:38:05 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7B3006E091; Tue, 7 Sep 2021 17:38:04 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10100"; a="217115343" X-IronPort-AV: E=Sophos;i="5.85,274,1624345200"; d="scan'208";a="217115343" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Sep 2021 10:38:03 -0700 X-IronPort-AV: E=Sophos;i="5.85,274,1624345200"; d="scan'208";a="524295384" Received: from tcarvalx-mobl.ger.corp.intel.com (HELO [10.252.28.225]) ([10.252.28.225]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Sep 2021 10:38:01 -0700 To: =?UTF-8?Q?Thomas_Hellstr=c3=b6m?= , intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: maarten.lankhorst@linux.intel.com References: <20210906165515.450541-1-thomas.hellstrom@linux.intel.com> <20210906165515.450541-4-thomas.hellstrom@linux.intel.com> From: Matthew Auld Message-ID: <51111df6-bf0c-ba09-4a9a-5bd59bec5fa9@intel.com> Date: Tue, 7 Sep 2021 18:37:58 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: <20210906165515.450541-4-thomas.hellstrom@linux.intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit Subject: Re: [Intel-gfx] [PATCH v2 3/6] drm/i915 Implement LMEM backup and restore for suspend / resume X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On 06/09/2021 17:55, Thomas Hellström wrote: > Just evict unpinned objects to system. For pinned LMEM objects, > make a backup system object and blit the contents to that. > > Backup is performed in three steps, > 1: Opportunistically evict evictable objects using the gpu blitter. > 2: After gt idle, evict evictable objects using the gpu blitter. This will > be modified in an upcoming patch to backup pinned objects that are not used > by the blitter itself. > 3: Backup remaining pinned objects using memcpy. > > Also move uC suspend to after 2) to make sure we have a functional GuC > during 2) if using GuC submission. > > v2: > - Major refactor to make sure gem_exec_suspend@hang-SX subtests work, and > suspend / resume works with a slightly modified GuC submission enabling > patch series. > > Signed-off-by: Thomas Hellström > + > +static int i915_ttm_backup(struct i915_gem_apply_to_region *apply, > + struct drm_i915_gem_object *obj) > +{ > + struct i915_gem_ttm_pm_apply *pm_apply = > + container_of(apply, typeof(*pm_apply), base); > + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); > + struct ttm_buffer_object *backup_bo; > + struct drm_i915_private *i915 = > + container_of(bo->bdev, typeof(*i915), bdev); > + struct intel_memory_region *sys_region; > + struct drm_i915_gem_object *backup; > + struct ttm_operation_ctx ctx = {}; > + int err = 0; > + > + if (bo->resource->mem_type == I915_PL_SYSTEM || obj->ttm.backup) > + return 0; > + > + if (pm_apply->allow_gpu && i915_gem_object_evictable(obj)) > + return ttm_bo_validate(bo, i915_ttm_sys_placement(), &ctx); > + > + if (!pm_apply->backup_pinned) > + return 0; > + > + sys_region = i915->mm.regions[INTEL_REGION_SMEM]; > + backup = i915_gem_object_create_region(sys_region, > + obj->base.size, > + 0, 0); create_shmem()? > + if (IS_ERR(backup)) > + return PTR_ERR(backup); > + > + err = i915_gem_object_lock(backup, apply->ww); > + if (err) > + goto out_no_lock; > + > + backup_bo = i915_gem_to_ttm(backup); > + err = ttm_tt_populate(backup_bo->bdev, backup_bo->ttm, &ctx); > + if (err) > + goto out_no_populate; > + > + err = i915_gem_obj_copy_ttm(backup, obj, pm_apply->allow_gpu, false); > + GEM_WARN_ON(err); > + > + obj->ttm.backup = backup; > + return 0; > + > +out_no_populate: > + i915_gem_ww_unlock_single(backup); > +out_no_lock: > + i915_gem_object_put(backup); > + > + return err; > +} > + > +static int i915_ttm_recover(struct i915_gem_apply_to_region *apply, > + struct drm_i915_gem_object *obj) > +{ > + i915_ttm_backup_free(obj); > + return 0; > +} > + > +/** > + * i915_ttm_recover_region - Free the backup of all objects of a region > + * @mr: The memory region > + * > + * Checks all objects of a region if there is backup attached and if so > + * frees that backup. Typically this is called to recover after a partially > + * performed backup. > + */ > +void i915_ttm_recover_region(struct intel_memory_region *mr) > +{ > + static const struct i915_gem_apply_to_region_ops recover_ops = { > + .process_obj = i915_ttm_recover, > + }; > + struct i915_gem_apply_to_region apply = {.ops = &recover_ops}; > + int ret; > + > + ret = i915_gem_process_region(mr, &apply); > + GEM_WARN_ON(ret); > +} > + > +/** > + * i915_ttm_backup_region - Back up all objects of a region to smem. > + * @mr: The memory region > + * @allow_gpu: Whether to allow the gpu blitter for this backup. > + * @backup_pinned: Backup also pinned objects. > + * > + * Loops over all objects of a region and either evicts them if they are > + * evictable or backs them up using a backup object if they are pinned. > + * > + * Return: Zero on success. Negative error code on error. > + */ > +int i915_ttm_backup_region(struct intel_memory_region *mr, bool allow_gpu, > + bool backup_pinned) > +{ > + static const struct i915_gem_apply_to_region_ops backup_ops = { > + .process_obj = i915_ttm_backup, > + }; > + struct i915_gem_ttm_pm_apply pm_apply = { > + .base = {.ops = &backup_ops}, > + .allow_gpu = allow_gpu, > + .backup_pinned = backup_pinned, > + }; > + > + return i915_gem_process_region(mr, &pm_apply.base); > +} > + > +static int i915_ttm_restore(struct i915_gem_apply_to_region *apply, > + struct drm_i915_gem_object *obj) > +{ > + struct i915_gem_ttm_pm_apply *pm_apply = > + container_of(apply, typeof(*pm_apply), base); > + struct drm_i915_gem_object *backup = obj->ttm.backup; > + struct ttm_buffer_object *backup_bo = i915_gem_to_ttm(backup); > + struct ttm_operation_ctx ctx = {}; > + int err; > + > + if (!obj->ttm.backup) if (!backup) > + return 0; > + > + if (!pm_apply->allow_gpu && (obj->flags & I915_BO_ALLOC_USER)) > + return 0; > + > + err = i915_gem_object_lock(backup, apply->ww); > + if (err) > + return err; > + > + /* Content may have been swapped. */ > + err = ttm_tt_populate(backup_bo->bdev, backup_bo->ttm, &ctx); > + if (!err) { > + err = i915_gem_obj_copy_ttm(obj, backup, pm_apply->allow_gpu, > + false); > + GEM_WARN_ON(err); > + > + obj->ttm.backup = NULL; > + err = 0; > + } > + > + i915_gem_ww_unlock_single(backup); > + i915_gem_object_put(backup); I assume we need to set ttm.backup = NULL somewhere here on the failure path, or don't drop the ref? Or at least it looks like potential uaf later? > + > + return err; > +} > +