From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCED2CA0EC7 for ; Thu, 29 Aug 2024 22:16:55 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5F92F10E7A3; Thu, 29 Aug 2024 22:16:55 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.b="gBCL5MEO"; dkim-atps=neutral Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2054.outbound.protection.outlook.com [40.107.96.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 10CA110E7A2 for ; Thu, 29 Aug 2024 22:16:54 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xnt+HD2YxeluysGMJ1UUAVskcyttZOT+maYYgn0QqlPy3oS9WcHQM6PoCYG22Bkjyj62TtHCaqVyF/owsFX/SuiQC1hs/LGxl0hHpiqwE+78+/iIx0DvNC8deNwjbMcZvTJETeO70vK+mWw5JlvzwqgXk7GlBZbku4WQHyMOHjOIlMAZjHtMV32mtDDur/KB+nuVecUvfJ0eTCeK1ntM//VrVRI0w+k7WWknZnjkIo6lDpylOMa5jl34KIBDwWwWwaF0ax8jO4goDu8qiiP8m57avkWBc0k4hUZrjfMfemfE0SQKJox5HJ2AQaswa7vX3Z+OUU5ROwkRwfdB0BwJtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I5NU7fgBPCdNs+W8SefwIs9D2emJCsfd+eAtwYwIIgs=; b=eGOmwr4D0+oIL4+CPZLofPqrJjUjHnlVRM9IsTd+TSvGTJNSsCS3el56sy077Xpw3+o0h8s4ecHvUEDs1458sdJ12KVghINbsv+lRFf3ozfEVA1ebelbDEGBuu2QkytGYklyAzFTV3JeVpeIkhN1dcTKtKW6jXIDtnBqChVOK9F6pcCgar7eqaWxUwurH49jq1+Ud1oidY/ap+3DovG6AdS7tV1mzDI+i4Y5yhhrCF1m8azJHmn3JGjbXycdqm7FhdIXUnzpsKMoSdpBWKxEhg8fIw/P2RoS9rn+UhC5bXTiuEb9mz2giCFyMiVfoJfxh25V4v3J5ue6Vna1eeMX8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I5NU7fgBPCdNs+W8SefwIs9D2emJCsfd+eAtwYwIIgs=; b=gBCL5MEO6MeVVnFpEDGQK36dHdcc3JsNSBp4+kFcKK76eUBhfs1O80oRX+TGToIdw7GYWtJ+VipeCJc0QoMbCCInKO9MRwTpdXf4vFms4EzmIz2cbHWznvFDuXbHIdL1AHjbjy9ZvgId0COqQMC6zi2/XgLfo9gg/0sVz4QCZ7w= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from DM4PR12MB5149.namprd12.prod.outlook.com (2603:10b6:5:390::14) by DM6PR12MB4298.namprd12.prod.outlook.com (2603:10b6:5:21e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28; Thu, 29 Aug 2024 22:16:50 +0000 Received: from DM4PR12MB5149.namprd12.prod.outlook.com ([fe80::36fa:deca:aaeb:75da]) by DM4PR12MB5149.namprd12.prod.outlook.com ([fe80::36fa:deca:aaeb:75da%4]) with mapi id 15.20.7897.027; Thu, 29 Aug 2024 22:16:50 +0000 Message-ID: <83273bfa-7bbf-4eb5-17c0-05d1fa9b72a4@amd.com> Date: Thu, 29 Aug 2024 18:16:48 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH] drm/amdkfd: restore_process_worker race with GPU reset Content-Language: en-US To: Felix Kuehling , Philip Yang , amd-gfx@lists.freedesktop.org, =?UTF-8?Q?Christian_K=c3=b6nig?= Cc: lijo.lazar@amd.com References: <20240823194901.25068-1-Philip.Yang@amd.com> <1376d3e2-f978-418d-9aee-c8282565d236@amd.com> From: Philip Yang In-Reply-To: <1376d3e2-f978-418d-9aee-c8282565d236@amd.com> Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: YT3PR01CA0111.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:85::12) To DM4PR12MB5149.namprd12.prod.outlook.com (2603:10b6:5:390::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR12MB5149:EE_|DM6PR12MB4298:EE_ X-MS-Office365-Filtering-Correlation-Id: cbfd1e33-7caf-4d9e-e7a6-08dcc878477b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?ZzlkRnlKRXRrd1ZzSmsyVEkvNDh2REYvamUxODhnaG1iV2NxdEN0WG1QYzRu?= =?utf-8?B?WjlmWnpkakJhMmFmT0wzbzdpTDh0TCtpUm5QaEd4cTdZNzErSnRXNWVCekZ6?= =?utf-8?B?Njh5Y0xpT1Zham1uVndLMmtzNXM0WklmbXRqTXdaTVhRSC9nVWczTkowcUVF?= =?utf-8?B?VFk3VWgzN1JSTGxkalc5dnBtZkpuWTNRZ0lQODA0cUU2MnVvVkgzNEk3ZzM4?= =?utf-8?B?WUVTZXJKVnllUXdUVlQ3TUpURlN2dEtZaHBOYzRJYnJkTzJ5bWhKUW16TWRK?= =?utf-8?B?QnZwb1d1QmNDOXlVOTFnMUhndEh6WmpZVStuamFHR0pidlZyR2N5djFwZU9w?= =?utf-8?B?YmRxdUR4dTY1MmxGTWU0MHkzRUViUzJtT0Q4cUpvcGVzQkpmMmJpRjN3Vkhn?= =?utf-8?B?SS82RExmR2hUMUV2YnhmdnJha1d4cUJSd3g2aHA1OXVhY3BXVXhjZHdEL2Z3?= =?utf-8?B?QVJneDRUdGRaamQyRWtUOEdVUkNEK3VveHlBZ1JGNFljaktkQTl6YU9SQldR?= =?utf-8?B?eXFlWHpVbE50QXB4d3NzTjJDUXMvWmJwQW5FVTZlTlprZXlFK3puYmIrZXRi?= =?utf-8?B?T25aSnJGMHR5QW5OMFNxOUdkcjJBaXIzcklHVGhBUzhQM0dBMDJGQ2cwY1B4?= =?utf-8?B?YkxJc0lrSGU2dG5sY3BlR3A1N0l0RDRtbkhlWlhKZ2VOUVFBMHhSRzNwRXl6?= =?utf-8?B?SzRGVy82ekoyMTY0b1JiR09MUnhPQWRSRTk1cjAyTEUrYmpoZkFCL3c5VFRI?= =?utf-8?B?TTlmRklvS0V2ZXRjajdUMS83Ly9NSjJKdzlCTlFZR1FBbEFISjVhSytPMmRx?= =?utf-8?B?dzVzOU1YVVc2NU8xb0M5d040SDBRSVNyUlB4bkMzQVZkQ05rNUdEQ0NSSjFB?= =?utf-8?B?NmpHbElHRDI0MGZKKzZNMFQ1NnZ1dDErbTRUNlhDN2Z4L2grRHpvTTkra3V2?= =?utf-8?B?SExaZG4rWFhWQXZVZFJqVWRaMHA0Zm5reWFlZG1YRnQxSy9VNnlMQW9nY1Nm?= =?utf-8?B?cy9adkhPVGkrSTM1dzJuTzdhNEFZQStOcUdCbE5ldGVsT3hFZWpGeWRyV0Vo?= =?utf-8?B?Ym1Da3AxTk5UOWtPeWJGRzR0NU5lQWYwdFdyZ0p4cFFiVkdFQlViN2daaU4w?= =?utf-8?B?TmtHSDAwR05xUlAwMTU0eTd6L2xudkRwV05sZEROZzBXZE9kSXFubzFNaUpI?= =?utf-8?B?aXZGMGFyL2ZNVWlUU1pRdjRkNlB2Z29IdXpUcWlBYVFDQ0dBOXBxcUhNUUFy?= =?utf-8?B?SEc1OElLOTlEQjJLNmpQUmZmcEJpUjd4ZDEyTWJEVUVvM2VJeFU3cCt2MHlL?= =?utf-8?B?bFcrVTBaZ0VLUGZqWEZiZ0txTXRmQ3B0eG9OUm5lTDB6blUwbGl0clUxcnR4?= =?utf-8?B?MDA0Z0lMZEhuT1ZJcVZTOXVYMXVFY0syMysyRFEzRnFZMDJkTURFN29FeW1J?= =?utf-8?B?M3FBQXlFU0FpZzlzd3ExK3R0ZUt3N2Y4VjlyVU5Xa2JqU1NzTFdvL1UzVkgz?= =?utf-8?B?RE9LNXErcGN1N0RsbDBTNmlHeGRQY1NwS0V1ZXRpQm45b0kwV0JMUERwVEIw?= =?utf-8?B?ZUhldk5vTjR4a1IxNEVCRjUzZXJLSk9qMnUvUjFEZzJxQ1NWb1hiWmd0d2Qz?= =?utf-8?B?THoxRHN3L0xmcTVJVkhJZmQxc1NmUTdYeTEvTzNMenVIRUE5aVM3OHJNQ0s5?= =?utf-8?B?UnNmL04rYjNNZHNOMU1nOU5DMTlabmRyV1dkWmJML05zbFhPT1E2b29HdzRu?= =?utf-8?B?Znl0d2FMZTBnV3JxZkNNVVA1WGp5aFRESTE4TFpIYnkwMjN2b0RXOVRDaXE5?= =?utf-8?B?SWM0cVZCLy81bGt2RjBjUT09?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR12MB5149.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b0JXcE9WbVVzTG1IS1BuTXdOaGZRbE0zUXo3cjFMTk9iN3JLOVNFTE1Zc0Uy?= =?utf-8?B?SU9DR0dGVmI3dTFLTGRrR2lEN3o5WldOb2pJWUp4bjBCcUZqTUs1ZkVhd29Y?= =?utf-8?B?YmN6REFWWmUwR1FKT3k0Yytlb1hUNXdzYnl6SFlDbDVST3BjZkhpNHNzbXRQ?= =?utf-8?B?Zm8zbFlvdm5HVjg4dlBTMWppc1lTeGVlK1l0cGxBaWRPMC85dytCU3BmZ1Ay?= =?utf-8?B?b0FLR2EwYVBmUnkyaURqaStsRG9kejNFTUl1ZWtvc1pCTW4yMG5rY3RQd0J5?= =?utf-8?B?UlBoRTNCcXgyeXZhVldEZEZ0MWo0M25lclgvZ3NpUSs5L0FFcW4xekFYc0pR?= =?utf-8?B?VERlNWhENy83R1M0Z2ZZdEZnbU1IYWhKQ2NlNnFUcDVnSVdyU3JPR3ZzWHly?= =?utf-8?B?d0paeEdFVWdxSHRMWUJEWDZjOCtTSHIyK0hVL0ZsS2NGOU1JUnE4bS83UTA2?= =?utf-8?B?OEZwQ1pKUndtNlZWdXZwRForc0kzSXZrdFo1Mm0vdFZNelRKZE9la29TdXpa?= =?utf-8?B?WVBSbFNTaGU0ejBubGUySnE1bzY5WkhIYkdzcklRQjloWXJDeE1kbllGZy9q?= =?utf-8?B?MmJZbjhvdHB6U0tsM3FyaW50bmE3dnpLa2RJQ2FRVU1UeHlBTm1lKy9EaHp5?= =?utf-8?B?eHhvcVJWYlJtY2QzakIrZndFOUhWdXFPOUxhRTUwdjNNWkw0Qi8vZlZzUEVs?= =?utf-8?B?MVExNU9OcXNsRVpwT0NIRFRuYmkwbUQ5aHZyUmloSHVXWTRuNzFVTWVaU21t?= =?utf-8?B?WjlCWVc4ZWVzS1N5bDRVU1lLaGN1ejVpZklDVjh5MXdTMzJNZ2hWRkNoSlEy?= =?utf-8?B?dUJJaDRZU2NiT0k4MWJKcWgvcmRnOXd5MFBmKzROQ0JkWGZ0QkdRckZLNjJq?= =?utf-8?B?V01IaXJGY3EwTzFwQStiMkxpNGdTWEFQUWowMk1RbzgwOXRsQkNMRytmNUd0?= =?utf-8?B?dmdhOFFHUEJTUHN4R09yaE1lcjRzQWxhNGE2cTVOSlJVcnlMMjNCM1ZYYjFJ?= =?utf-8?B?RFh1VEZRSDQxekxJR29IRFVuaU92STNsa0dJZldHSUIxNHRqOUY0TTJkL0ZS?= =?utf-8?B?aVhLSHRMb3dBOVFTOTdjNTBDRzBLUXVaQ2lWMXFiSWlZZXRBWnRkazgrTVdJ?= =?utf-8?B?dll0NWU3UzVYNWdjeFVzT1djU0RZZGg3endXS3ROZ1FxOXE2ZG44elNIVWxC?= =?utf-8?B?UU16dFVadEpwZ1llL1gxbUpFQyt5c3gyamxoM1BHTHJ3YWJEbDRyVUpQRk4x?= =?utf-8?B?TUs1UVozK2tqclVSaW1CWHBVN083Y1duSnlXWVM5c1loV0VUenpkU1UvNFY2?= =?utf-8?B?NzFKTndNc293VVlpSk1zdWtUQ0MzOWZMbmg4ZXVIY1gwS2ZMTy9UMEVRKzFp?= =?utf-8?B?OUMwcThndWVucXhKMjE3UkxKY2JkVmh6YUlkeG9UWTlCMTVjV205RS9WOHFJ?= =?utf-8?B?MlE1T1BhU3g4dmNCd1A2eVNtcUhFY0MySjFDYmpmVnc2ejBjNzg1YWlUZXhS?= =?utf-8?B?UkJCem9JUCtSS1R5OVVqZDRSQk9nVUg5b1VBQk1ESlRVRkw4N3RpY28wWm94?= =?utf-8?B?NVp6Q3RtWkdSSDNzdXdNT1I4ZXA2Z2hSK1N2Ny84ZkJFYkJOSjhTVEQ0aFVy?= =?utf-8?B?WThIejcrdUQvUDZleU9CUSttaWJWMEtaMUh0Y0x4UHFrREJiaEIrS3F2V3I5?= =?utf-8?B?ZVV3NmhJQlM4VnMzRWx0ZUZDZGY0WHJsOGMzcHB4VFNDQTdpODRwWkErbVV2?= =?utf-8?B?QVUySTlxeUJmZWZCUzhEMjcxQldXTGxPQS9EQ01jM1Y4MkNiZXlSbU1OUTdF?= =?utf-8?B?SENVaVdEN3ZvOWFoYUdxTFBnM2NXRUtGaU5yZWM4bTQ5ZWduaDMwVC9saU5K?= =?utf-8?B?M0VlTWY2WFhnRkdWdjYxMGFRWnR6eGs2aGZPSS9FRnRVNWNPL0E0alZHWkQ4?= =?utf-8?B?eFgrTk9acmpKSzJVM3lXQzFORi9qbDJpTjlBVVJaOGxWSzFiTEZLYjIrMkZV?= =?utf-8?B?MVdqdzZZcXVvZVBnN3h2bjdTckQ2QU5CRFYvWFhXRkp3eStMWGp6VUhqY3Rn?= =?utf-8?B?TSt2QncxUE5SMkNiMlkxRnpUWjhqSVY5UStIbFJlUXRPWEMwSGtaVEdJUjZW?= =?utf-8?Q?j7mg=3D?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: cbfd1e33-7caf-4d9e-e7a6-08dcc878477b X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5149.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2024 22:16:50.2188 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8ih8f6W9DoTDdzDznSdNZe61pNpY7PUPzyF3Bqb27bNp4tqebD6ujKJnO3axWIRw X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4298 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx"


On 2024-08-29 17:15, Felix Kuehling wrote:
On 2024-08-23 15:49, Philip Yang wrote:
If GPU reset kick in while KFD restore_process_worker running, this may
causes different issues, for example below rcu stall warning, because
restore work may move BOs and evict queues under VRAM pressure.

Fix this race by taking adev reset_domain read semaphore to prevent GPU
reset in restore_process_worker, the reset read semaphore can be taken
recursively if adev have multiple partitions.

Then there is live locking issue if CP hangs while
restore_process_worker runs, then GPU reset wait for semaphore to start
and restore_process_worker cannot finish to release semaphore. We need
signal eviction fence to solve the live locking if evict queue return
-ETIMEOUT (for MES path) or -ETIME (for HWS path) because CP hangs,

  amdgpu 0000:af:00.0: amdgpu: GPU reset(21) succeeded!
  rcu: INFO: rcu_sched self-detected stall on CPU

  Workqueue: kfd_restore_wq restore_process_worker [amdgpu]
  Call Trace:
   update_process_times+0x94/0xd0
  RIP: 0010:amdgpu_vm_handle_moved+0x9a/0x210 [amdgpu]
   amdgpu_amdkfd_gpuvm_restore_process_bos+0x3d6/0x7d0 [amdgpu]
   restore_process_helper+0x27/0x80 [amdgpu]

Signed-off-by: Philip Yang <Philip.Yang@amd.com>

See comments inline. I'd also like Christian to take a look at this patch since he's the expert on the reset locking stuff.


---
  drivers/gpu/drm/amd/amdkfd/kfd_process.c | 56 +++++++++++++++++++++++-
  1 file changed, 55 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index a902950cc060..53a814347522 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -35,6 +35,7 @@
  #include <linux/pm_runtime.h>
  #include "amdgpu_amdkfd.h"
  #include "amdgpu.h"
+#include "amdgpu_reset.h"
    struct mm_struct;
  @@ -1972,8 +1973,14 @@ static void evict_process_worker(struct work_struct *work)
              kfd_process_restore_queues(p);
            pr_debug("Finished evicting pasid 0x%x\n", p->pasid);
-    } else
+    } else if (ret == -ETIMEDOUT || ret == -ETIME) {
+        /* If CP hangs, signal the eviction fence, then restore_bo_worker
+         * can finish to up_read GPU reset semaphore to start GPU reset.
+         */
+        signal_eviction_fence(p);
+    } else {
          pr_err("Failed to evict queues of pasid 0x%x\n", p->pasid);
+    }
  }
    static int restore_process_helper(struct kfd_process *p)
@@ -1997,6 +2004,45 @@ static int restore_process_helper(struct kfd_process *p)
      return ret;
  }
  +/*
+ * kfd_hold_devices_reset_semaphore
+ *
+ * return:
+ *   true : hold reset domain semaphore to prevent device reset
+ *   false: one of the device is resetting or already reset
+ *
+ */
+static bool kfd_hold_devices_reset_semaphore(struct kfd_process *p)

I find the function naming of these functions (hold/unhold) a bit weird. I'd suggest kfd_process_trylock_reset_sems/kfd_process_unlock_reset_sems.
ok


+{
+    struct amdgpu_device *adev;
+    int i;
+
+    for (i = 0; i < p->n_pdds; i++) {
+        adev = p->pdds[i]->dev->adev;
+        if (!down_read_trylock(&adev->reset_domain->sem))
+            goto out_upread;
+    }
+    return true;
+
+out_upread:
+    while (i--) {
+        adev = p->pdds[i]->dev->adev;
+        up_read(&adev->reset_domain->sem);
+    }
+    return false;
+}
+
+static void kfd_unhold_devices_reset_semaphore(struct kfd_process *p)
+{
+    struct amdgpu_device *adev;
+    int i;
+
+    for (i = 0; i < p->n_pdds; i++) {
+        adev = p->pdds[i]->dev->adev;
+        up_read(&adev->reset_domain->sem);
+    }
+}
+
  static void restore_process_worker(struct work_struct *work)
  {
      struct delayed_work *dwork;
@@ -2009,6 +2055,12 @@ static void restore_process_worker(struct work_struct *work)
       * lifetime of this thread, kfd_process p will be valid
       */
      p = container_of(dwork, struct kfd_process, restore_work);
+
+    if (!kfd_hold_devices_reset_semaphore(p)) {
+        pr_debug("GPU resetting, restore bo and queue skipped\n");

Should we reschedule the restore worker to make sure it runs again after the reset is done?

After GPU mode1 reset, user processes already aborted, it is meaningless to reschedule restore worker to update GPU mapping.

Regards,

Philip


Thanks,
  Felix


+        return;
+    }
+
      pr_debug("Started restoring pasid 0x%x\n", p->pasid);
        /* Setting last_restore_timestamp before successful restoration.
@@ -2031,6 +2083,8 @@ static void restore_process_worker(struct work_struct *work)
                       msecs_to_jiffies(PROCESS_RESTORE_TIME_MS)))
              kfd_process_restore_queues(p);
      }
+
+    kfd_unhold_devices_reset_semaphore(p);
  }
    void kfd_suspend_all_processes(void)