From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2B58C433F5 for ; Thu, 17 Mar 2022 17:29:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236875AbiCQRav (ORCPT ); Thu, 17 Mar 2022 13:30:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236010AbiCQRav (ORCPT ); Thu, 17 Mar 2022 13:30:51 -0400 Received: from mail-wr1-x444.google.com (mail-wr1-x444.google.com [IPv6:2a00:1450:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8933C214FA5 for ; Thu, 17 Mar 2022 10:29:33 -0700 (PDT) Received: by mail-wr1-x444.google.com with SMTP id b19so8305119wrh.11 for ; Thu, 17 Mar 2022 10:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=8bKHLTXLh3Prk0x7Kjt1KuDZoyhkHjZ+7FeBGH5Uapo=; b=KGxafnM3qSYdU4CWYMi0SB6NNYPDBifAXbe2jqJL/qXULaaQ+MT9nrXnwx6FBn4e7X ESQY1KvTvkAnL8cj9hWkHb5PKlmdM5BFnaP4dgIUu6pD7+G1hEyRdZZZs8US0zhnKmEi fSiqWxuJQV5wCsL6OUaXZbDvIXAVamf91ff6U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to; bh=8bKHLTXLh3Prk0x7Kjt1KuDZoyhkHjZ+7FeBGH5Uapo=; b=XqJw92rVnLfkJzcPziXpx7PQlHtlf4i/Sqhps1ITdJyXzRbsR2pYfVEPfmlw5jFaEk uLbu10mGr2AmyPIGvJ/LCHLQqr43oqxAaF/6KzgXyjP2Gfc6gu3z+4tMUFLTHx1Kgyyu eM2f9DwRsXGCSMDl/tvyIgl1DuPuwokOAJ0nuDYMYB/uFyegJ29xftD//Pbr641X/egs tj1I57Cj4MiSufiUa3IJmBLGBp+fEaztbXDmO1U2oSgBgir7XO7F2XFWy738hFmRoLlM qa9O5ZG417qqEnqMRitAMPNdQ0y73IVldjRDxc1DEdm2FHdYJg8YCy7G+lZPoXgWWHus w8Tw== X-Gm-Message-State: AOAM533u7JCfNx87xuXYi2Kj2mR3dcsmQYSE96hdjrwVPIJ6/k8R3eks 4YQ4o7SWO4DwG+oaiqzT500HjAlGXFSOnri5 X-Google-Smtp-Source: ABdhPJzoRAo3np0fazhW9XxJ7KsRXfdgnZ3L0q1YWfSdtT6Ap0qveKDccvepjyTYUjg/8j2AHQh5Hg== X-Received: by 2002:a5d:528b:0:b0:203:d928:834c with SMTP id c11-20020a5d528b000000b00203d928834cmr5178984wrv.500.1647538172004; Thu, 17 Mar 2022 10:29:32 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id c11-20020a05600c0a4b00b0037c91e085ddsm9709824wmq.40.2022.03.17.10.29.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 10:29:31 -0700 (PDT) Date: Thu, 17 Mar 2022 18:29:29 +0100 From: Daniel Vetter To: Christian =?iso-8859-1?Q?K=F6nig?= Cc: Rob Clark , Rob Clark , Jonathan Marek , David Airlie , freedreno , Vladimir Lypak , Abhinav Kumar , dri-devel , Bjorn Andersson , Akhil P Oommen , linux-arm-msm , Sean Paul , open list , AngeloGioacchino Del Regno Subject: Re: [PATCH 2/3] drm/msm/gpu: Park scheduler threads for system suspend Message-ID: Mail-Followup-To: Christian =?iso-8859-1?Q?K=F6nig?= , Rob Clark , Rob Clark , Jonathan Marek , David Airlie , freedreno , Vladimir Lypak , Abhinav Kumar , dri-devel , Bjorn Andersson , Akhil P Oommen , linux-arm-msm , Sean Paul , open list , AngeloGioacchino Del Regno References: <20220310234611.424743-1-robdclark@gmail.com> <20220310234611.424743-3-robdclark@gmail.com> <3945551d-47d2-1974-f637-1dbc61e14702@amd.com> <865abcff-9f52-dca4-df38-b11189c739ff@amd.com> <915537e2-ac5b-ab0e-3697-2b16a9ec8f91@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <915537e2-ac5b-ab0e-3697-2b16a9ec8f91@amd.com> X-Operating-System: Linux phenom 5.10.0-8-amd64 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On Thu, Mar 17, 2022 at 05:44:57PM +0100, Christian König wrote: > Am 17.03.22 um 17:18 schrieb Rob Clark: > > On Thu, Mar 17, 2022 at 9:04 AM Christian König > > wrote: > > > Am 17.03.22 um 16:10 schrieb Rob Clark: > > > > [SNIP] > > > > userspace frozen != kthread frozen .. that is what this patch is > > > > trying to address, so we aren't racing between shutting down the hw > > > > and the scheduler shoveling more jobs at us. > > > Well exactly that's the problem. The scheduler is supposed to shoveling > > > more jobs at us until it is empty. > > > > > > Thinking more about it we will then keep some dma_fence instance > > > unsignaled and that is and extremely bad idea since it can lead to > > > deadlocks during suspend. > > Hmm, perhaps that is true if you need to migrate things out of vram? > > It is at least not a problem when vram is not involved. > > No, it's much wider than that. > > See what can happen is that the memory management shrinkers want to wait for > a dma_fence during suspend. > > And if you stop the scheduler they will just wait forever. > > What you need to do instead is to drain the scheduler, e.g. call > drm_sched_entity_flush() with a proper timeout for each entity you have > created. Yeah I think properly flushing the scheduler and stopping it and cutting all drivers over to that sounds like the right approach. Generally suspend shouldn't be such a critical path that this will hurt us, all the other io queues get flushed too afaik. Resume is the thing that needs to go real fast. So a patch set to move all drivers that open code the kthread_park to the right scheduler function sounds like the right idea here to me. -Daniel > > Regards, > Christian. > > > > > > So this patch here is an absolute clear NAK from my side. If amdgpu is > > > doing something similar that is a severe bug and needs to be addressed > > > somehow. > > I think amdgpu's use of kthread_park is not related to suspend, but > > didn't look too closely. > > > > And perhaps the solution for this problem is more complex in the case > > of amdgpu, I'm not super familiar with the constraints there. But I > > think it is a fine solution for integrated GPUs. > > > > BR, > > -R > > > > > Regards, > > > Christian. > > > > > > > BR, > > > > -R > > > > > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch