From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Christian_K=c3=b6nig?= Subject: Re: [PATCH 11/11] drm/amdgpu: stop removing BOs from the LRU during CS Date: Wed, 15 May 2019 09:04:52 +0200 Message-ID: <5d6b76f9-5402-989d-1fa8-d807cea1bfe3@gmail.com> References: <20190514123127.1650-1-christian.koenig@amd.com> <20190514123127.1650-11-christian.koenig@amd.com> Reply-To: christian.koenig-5C7GfCeVMHo@public.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0469999565==" Return-path: In-Reply-To: Content-Language: en-US List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: "Liang, Prike" , =?UTF-8?B?TWFyZWsgT2zFocOhaw==?= Cc: "Zhou, David(ChunMing)" , amd-gfx mailing list , dri-devel This is a multi-part message in MIME format. --===============0469999565== Content-Type: multipart/alternative; boundary="------------00764539B5BAA9568D6C6EDA" Content-Language: en-US This is a multi-part message in MIME format. --------------00764539B5BAA9568D6C6EDA Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi Prike, no, that can lead to massive problems in a real OOM situation and is not something we can do here. Christian. Am 15.05.19 um 04:00 schrieb Liang, Prike: > > Hi Christian , > > I just wonder when encounter ENOMEM error during pin amdgpu BOs can we > retry validate again as below. > > With the following simply patch the Abaqus pinned issue not observed. > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > > index 11cbf63..72a32f5 100644 > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c > > @@ -902,11 +902,15 @@ int amdgpu_bo_pin_restricted(struct amdgpu_bo > *bo, u32 domain, > > bo->placements[i].lpfn = lpfn; > >                 bo->placements[i].flags |= TTM_PL_FLAG_NO_EVICT; > >         } > > - > > +retry: > >         r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); > >         if (unlikely(r)) { > > -               dev_err(adev->dev, "%p pin failed\n", bo); > > -               goto error; > > +                if (r == -ENOMEM){ > > +                        goto retry; > > +                } else { > > + dev_err(adev->dev, "%p pin failed\n", bo); > > +                       goto error; > > +                } > >         } > >         bo->pin_count = 1; > > Thanks, > > Prike > > *From:* Marek Olšák > *Sent:* Wednesday, May 15, 2019 3:33 AM > *To:* Christian König > *Cc:* Zhou, David(ChunMing) ; Liang, Prike > ; dri-devel ; > amd-gfx mailing list > *Subject:* Re: [PATCH 11/11] drm/amdgpu: stop removing BOs from the > LRU during CS > > [CAUTION: External Email] > > This series fixes the OOM errors. However, if I torture the kernel > driver more, I can get it to deadlock and end up with unkillable > processes. I can also get an OOM error. I just ran the test 5 times: > > AMD_DEBUG=testgdsmm glxgears & AMD_DEBUG=testgdsmm glxgears & > AMD_DEBUG=testgdsmm glxgears & AMD_DEBUG=testgdsmm glxgears & > AMD_DEBUG=testgdsmm glxgears > > Marek > > On Tue, May 14, 2019 at 8:31 AM Christian König > > wrote: > > This avoids OOM situations when we have lots of threads > submitting at the same time. > > Signed-off-by: Christian König > > --- >  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +- >  1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > index fff558cf385b..f9240a94217b 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > @@ -648,7 +648,7 @@ static int amdgpu_cs_parser_bos(struct > amdgpu_cs_parser *p, >         } > >         r = ttm_eu_reserve_buffers(&p->ticket, &p->validated, true, > -                                  &duplicates, true); > +                                  &duplicates, false); >         if (unlikely(r != 0)) { >                 if (r != -ERESTARTSYS) > DRM_ERROR("ttm_eu_reserve_buffers failed.\n"); > -- > 2.17.1 > > _______________________________________________ > amd-gfx mailing list > amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > --------------00764539B5BAA9568D6C6EDA Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit
Hi Prike,

no, that can lead to massive problems in a real OOM situation and is not something we can do here.

Christian.

Am 15.05.19 um 04:00 schrieb Liang, Prike:

Hi Christian ,

 

I just wonder when encounter ENOMEM error during pin amdgpu BOs can we retry validate again as below.

With the following simply patch the Abaqus pinned issue not observed.

 

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c

index 11cbf63..72a32f5 100644

--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c

+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c

@@ -902,11 +902,15 @@ int amdgpu_bo_pin_restricted(struct amdgpu_bo *bo, u32 domain,

                        bo->placements[i].lpfn = lpfn;

                bo->placements[i].flags |= TTM_PL_FLAG_NO_EVICT;

        }

-

+retry:

        r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);

        if (unlikely(r)) {

-               dev_err(adev->dev, "%p pin failed\n", bo);

-               goto error;

+                if (r == -ENOMEM){

+                        goto retry;

+                } else {

+                       dev_err(adev->dev, "%p pin failed\n", bo);

+                       goto error;

+                }

        }

 

        bo->pin_count = 1;

 

 

Thanks,

Prike

 

From: Marek Olšák <maraeo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Sent: Wednesday, May 15, 2019 3:33 AM
To: Christian König <ckoenig.leichtzumerken-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Zhou, David(ChunMing) <David1.Zhou-5C7GfCeVMHo@public.gmane.org>; Liang, Prike <Prike.Liang-5C7GfCeVMHo@public.gmane.org>; dri-devel <dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org>; amd-gfx mailing list <amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org>
Subject: Re: [PATCH 11/11] drm/amdgpu: stop removing BOs from the LRU during CS

 

[CAUTION: External Email]

This series fixes the OOM errors. However, if I torture the kernel driver more, I can get it to deadlock and end up with unkillable processes. I can also get an OOM error. I just ran the test 5 times:

 

AMD_DEBUG=testgdsmm glxgears & AMD_DEBUG=testgdsmm glxgears & AMD_DEBUG=testgdsmm glxgears & AMD_DEBUG=testgdsmm glxgears & AMD_DEBUG=testgdsmm glxgears

 

Marek

 

On Tue, May 14, 2019 at 8:31 AM Christian König <ckoenig.leichtzumerken-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

This avoids OOM situations when we have lots of threads
submitting at the same time.

Signed-off-by: Christian König <christian.koenig-5C7GfCeVMHo@public.gmane.org>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index fff558cf385b..f9240a94217b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -648,7 +648,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
        }

        r = ttm_eu_reserve_buffers(&p->ticket, &p->validated, true,
-                                  &duplicates, true);
+                                  &duplicates, false);
        if (unlikely(r != 0)) {
                if (r != -ERESTARTSYS)
                        DRM_ERROR("ttm_eu_reserve_buffers failed.\n");
--
2.17.1

_______________________________________________
amd-gfx mailing list
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx


--------------00764539B5BAA9568D6C6EDA-- --===============0469999565== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KYW1kLWdmeCBt YWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5m cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4 --===============0469999565==--