From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 99312] Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini Date: Sat, 07 Jan 2017 18:25:19 +0000 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2094792658==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [131.252.210.165]) by gabe.freedesktop.org (Postfix) with ESMTP id AD05F6EB5D for ; Sat, 7 Jan 2017 18:25:19 +0000 (UTC) List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============2094792658== Content-Type: multipart/alternative; boundary="14838135190.562B1526.14775"; charset="UTF-8" --14838135190.562B1526.14775 Date: Sat, 7 Jan 2017 18:25:19 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D99312 Bug ID: 99312 Summary: Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini Product: Mesa Version: 13.0 Hardware: Other OS: All Status: NEW Severity: normal Priority: medium Component: Drivers/Gallium/radeonsi Assignee: dri-devel@lists.freedesktop.org Reporter: vedran@miletic.net QA Contact: dri-devel@lists.freedesktop.org Running long lasting OpenCL kernels (e.g. GROMACS with a system of many ato= ms) using kernel 4.8.15, Mesa git, and LLVM git on Kabini APU: vendor_id : AuthenticAMD cpu family : 22 model : 0 model name : AMD Athlon(tm) 5350 APU with Radeon(tm) R3 stepping : 1 microcode : 0x700010b with GPU: 00:01.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Kabini [Radeon HD 8400 / R3 Series] [1002:9830] causes GPU lockups like: [338584.980657] radeon 0000:00:01.0: ring 0 stalled for more than 10351msec [338584.980811] radeon 0000:00:01.0: GPU lockup (current fence id 0x00000000000827c1 last fence id 0x00000000000827c2 on ring 0) [338585.484633] radeon 0000:00:01.0: ring 0 stalled for more than 10855msec [338585.484789] radeon 0000:00:01.0: GPU lockup (current fence id 0x00000000000827c1 last fence id 0x00000000000827c2 on ring 0) [338585.988632] radeon 0000:00:01.0: ring 0 stalled for more than 11359msec [338585.988787] radeon 0000:00:01.0: GPU lockup (current fence id 0x00000000000827c1 last fence id 0x00000000000827c2 on ring 0) Machine does not hang. This is reliably reproducible. Any other info I can provide? --=20 You are receiving this mail because: You are the assignee for the bug.= --14838135190.562B1526.14775 Date: Sat, 7 Jan 2017 18:25:19 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated
Bug ID 99312
Summary Long-running OpenCL kernels cause ring stalls and GPU lockups= on Kabini
Product Mesa
Version 13.0
Hardware Other
OS All
Status NEW
Severity normal
Priority medium
Component Drivers/Gallium/radeonsi
Assignee dri-devel@lists.freedesktop.org
Reporter vedran@miletic.net
QA Contact dri-devel@lists.freedesktop.org

Running long lasting OpenCL kernels (e.g. GROMACS with a syste=
m of many atoms)
using kernel 4.8.15, Mesa git, and LLVM git on Kabini APU:

vendor_id       : AuthenticAMD
cpu family      : 22
model           : 0
model name      : AMD Athlon(tm) 5350 APU with Radeon(tm) R3
stepping        : 1
microcode       : 0x700010b

with GPU:

00:01.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc.
[AMD/ATI] Kabini [Radeon HD 8400 / R3 Series] [1002:9830]

causes GPU lockups like:

[338584.980657] radeon 0000:00:01.0: ring 0 stalled for more than 10351msec
[338584.980811] radeon 0000:00:01.0: GPU lockup (current fence id
0x00000000000827c1 last fence id 0x00000000000827c2 on ring 0)
[338585.484633] radeon 0000:00:01.0: ring 0 stalled for more than 10855msec
[338585.484789] radeon 0000:00:01.0: GPU lockup (current fence id
0x00000000000827c1 last fence id 0x00000000000827c2 on ring 0)
[338585.988632] radeon 0000:00:01.0: ring 0 stalled for more than 11359msec
[338585.988787] radeon 0000:00:01.0: GPU lockup (current fence id
0x00000000000827c1 last fence id 0x00000000000827c2 on ring 0)

Machine does not hang. This is reliably reproducible. Any other info I can
provide?


You are receiving this mail because:
  • You are the assignee for the bug.
= --14838135190.562B1526.14775-- --===============2094792658== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== --===============2094792658==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 99312] Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini Date: Sat, 07 Jan 2017 18:25:45 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0871429838==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [IPv6:2610:10:20:722:a800:ff:fe98:4b55]) by gabe.freedesktop.org (Postfix) with ESMTP id 50DE66EB5E for ; Sat, 7 Jan 2017 18:25:45 +0000 (UTC) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============0871429838== Content-Type: multipart/alternative; boundary="14838135450.6eF1ba52.14933"; charset="UTF-8" --14838135450.6eF1ba52.14933 Date: Sat, 7 Jan 2017 18:25:45 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D99312 Vedran Mileti=C4=87 changed: What |Removed |Added ---------------------------------------------------------------------------- Hardware|Other |x86-64 (AMD64) Severity|normal |major Version|13.0 |git OS|All |Linux (All) --=20 You are receiving this mail because: You are the assignee for the bug.= --14838135450.6eF1ba52.14933 Date: Sat, 7 Jan 2017 18:25:45 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated Vedran Mileti=C4=87 changed bug 99312<= /a>
What Removed Added
Hardware Other x86-64 (AMD64)
Severity normal major
Version 13.0 git
OS All Linux (All)


You are receiving this mail because:
  • You are the assignee for the bug.
= --14838135450.6eF1ba52.14933-- --===============0871429838== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== --===============0871429838==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 99312] Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini Date: Sat, 07 Jan 2017 18:36:56 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0950619989==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [131.252.210.165]) by gabe.freedesktop.org (Postfix) with ESMTP id 65D926EB60 for ; Sat, 7 Jan 2017 18:36:56 +0000 (UTC) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============0950619989== Content-Type: multipart/alternative; boundary="14838142161.dedC.15770"; charset="UTF-8" --14838142161.dedC.15770 Date: Sat, 7 Jan 2017 18:36:56 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D99312 --- Comment #1 from John Bridgman --- If you have not already done so, try disabling the watchdog timer: MODULE_PARM_DESC(lockup_timeout, "GPU lockup timeout in ms (default 10000 = =3D 10 seconds, 0 =3D disable)"); module_param_named(lockup_timeout, radeon_lockup_timeout, int, 0444); As part of HSA/ROC development we dropped the priority of compute work rela= tive to graphics which improved interactivity and *almost* eliminated timeouts without having to disable the timer - when I get back in the office I'll d= ig up the changes. In the meantime, I think disabling the timer will do what y= ou need although you will still have sluggish graphics while long-running kern= els are active. Lowering the priority of compute waves across the board won't be a fully general solution because there are going to be some cases (eg Valve's recent work with using high priority compute to improve VR smoothness) where compu= te will need to be *higher* priority than graphics but it should cover most ca= ses other than "simultaneously running GROMACS and VR". --=20 You are receiving this mail because: You are the assignee for the bug.= --14838142161.dedC.15770 Date: Sat, 7 Jan 2017 18:36:56 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated

Comment= # 1 on bug 99312<= /a> from John Bridgman
If you have not already done so, try disabling the watchdog ti=
mer:


MODULE_PARM_DESC(lockup_timeout, "GPU lockup timeout in ms (default 10=
000 =3D 10
seconds, 0 =3D disable)");
module_param_named(lockup_timeout, radeon_lockup_timeout, int, 0444);

As part of HSA/ROC development we dropped the priority of compute work rela=
tive
to graphics which improved interactivity and *almost* eliminated timeouts
without having to disable the timer  - when I get back in the office I'll d=
ig
up the changes. In the meantime, I think disabling the timer will do what y=
ou
need although you will still have sluggish graphics while long-running kern=
els
are active.

Lowering the priority of compute waves across the board won't be a fully
general solution because there are going to be some cases (eg Valve's recent
work with using high priority compute to improve VR smoothness) where compu=
te
will need to be *higher* priority than graphics but it should cover most ca=
ses
other than "simultaneously running GROMACS and VR".


You are receiving this mail because:
  • You are the assignee for the bug.
= --14838142161.dedC.15770-- --===============0950619989== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== --===============0950619989==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 99312] Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini when radeon.lockup_timeout is enabled Date: Mon, 09 Jan 2017 17:02:43 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0303932742==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [131.252.210.165]) by gabe.freedesktop.org (Postfix) with ESMTP id 4AF656E1CA for ; Mon, 9 Jan 2017 17:02:43 +0000 (UTC) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============0303932742== Content-Type: multipart/alternative; boundary="14839813630.551C.11706"; charset="UTF-8" --14839813630.551C.11706 Date: Mon, 9 Jan 2017 17:02:43 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D99312 Vedran Mileti=C4=87 changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Long-running OpenCL kernels |Long-running OpenCL kernels |cause ring stalls and GPU |cause ring stalls and GPU |lockups on Kabini |lockups on Kabini when | |radeon.lockup_timeout is | |enabled --- Comment #2 from Vedran Mileti=C4=87 --- (In reply to John Bridgman from comment #1) > If you have not already done so, try disabling the watchdog timer: >=20 >=20 > MODULE_PARM_DESC(lockup_timeout, "GPU lockup timeout in ms (default 10000= =3D > 10 seconds, 0 =3D disable)"); > module_param_named(lockup_timeout, radeon_lockup_timeout, int, 0444); >=20 Yup, that works around the problem. > As part of HSA/ROC development we dropped the priority of compute work > relative to graphics which improved interactivity and *almost* eliminated > timeouts without having to disable the timer - when I get back in the > office I'll dig up the changes. In the meantime, I think disabling the ti= mer > will do what you need although you will still have sluggish graphics while > long-running kernels are active. >=20 Eager to hear the details. --=20 You are receiving this mail because: You are the assignee for the bug.= --14839813630.551C.11706 Date: Mon, 9 Jan 2017 17:02:43 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated Vedran Mileti=C4=87 changed bug 99312<= /a>
What Removed Added
Summary Long-running OpenCL kernels cause ring stalls and GPU lockup= s on Kabini Long-running OpenCL kernels cause ring stalls and GPU lockup= s on Kabini when radeon.lockup_timeout is enabled

Comment= # 2 on bug 99312<= /a> from Vedran Mileti=C4=87
(In reply to John Bridgman from comment #1)
> If you have not already done so, try disabling t=
he watchdog timer:
>=20
>=20
> MODULE_PARM_DESC(lockup_timeout, "GPU lockup timeout in ms (defau=
lt 10000 =3D
> 10 seconds, 0 =3D disable)");
> module_param_named(lockup_timeout, radeon_lockup_timeout, int, 0444);
> 

Yup, that works around the problem.

> As part of HSA/ROC development we dropped the pr=
iority of compute work
> relative to graphics which improved interactivity and *almost* elimina=
ted
> timeouts without having to disable the timer  - when I get back in the
> office I'll dig up the changes. In the meantime, I think disabling the=
 timer
> will do what you need although you will still have sluggish graphics w=
hile
> long-running kernels are active.
> 

Eager to hear the details.


You are receiving this mail because:
  • You are the assignee for the bug.
= --14839813630.551C.11706-- --===============0303932742== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== --===============0303932742==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 99312] Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini when radeon.lockup_timeout is enabled Date: Sat, 11 May 2019 20:07:08 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1340292947==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [IPv6:2610:10:20:722:a800:ff:fe98:4b55]) by gabe.freedesktop.org (Postfix) with ESMTP id 9979B8979F for ; Sat, 11 May 2019 20:07:08 +0000 (UTC) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============1340292947== Content-Type: multipart/alternative; boundary="15576052280.FdE9.1964" Content-Transfer-Encoding: 7bit --15576052280.FdE9.1964 Date: Sat, 11 May 2019 20:07:08 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D99312 Jan Vesely changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |99553 Referenced Bugs: https://bugs.freedesktop.org/show_bug.cgi?id=3D99553 [Bug 99553] Tracker bug for runnning OpenCL applications on Clover --=20 You are receiving this mail because: You are the assignee for the bug.= --15576052280.FdE9.1964 Date: Sat, 11 May 2019 20:07:08 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated Jan Vesely changed bug 99312<= /a>
What Removed Added
Blocks   99553



You are receiving this mail because:
  • You are the assignee for the bug.
= --15576052280.FdE9.1964-- --===============1340292947== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs --===============1340292947==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 99312] Long-running OpenCL kernels cause ring stalls and GPU lockups on Kabini when radeon.lockup_timeout is enabled Date: Wed, 25 Sep 2019 17:56:39 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0561181477==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [131.252.210.165]) by gabe.freedesktop.org (Postfix) with ESMTP id A705F6EC26 for ; Wed, 25 Sep 2019 17:56:39 +0000 (UTC) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============0561181477== Content-Type: multipart/alternative; boundary="15694341991.e36396D.31656" Content-Transfer-Encoding: 7bit --15694341991.e36396D.31656 Date: Wed, 25 Sep 2019 17:56:39 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D99312 GitLab Migration User changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |MOVED Status|NEW |RESOLVED --- Comment #3 from GitLab Migration User --- -- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this = link to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/124= 6. --=20 You are receiving this mail because: You are the assignee for the bug.= --15694341991.e36396D.31656 Date: Wed, 25 Sep 2019 17:56:39 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated
GitLab Migration User changed bug 99312<= /a>
What Removed Added
Resolution --- MOVED
Status NEW RESOLVED

Comment= # 3 on bug 99312<= /a> from GitLab Migration User
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been
closed from further activity.

You can subscribe and participate further through the new bug through this =
link
to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/1246.
        


You are receiving this mail because:
  • You are the assignee for the bug.
= --15694341991.e36396D.31656-- --===============0561181477== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs --===============0561181477==--