All of lore.kernel.org
 help / color / mirror / Atom feed
* [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
@ 2014-10-04 15:19 bugzilla-daemon
  2014-10-06  8:41 ` [Bug 84663] " bugzilla-daemon
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-04 15:19 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 2495 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

            Bug ID: 84663
           Summary: high cpu usage, poor performance in Borderlands 2 with
                    radeonsi, PRIME
           Product: Mesa
           Version: git
          Hardware: Other
                OS: All
            Status: NEW
          Severity: normal
          Priority: medium
         Component: Drivers/Gallium/radeonsi
          Assignee: dri-devel@lists.freedesktop.org
          Reporter: haagch@frickel.club

Created attachment 107328
  --> https://bugs.freedesktop.org/attachment.cgi?id=107328&action=edit
sysprof recording from borderlands 2 only

00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor
Graphics Controller (rev 09)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI]
Wimbledon XT [Radeon HD 7970M] (rev ff)

xorg stable, mesa git, linux 3.17-rc7.

I have had something similar in some games I think, but most recently with
Borderlands 2.

Here is a random screenshot with the HUD fps display from someone with a HD
7870 that shows that it runs mostly with 60 fps:
https://i.imgur.com/qH0sBkl.jpg

And here is a short clip of how it runs for me that shows it runs with 20-30
fps: https://www.youtube.com/watch?v=ZeZreRntt3k
Radeontop says that the gpu is only used to ~30%.
While running Borderlands 2 the CPU usage is always at 100+% on my i7 3632qm.


I was undecided whether to report this here, but the difference is quite large
so I thought I'd give it a try because I think the game itself is not supposed
to use this much cpu time, so maybe it has something to do with the driver.

Theories:

< glennk> guessing from that output that the game engine uses a lot of
occlusion queries and is stalling on them

I haven't really found anything to test that yet.

< agd5f> haagch, hybrid laptops have to do a lot of extra copying to get the
frame from the rendering GPU to the display GPU

I hope that the overhead is not *that* large because losing 70+% of gpu time
would make it kind of useless for the affected games.

Fortunately many (most?) games run much better, for example unigine valley
shows good gpu usage: https://www.youtube.com/watch?v=sLWvYJlfvWM
which makes me believe that there is a specific bottleneck.

Attached is a sysprof profile of borderlands 2 but I don't know which of it is
normal (like 25% total cpu time for glDrawRangeElements?).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 4074 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
@ 2014-10-06  8:41 ` bugzilla-daemon
  2014-10-06 11:48 ` bugzilla-daemon
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-06  8:41 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 590 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

--- Comment #1 from Michel Dänzer <michel@daenzer.net> ---
(In reply to Christoph Haag from comment #0)
> sysprof recording from borderlands 2 only

Can you create another profile including all processes?

BTW, if you can make sure that at least Mesa and libdrm are compiled with
-fno-omit-frame-pointer (after any other compiler flags implying
-fomit-frame-pointer, such as -O), that might improve the quality of the
callgraphs in the profile.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
  2014-10-06  8:41 ` [Bug 84663] " bugzilla-daemon
@ 2014-10-06 11:48 ` bugzilla-daemon
  2014-10-07  1:58 ` bugzilla-daemon
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-06 11:48 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1693 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

Christoph Haag <haagch@frickel.club> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #107328|0                           |1
        is obsolete|                            |

--- Comment #2 from Christoph Haag <haagch@frickel.club> ---
Created attachment 107418
  --> https://bugs.freedesktop.org/attachment.cgi?id=107418&action=edit
sysprof

(In reply to Michel Dänzer from comment #1)
> (In reply to Christoph Haag from comment #0)
> > sysprof recording from borderlands 2 only
> 
> Can you create another profile including all processes?
> 
> BTW, if you can make sure that at least Mesa and libdrm are compiled with
> -fno-omit-frame-pointer (after any other compiler flags implying
> -fomit-frame-pointer, such as -O), that might improve the quality of the
> callgraphs in the profile.

Ok, so in my first sysprof profile I had mesa compiled with -O0 -ggdb and
--enable-debug. That was probably overkill, but it had hardly any effect on
perceived performance.

Now this is probably more realistic. mesa is *not* compiled with --enable-debug
and it *is* compiled with with "-Og -ggdb". Also libdrm is compiled with "-Og
-ggdb". If I'm reading this correctly now it unfortunately looks like
Borderlands 2 is consuming most of the CPU time itself now...
It's still the same cpu consumption: 112-114% CPU.

So far I really have only to go on that my cpu should be good enough to not
bottleneck the game this much...

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 3328 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
  2014-10-06  8:41 ` [Bug 84663] " bugzilla-daemon
  2014-10-06 11:48 ` bugzilla-daemon
@ 2014-10-07  1:58 ` bugzilla-daemon
  2014-10-07 11:04 ` bugzilla-daemon
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-07  1:58 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 465 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

--- Comment #3 from Michel Dänzer <michel@daenzer.net> ---
(In reply to Christoph Haag from comment #2)
> mesa [...] *is* compiled with with "-Og -ggdb". Also libdrm is compiled with
> "-Og -ggdb".

Looks like you didn't add -fno-omit-frame-pointer though? Can you get another
profile with Mesa and libdrm built with that?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1386 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (2 preceding siblings ...)
  2014-10-07  1:58 ` bugzilla-daemon
@ 2014-10-07 11:04 ` bugzilla-daemon
  2014-10-08  2:50 ` bugzilla-daemon
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-07 11:04 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 821 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

Christoph Haag <haagch@frickel.club> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #107418|0                           |1
        is obsolete|                            |

--- Comment #4 from Christoph Haag <haagch@frickel.club> ---
Created attachment 107479
  --> https://bugs.freedesktop.org/attachment.cgi?id=107479&action=edit
hopefully with -fno-omit-frame-pointer (xz compressed)

Damn, I really thought -Og would activate that too. Sorry.

I hope this is better.

By the way, Is there a better way to profile something likethis or is sysprof
already the most appropriate tool?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2361 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (3 preceding siblings ...)
  2014-10-07 11:04 ` bugzilla-daemon
@ 2014-10-08  2:50 ` bugzilla-daemon
  2014-10-09  7:54 ` bugzilla-daemon
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-08  2:50 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 624 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

--- Comment #5 from Michel Dänzer <michel@daenzer.net> ---
(In reply to Christoph Haag from comment #4)
> I hope this is better.

Yes, looks better, thanks.

> By the way, Is there a better way to profile something likethis or is
> sysprof already the most appropriate tool?

I find sysprof's UI the most intuitive, but perf is nice as well.


I don't notice anything using obviously too much CPU in your profile. Can you
get a screenshot with GALLIUM_HUD like in bug 84570?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1752 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (4 preceding siblings ...)
  2014-10-08  2:50 ` bugzilla-daemon
@ 2014-10-09  7:54 ` bugzilla-daemon
  2014-10-10  2:41 ` bugzilla-daemon
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-09  7:54 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1142 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

--- Comment #6 from Christoph Haag <haagch@frickel.club> ---
Created attachment 107595
  --> https://bugs.freedesktop.org/attachment.cgi?id=107595&action=edit
GALLIUM_HUD="fps,VRAM-usage+GTT-usage,requested-VRAM+requested-GTT,num-bytes-moved;cpu0+cpu1+cpu2+cpu3;cpu4+cpu5+cpu6+cpu7"

(In reply to Michel Dänzer from comment #5)
> I don't notice anything using obviously too much CPU in your profile. Can
> you get a screenshot with GALLIUM_HUD like in bug 84570?

For the screenshot I pinned the game to the cpus 3,7 with taskset to stop it
from polluting the graphs so much when jumping between cpus. Didn't really
decrease or increase the performance much.

I really don't know right now... When the game loses the input focus, it pauses
the game and the game rendering itself is not updated anymore I think. The cpu
usage will drop from 112-114% cpu usage to 100% cpu usage and the fps go up to
60 fps. When the game is focussed again, it unpauses, and then it's again low
performance.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2608 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (5 preceding siblings ...)
  2014-10-09  7:54 ` bugzilla-daemon
@ 2014-10-10  2:41 ` bugzilla-daemon
  2014-10-10 13:46 ` bugzilla-daemon
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-10  2:41 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 798 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

--- Comment #7 from Michel Dänzer <michel@daenzer.net> ---
(In reply to Christoph Haag from comment #6)
> For the screenshot I pinned the game to the cpus 3,7 with taskset to stop it
> from polluting the graphs so much when jumping between cpus.

I'm more interested in the graphics memory usage graphs anyway.

BTW, I find 'requested-GTT+GTT-usage,requested-VRAM+VRAM-usage' more useful
than 'VRAM-usage+GTT-usage,requested-VRAM+requested-GTT' because it allows to
see immediately how well (or not) the actuall VRAM/GTT usage matches what
radeonsi wants to use.

Anyway, it looks like current Mesa Git and/or the patches from bug 84662 might
help.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1922 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (6 preceding siblings ...)
  2014-10-10  2:41 ` bugzilla-daemon
@ 2014-10-10 13:46 ` bugzilla-daemon
  2014-11-10 15:21 ` bugzilla-daemon
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-10-10 13:46 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1120 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

Christoph Haag <haagch@frickel.club> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #107595|0                           |1
        is obsolete|                            |

--- Comment #8 from Christoph Haag <haagch@frickel.club> ---
Created attachment 107663
  --> https://bugs.freedesktop.org/attachment.cgi?id=107663&action=edit
borderlands 2 with GALLIUM_HUD with patches and drm-next-3.18

Well, the good news is that I can confirm that the random huge pauses in
borderlands 2 and in unreal engine 4 demos with drm-next-3.18 are gone with the
patches.

If the CPU usage is all in Borderlands 2 itself and not in mesa, then maybe
I'll need to start asking around what other people's cpus are and how much cpu
time it uses. Maybe it really is some physics/internal rendering calculation
that is so slow, when I get almost 80 fps when the game is paused in the menu
or unfocussed..

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2674 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (7 preceding siblings ...)
  2014-10-10 13:46 ` bugzilla-daemon
@ 2014-11-10 15:21 ` bugzilla-daemon
  2017-03-12 11:11 ` bugzilla-daemon
  2017-03-13 20:04 ` bugzilla-daemon
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2014-11-10 15:21 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 1331 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

--- Comment #9 from Christoph Haag <haagch@frickel.club> ---
I wish I had an idea what causes this. I think this is a general problem, but I
can't put my finger on it.

Yesterday at home I tried the oculus rift unity3d demo with -force-opengl in
wine and got pretty much 75fps everywhere. Today at uni it drops to 45 fps when
looking at the more complex objects in the demo. Then CPU usage is at ~113% and
the GPU usage is relatively low.

Something from disconnecting audio jacks, dvi monitor, usb hub with keyboard,
mouse and usb DAC, suspending, waking up on battery, suspending again, waking
up on power must have caused the state where it's performing badly.

Trying with dri2 instead of dri3, setting vblank_mode=0, switching to another
window manager, restarting X, connecting another dvi monitor, setting the
display to another crtc, nothing seems to have any effect. I think at one point
I disabled intel pstate cpu scaling and this didn't help either...

The state where it's performing well happens rarely, but it seems that it
randomly does happen. The next time I'll make sure to test borderlands 2 too to
confirm whether there is some underlying problem and collect some profiling
data...

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2140 bytes --]

[-- Attachment #2: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (8 preceding siblings ...)
  2014-11-10 15:21 ` bugzilla-daemon
@ 2017-03-12 11:11 ` bugzilla-daemon
  2017-03-13 20:04 ` bugzilla-daemon
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2017-03-12 11:11 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 507 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

Vedran Miletić <vedran@miletic.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |77449


Referenced Bugs:

https://bugs.freedesktop.org/show_bug.cgi?id=77449
[Bug 77449] Tracker bug for all bugs related to Steam titles
-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1583 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Bug 84663] high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME
  2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
                   ` (9 preceding siblings ...)
  2017-03-12 11:11 ` bugzilla-daemon
@ 2017-03-13 20:04 ` bugzilla-daemon
  10 siblings, 0 replies; 12+ messages in thread
From: bugzilla-daemon @ 2017-03-13 20:04 UTC (permalink / raw)
  To: dri-devel


[-- Attachment #1.1: Type: text/plain, Size: 573 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=84663

Marek Olšák <maraeo@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #10 from Marek Olšák <maraeo@gmail.com> ---
Closing. The shader cache should fix the pauses. glthread will improve
performance.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2129 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2017-03-13 20:04 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-10-04 15:19 [Bug 84663] New: high cpu usage, poor performance in Borderlands 2 with radeonsi, PRIME bugzilla-daemon
2014-10-06  8:41 ` [Bug 84663] " bugzilla-daemon
2014-10-06 11:48 ` bugzilla-daemon
2014-10-07  1:58 ` bugzilla-daemon
2014-10-07 11:04 ` bugzilla-daemon
2014-10-08  2:50 ` bugzilla-daemon
2014-10-09  7:54 ` bugzilla-daemon
2014-10-10  2:41 ` bugzilla-daemon
2014-10-10 13:46 ` bugzilla-daemon
2014-11-10 15:21 ` bugzilla-daemon
2017-03-12 11:11 ` bugzilla-daemon
2017-03-13 20:04 ` bugzilla-daemon

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.