* [Bug 71662] New: [nvd9 regression] 3d application hang
@ 2013-11-15 22:32 bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
0 siblings, 1 reply; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-15 22:32 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 2963 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
Priority: medium
Bug ID: 71662
Assignee: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Summary: [nvd9 regression] 3d application hang
Severity: normal
Classification: Unclassified
OS: All
Reporter: jano.vesely-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Hardware: Other
Status: NEW
Version: unspecified
Component: Drivers/DRI/nouveau
Product: Mesa
This is a regression from 3.12 to 3.13 (still present on nouveau master).
I started seeing these in dmesg:
[ 64.358170] hda-codec: out of range cmd 0:6:707:fffffffc
[ 2310.511292] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2310.778618] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2310.837097] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2310.895963] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2311.231493] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2311.326618] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2311.466523] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2312.081457] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2312.118070] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2312.484497] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2312.690892] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2312.854135] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2313.160414] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2313.421500] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2313.878647] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2314.296446] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2314.440653] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2315.068705] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2315.089576] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2315.414808] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2315.777496] nouveau W[ PFIFO][0000:01:00.0] INTR 0x01000000: 0x00000005
[ 2315.900813] nouveau ![ PFIFO][0000:01:00.0] unhandled status 0x00800000
I'm using mesa git master. Sometimes openarena (exec anholt) finishes OK with
just the warnings (tons of them), occasionally it hangs with the last message.
The hang appears to happen more often on nouveau master than vanilla 3.12
The card is:
[ 10.238807] nouveau [ DEVICE][0000:01:00.0] Chipset: GF119 (NVD9)
[ 10.238809] nouveau [ DEVICE][0000:01:00.0] Family : NVD0
couple with snb intel.
the bug is perfectly reproducable, let me know what other info I can provide.
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 4198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9 regression] 3d application hang
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
@ 2013-11-15 22:35 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-15 22:38 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-15 22:35 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 349 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
--- Comment #1 from Jan Vesely <jano.vesely-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> ---
(In reply to comment #0)
> This is a regression from 3.12 to 3.13 (still present on nouveau master).
should have been 3.11 to 3.12.
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 1138 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9 regression] 3d application hang
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
2013-11-15 22:35 ` [Bug 71662] " bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-15 22:38 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:23 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-15 22:38 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 379 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
--- Comment #2 from Ilia Mirkin <imirkin-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org> ---
A git bisect would be useful. You should be able to just run the bisect under
drivers/gpu/drm/nouveau, i.e.
git bisect start v3.12 v3.11 -- drivers/gpu/drm/nouveau
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 1101 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9 regression] 3d application hang
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
2013-11-15 22:35 ` [Bug 71662] " bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-15 22:38 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-16 3:23 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:32 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-16 3:23 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 1802 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
--- Comment #3 from Jan Vesely <jano.vesely-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> ---
(In reply to comment #2)
> A git bisect would be useful. You should be able to just run the bisect
> under drivers/gpu/drm/nouveau, i.e.
>
> git bisect start v3.12 v3.11 -- drivers/gpu/drm/nouveau
$ git bisect log
# bad: [5e01dc7b26d9f24f39abace5da98ccbd6a5ceb52] Linux 3.12
# good: [6e4664525b1db28f8c4e1130957f70a94c19213e] Linux 3.11
git bisect start 'v3.12' 'v3.11' 'drivers/gpu/drm/nouveau/'
# bad: [bd9c5a2016307164c419c5e24a46921c10e620a0] drm/nouveau: require
contiguous bo for framebuffer
git bisect bad bd9c5a2016307164c419c5e24a46921c10e620a0
# bad: [0d69704ae348c03bc216b01e32a0e9a2372be419] gpu/vga_switcheroo: add
driver control power feature. (v3)
git bisect bad 0d69704ae348c03bc216b01e32a0e9a2372be419
# bad: [baa7094355a10b432bbccacb925da4bdac861c8d] drm: const'ify ioctls table
(v2)
git bisect bad baa7094355a10b432bbccacb925da4bdac861c8d
# good: [72525b3f333de54fa0c42ef87f27861e41478f1e] drm/ttm: convert to unified
vma offset manager
git bisect good 72525b3f333de54fa0c42ef87f27861e41478f1e
# bad: [43387b37fa2d0f368142b8fa8c9440da92e5381b] drm/gem: create
drm_gem_dumb_destroy
git bisect bad 43387b37fa2d0f368142b8fa8c9440da92e5381b
# first bad commit: [43387b37fa2d0f368142b8fa8c9440da92e5381b] drm/gem: create
drm_gem_dumb_destroy
note that the only good commit was a bit weird, plymouth won't run, gnome-shell
segfaults immediatelly, and DRI_PRIME=13d apps showed no output (both glxgears
and oa). However, both were running (displayed fps, ...) and the hang was
always casued by running oa (never glxgears) so I figured the black screen does
not matter.
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 2600 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9 regression] 3d application hang
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
` (2 preceding siblings ...)
2013-11-16 3:23 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-16 3:32 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:33 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-16 3:32 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 649 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
--- Comment #4 from Ilia Mirkin <imirkin-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org> ---
Hmm... that commit is outside of nouveau. I'm not 100% sure that the bisect
results are accurate in that case. (But all the work you did is still useful,
future bisects can use the same good/bad information.)
Can you verify that 43387b37fa2d0^ is good? (i.e. commit 86e81f0e62)
If that is not the case, you can restart the bisect with
git bisect v3.12 v3.11 72525b3f333de54 -- drivers/gpu/drm
which will include everything in drm.
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 1371 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9 regression] 3d application hang
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
` (3 preceding siblings ...)
2013-11-16 3:32 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-16 3:33 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:36 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-16 3:33 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 830 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
--- Comment #5 from Jan Vesely <jano.vesely-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> ---
If oa does not hang, I also see a lot of:
[ 683.090527] dmar: DRHD: handling fault status reg 3
[ 683.090531] dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr
fe000000
DMAR:[fault reason 05] PTE Write access is not set
There's 1400+ entries in generally ascending order (sometimes it goes back or
stays on the same address). The last one is:
[ 683.093190] dmar: DRHD: handling fault status reg 3
[ 683.093192] dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr
fe579000
DMAR:[fault reason 05] PTE Write access is not set
I'm not sure if it's a direct correlation (hang xor dma errors)
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 1549 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9 regression] 3d application hang
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
` (4 preceding siblings ...)
2013-11-16 3:33 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-16 3:36 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 5:02 ` [Bug 71662] [nvd9] hang: nouveau ![ PFIFO][0000:01:00.0] unhandled status 0x00800000 bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-27 5:06 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-16 3:36 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 385 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
--- Comment #6 from Ilia Mirkin <imirkin-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org> ---
As for the DMAR errors, see if disabling the IOMMU (aka VT-d for intel) helps.
(It'll definitely get rid of the messages, but might not get rid of the
underlying issues.)
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 1107 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9] hang: nouveau ![ PFIFO][0000:01:00.0] unhandled status 0x00800000
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
` (5 preceding siblings ...)
2013-11-16 3:36 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-16 5:02 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-27 5:06 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-16 5:02 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 784 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
Jan Vesely <jano.vesely-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|[nvd9 regression] 3d |[nvd9] hang: nouveau ![
|application hang |PFIFO][0000:01:00.0]
| |unhandled status 0x00800000
--- Comment #7 from Jan Vesely <jano.vesely-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> ---
nvm, I just reproduced it on 3.11.6. It was a bit harder to hit but after cca
20 runs I got unhandled status 0x00800000. Not a regression.
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 2021 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Bug 71662] [nvd9] hang: nouveau ![ PFIFO][0000:01:00.0] unhandled status 0x00800000
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
` (6 preceding siblings ...)
2013-11-16 5:02 ` [Bug 71662] [nvd9] hang: nouveau ![ PFIFO][0000:01:00.0] unhandled status 0x00800000 bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
@ 2013-11-27 5:06 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
7 siblings, 0 replies; 9+ messages in thread
From: bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ @ 2013-11-27 5:06 UTC (permalink / raw)
To: nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
[-- Attachment #1.1: Type: text/plain, Size: 599 bytes --]
https://bugs.freedesktop.org/show_bug.cgi?id=71662
Ilia Mirkin <imirkin-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |DUPLICATE
--- Comment #8 from Ilia Mirkin <imirkin-FrUbXkNCsVf2fBVCVOL8/A@public.gmane.org> ---
*** This bug has been marked as a duplicate of bug 71659 ***
--
You are receiving this mail because:
You are the assignee for the bug.
[-- Attachment #1.2: Type: text/html, Size: 2274 bytes --]
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
Nouveau mailing list
Nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
http://lists.freedesktop.org/mailman/listinfo/nouveau
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-11-27 5:06 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-11-15 22:32 [Bug 71662] New: [nvd9 regression] 3d application hang bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
[not found] ` <bug-71662-8800-V0hAGp6uBxMKqLRl/0Ahz6D7qz1kEfGD2LY78lusg7I@public.gmane.org/>
2013-11-15 22:35 ` [Bug 71662] " bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-15 22:38 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:23 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:32 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:33 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 3:36 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-16 5:02 ` [Bug 71662] [nvd9] hang: nouveau ![ PFIFO][0000:01:00.0] unhandled status 0x00800000 bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
2013-11-27 5:06 ` bugzilla-daemon-CC+yJ3UmIYqDUpFQwHEjaQ
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.